Sync data
Copy data from Google Sheets, MySQL and hundreds of apps into your warehouse on a schedule: how syncs run, their statuses, and who manages them.
Some data can’t be queried where it lives: spreadsheets, databases Orcabase doesn’t connect to directly, and business apps. Sync data copies it into your warehouse and keeps the copy up to date on a schedule. Synced tables work like any other: in Explore, the SQL Editor, data models and the agent.
Built-in syncs and Fivetran
One page, two engines underneath. You pick the source; Orcabase picks the engine. Each source in the list is labeled Built-in or via Fivetran.
| Engine | Sources | Where the data lands |
|---|---|---|
| Built into Orcabase | Google Sheets and MySQL | Your DuckDB warehouse, one schema per source |
| Fivetran | Hundreds of apps and databases: Salesforce, HubSpot, Stripe, Google Ads and more | A hosted Postgres server, one schema per source |
When Orcabase has its own connector for something, the catalog offers that one and hides Fivetran’s: built-in MySQL covers MySQL, MariaDB and Aurora, and PostgreSQL and BigQuery are connected directly instead of synced.
Who can do what
Everyone in your organization can see the Sync data page and every source’s status. Adding, changing, running and removing sources is for admins, since syncs write into your warehouse (and Fivetran syncs count against a monthly allowance).
Add a source
Open the catalog
Go to Sync data then Add source. Built-in connectors are listed first, then databases you can query directly, then everything Fivetran offers.
Connect
Fill in the source’s details, pick the warehouse to sync into under Sync into, and click Connect. Orcabase checks the connection and lists the tables it found. See the page for each source: Google Sheets, MySQL, Fivetran.
Choose tables and a schedule
Tick the tables to sync. Tick Also sync tables added later to pick up new ones automatically. Then set a Name, the Schema the tables go in (lowercase letters, digits and underscores), and how often to Sync.
Start syncing
Click Start syncing. The first sync starts right away, and usually takes a couple of minutes.
How a built-in sync works
- A full copy every time. Each sync rebuilds every selected table from the source. The new copy swaps in all at once, so queries never see a half-loaded table.
- Failures are safe. If a sync fails, the previous copy stays in place. If one table fails (say, it was deleted at the source), the others still sync and the source shows Some tables failed.
- On your schedule. Every 15 minutes, 30 minutes, 1 hour, 3 hours, 6 hours (the default), 12 hours or once a day. Sync now runs one immediately.
- Passwords are encrypted, and never shown again after you save them.
Statuses
| Status | What it means |
|---|---|
| Waiting for first sync | The source is set up; its first sync hasn’t finished yet. |
| Syncing | A sync is running now. |
| Up to date | The last sync finished without errors. |
| Some tables failed | Most tables synced, but at least one didn’t. Open the source to see which, and why. |
| Failed | The last sync didn’t finish. The previous copy of the data is still there. |
| Needs reconnecting | Orcabase can’t sign in to the source any more. Update its connection details (or, for Fivetran, click Reconnect). |
| Finish setup | A Fivetran source whose sign-in wasn’t completed. Click Finish setup. |
| Paused | Someone paused it. Resume it from its menu. |
| Paused: limit reached | Your organization used this month’s Fivetran allowance. See Fivetran. |
Manage a source
Open a source to see when it Last synced, its Next sync, and its Recent syncs, each with the rows and tables it copied. From the source (or its ⋯ menu in the list), admins can:
- Sync now, or Pause and Resume it.
- Choose tables to add or remove tables, and change how often it syncs.
- Update its connection details. Orcabase tests them before saving.
- Remove it. Syncing stops, but the tables already synced stay in the warehouse; drop them with SQL if you don’t need them.
Finding synced tables
Synced tables live in the schema you chose, so a sheet synced into sheets shows up as sheets.orders. In Explore, open your warehouse, then that schema.
Something unclear or missing? Troubleshooting covers the common errors, or message us.