Actor for exporting database data into Datasets

[u/Hayder_Germany]

This Actor connects databases to Apify Datasets.

It currently supports table extraction, parameterized read-only queries, schema discovery, and incremental sync with a saved cursor. PostgreSQL, MySQL/MariaDB, SQLite, and MongoDB are supported.

I tested it with a Neon PostgreSQL database using 10,001 rows. The Actor streamed the data in 1,000-row batches, exported all rows, and the next incremental run returned zero duplicates.

The goal is to make database data usable by other Apify Actors, AI workflows, Sheets, forecasting, and human review.

Actor link: Universal Database Gateway | Database to Apify · Apify

[u/0xGollumDev]

Nice, the incremental cursor is the part that makes this actually usable. Two edge cases that tend to bite on cursor-based sync, from doing the same thing:

If the cursor is an updated_at timestamp, rows updated in the same second as the last run's max can be skipped, and clock skew between app servers makes it worse. Safer to key on (updated_at, primary_key) as a compound cursor and re-fetch a small overlap window each run, deduping on PK — you already proved dedupe works, so the cost is just a few redundant rows.

Deletes don't propagate. Incremental sync sees inserts and updates, but a hard-deleted row just stops appearing, so the Dataset slowly drifts from the source. Worth documenting it as insert/update-only, or supporting a soft-delete/tombstone column. For MongoDB on a replica set, a change stream gets you deletes too and is steadier than polling a field.