Ingestion that stops being a project.

Open source · Apache 2.0

Any of 400+ sources to Parquet in your bucket, one command at a time. No DAGs, no scheduler to operate, no YAML to debug. 

pip install dataspoc-pipe
Read the docs Source on GitHub

What Pipe does:

It moves data from where it is to where you can query it — and then gets out of the way.

Pipe wraps the Singer ecosystem in a single CLI. You name a source, it discovers the schema, extracts incrementally from a stored bookmark, and writes partitioned Parquet into the bucket you already own. The catalog it maintains is the same one Lens reads, so a table becomes queryable the moment it lands.

What you get

  • 400+ sources

    The Singer tap ecosystem — Postgres, MySQL, Stripe, HubSpot, Salesforce, Google Sheets and the long tail behind them.

  • Incremental by default

    Bookmarks are stored in the bucket. A rerun resumes where the last one stopped instead of replaying history.

  • Curated transforms

    Clean and reshape on the way in, so the curated layer is ready for analysts rather than raw dumps.

  • MCP server

    dataspoc-pipe mcp lets an agent add a source, trigger a run and read the logs without a wrapper API.

  • Any cloud

    S3, GCS or Azure Blob. Storage you already pay for, under IAM you already control.

In practice

Install, add a source, run it. The bookmark handles the rest.

$ pip install dataspoc-pipe
$ dataspoc-pipe add tap-postgres
  Source added. Configure in dataspoc-pipe.yaml
$ dataspoc-pipe run tap-postgres
  Resuming from bookmark: 2026-04-14T08:00:00Z
  Extracted 1,247 new records (3 tables)

Source to bucket, in one command.

Read the docs