Models on the lake, predictions back as tables.

Commercial license

Feature engineering, model selection, training and prediction run directly on the Parquet in your bucket — and the output lands back in the lake, where SQL can reach it. 

dataspoc-lens ml --help
Read the docs Ask about licensing

What ML does:

It closes the loop between the data you already curated and the decisions you want to automate.

Point it at a curated table and a target column. ML audits the data, engineers features, selects and trains a model, and writes the artifacts to ml/models/ in the same bucket. Predictions go to ml/predictions/ as Parquet — which means Lens discovers them automatically and an analyst can join them to anything else without a handoff.

What you get

  • Automated features

    Feature engineering and selection derived from the curated table, not hand-built for one model.

  • Train and evaluate

    Model selection with metrics written next to the artifact, so the tradeoff is on record.

  • Explainability

    ml explain produces a report of what drives the prediction — for the team that has to defend it.

  • Predictions as data

    Output is Parquet in the bucket. Queryable in Lens, joinable in SQL, readable by an agent.

  • Drift monitoring

    Metrics tracked over time so a model degrading is something you find before the business does.

In practice

Train against a curated table, predict against a new one.

$ dataspoc-lens ml train \
    --table curated.finance.customers \
    --target churn \
    --output s3://my-lake/ml/models/churn
$ dataspoc-lens ml predict \
    --model s3://my-lake/ml/models/churn \
    --input curated.finance.new_customers

From curated table to production model.

Read the docs