About Dagster
Dagster is a platform for building, scheduling and monitoring data pipelines. It organizes work around the data assets a team produces, so people can see what each asset depends on and whether it is fresh.
What it does
You describe pipelines in Python as software defined assets, and Dagster runs them on a schedule or when upstream data changes. The interface shows lineage, dependencies and freshness for each asset, and failed runs can be inspected and re-run. The aim is to give engineers and data consumers one shared view of how a data platform behaves.
Key features
- Pipelines defined in Python as software defined assets
- Lineage and dependency views across tools
- Schedules and sensors that trigger runs
- Freshness checks and failure monitoring
- Reusable components and shared standards for teams
Who it's for
Data platform teams and engineers who manage many pipelines across several tools. It is a good fit when several teams share one platform and need common standards for how work is built and monitored.
Pricing and good to know
The core project is open source under the Apache 2.0 licence and can be self-hosted. Hosted options are offered by the company behind it. The project's site notes that it is now part of Prefect, so check the current ownership and support terms before you adopt it for a long-term platform.
Highlights
- Define pipelines in Python as data assets
- View lineage and freshness for each asset
- Run jobs on schedules or when upstream data changes
- Inspect failed runs and re-run them from the interface
- Reuse shared components across teams
