Open source
Airbyte
Open source data integration with 700+ connectors for pipelines
Data PipelinesOpen source data integration
Best tools for Data engineers
48 tools in our directory are marked as best for Data engineers, from 8 sections: Data & Analytics (21), Databases (18), Monitoring & Observability (3), Cloud Platforms (2) and 4 more sections.
Tools
Curated
Aiven
Managed open source databases, streaming and analytics on any cloud
Cloud DatabasesManaged open source data
Open source
Apache Airflow
Python tool for authoring, scheduling and monitoring workflows
Data PipelinesWorkflow orchestration
Open source
Apache DevLake
Open source tool that builds engineering metrics from DevOps data
Code InsightsOpen source team metrics
Open source
Apache Doris
Open source real-time analytics database with SQL and search
Analytics DatabasesReal time analytical database
Open source
Apache Druid
Real-time analytics database for fast queries on streaming data
Analytics DatabasesReal time analytics database
Open source
Apache Zeppelin
Open source web notebook for SQL, Python, Scala and Spark analysis
Notebooks and SheetsWeb based data notebook
Curated
Backblaze B2
S3-compatible object storage for data-heavy apps and AI workloads
Object StorageLow cost object storage
Open source
ClickHouse
Open-source column database for fast analytics with SQL
Analytics DatabasesFast columnar analytics
Open source
Dagster
Platform for building, scheduling and monitoring data pipelines
Data PipelinesData orchestration platform
Popular
Databricks
Data and AI platform for lakehouse, ETL, analytics and agents
Analytics DatabasesData and AI lakehouse platform
Freemium
dbdiagram.io
Draw database ER diagrams by writing simple DBML code
Data ModelingDraw ER diagrams with code
Open source
Debezium
Open source change data capture for streaming database changes
Migration and BackupChange data capture platform
Open source
Dgraph
Open source distributed graph database for real-time workloads
NoSQL DatabasesDistributed graph database
Open source
dlt
Open source Python library for building data loading pipelines
Data PipelinesPython library for pipelines
Open source
DuckDB
In-process analytical SQL database for fast local data analysis
RelationalFast in-process analytics DB
Free trial
Estuary
Real time and batch data pipelines for warehouses, lakes and apps
Data PipelinesReal time data movement
Open source
Freemium
Fivetran
Managed connectors that move data into a warehouse or lake
Data PipelinesManaged ELT connectors
Freemium
Google BigQuery
Serverless data warehouse for running SQL on large datasets
Analytics DatabasesServerless data warehouse
Open source
Great Expectations
Open source framework for testing and documenting data quality
Data QualityData quality testing
Freemium
Open source
InfluxDB
Time series database for high-resolution sensor and telemetry data
Metrics and TracingTime series database platform
Open source
Instructor
Library for getting structured, validated data from LLM output
Agent FrameworksStructured outputs from LLMs
Open source
Jitsu
Open source event collection that sends clean data to your warehouse
Data PipelinesOpen source event pipeline
Open source
KEDA
Event-driven autoscaler that scales Kubernetes workloads, even to zero
KubernetesEvent driven autoscaling
Open source
Kestra
Open source orchestrator where workflows are written in YAML
Data PipelinesDeclarative orchestration
Open source
Logstash
Open source server-side pipeline that collects and transforms logs
Logs and SearchServer side data pipeline
Free trial
Materialize
SQL streaming database that keeps query results up to date
Analytics DatabasesStreaming SQL database
Open source
Meltano
Open source ELT platform that runs data pipelines as code
Data PipelinesOpen source ELT platform
Freemium
Memgraph
In-memory graph database for connected data and AI context
NoSQL DatabasesIn memory graph database
Popular
MinIO
S3-compatible object storage for data, AI training and agents
Object StorageS3 compatible object store
Curated
Monte Carlo
Data observability platform for detecting and fixing pipeline issues
Data QualityData observability platform
Open source
OpenMetadata
Open source metadata platform for cataloging, quality and lineage
Data QualityOpen metadata platform
Open source
QuestDB
Open source time-series database with fast SQL and live data
Analytics DatabasesFast SQL time series database
Open source
Quickwit
Open source log and trace search that runs on cloud object storage
Logs and SearchCloud native log search
Open source
RisingWave
Streaming database for real-time SQL analytics and serving
Analytics DatabasesStreaming SQL database
Free trial
RudderStack
Customer data platform that collects events and routes them to tools
Data PipelinesWarehouse first CDP
Curated
ScyllaDB
Low-latency NoSQL database for high-throughput workloads
NoSQL DatabasesFast Cassandra alternative
Free trial
Snowflake
Cloud data platform for analytics, data engineering and AI
Analytics DatabasesCloud data platform
Open source
Snowplow
Collects behavioral event data from websites and apps in real time
Data PipelinesBehavioral data collection
Curated
Soda
Data quality monitoring with checks written as code or in a UI
Data QualityData quality monitoring
Open source
StarRocks
Open source SQL engine for real-time analytics at scale
Analytics DatabasesFast analytical SQL engine
Free trial
Freemium
Tinybird
Managed ClickHouse with SQL APIs for real-time analytics
Data WarehouseReal time analytics APIs
Open source
Trino
Open source SQL engine that queries many data sources in one go
Analytics DatabasesFast distributed SQL engine
Freemium
Twilio Segment
Customer data platform that collects events and sends them to tools
Data PipelinesCustomer data platform
Open source
visx
Low-level React visualization primitives from Airbnb
Charts and DataLow level React visualization
Looking for something else? Browse the full directory.