dlt

dlt is an open source Python library for loading data from APIs, databases and files into a warehouse. It lets developers write pipelines as ordinary Python code instead of configuring a separate tool.

You write a source function that yields records, then pass it to dlt with a destination. The library infers the table structure, handles schema changes and loads the data in batches. Pipelines can run in a notebook, in a script or on your own infrastructure, and they can be scheduled with the tools you already use.

• Load from APIs, databases and files using Python
• Automatic schema inference and handling
• Runs in notebooks, scripts or your own servers
• Loads into common cloud warehouses
• A hosted platform, dltHub, is offered on top of the library

Python developers and data engineers who want to build custom ingestion for sources that no-code tools do not cover. It suits teams that prefer code review, tests and version control for every pipeline.

The library is open source under the Apache 2.0 licence and is free to use. Running it yourself means you handle scheduling, secrets and monitoring. The hosted dltHub platform adds deployment and observability on top of the library. Code-first pipelines need Python skills, so they suit developers more than business users.