Data Engineering Wiki

The best place to learn data engineering. Built and maintained by the data engineering community.

A collection of notes that are connected organically but loosely organized into the following categories:

1. Concepts: Concepts related to Data Engineering.
2. FAQ: Frequently asked questions about Data Engineering.
3. Guides: Understand how to make Data Engineering decisions.
4. Tools: Commonly used tools for Data Engineering.
5. Tutorials: Step-by-step instructions for Data Engineering tasks.
6. Learning Resources: Learn Data Engineering with resources recommended by the Data Engineering community.

The Data Engineering Wiki is an CC0-1.0-licensed open source project with its ongoing development made possible entirely by the support of these awesome backers. If you'd like to join them, please consider sponsoring the Data Engineering Wiki's development.

Gold Sponsors

Silver Sponsors

The wiki can be used offline and can be used as-is or incorporated into your own personal knowledge management system. It is built to be used with Obsidian (free, no affiliation) but is compatible with other tools as well such as Foam or Roam Research.

1. Download this GitHub repository.
2. Download the free Obsidian desktop app.
3. Run the Obsidian app and choose Open folder as vault, click Open.
4. In the file browser, choose the folder where you downloaded the GitHub repository, click Open.

See Obsidian help for questions on using Obsidian.

There are many different ways to contribute to the wiki's development. If you're interested, check out our contributing guidelines to learn how you can get involved.

Thank you to all of our contributors who shared their data engineering knowledge!