Giter VIP home page Giter VIP logo

awesome-duckdb's Introduction

Awesome DuckDB Awesome

A curated list of awesome DuckDB libraries, tools and resources.

DuckDB is an in-process SQL OLAP database management system.

Contents

Resources

Client APIs

Tools Powered by DuckDB

  • Rill Developer - Tool for effortlessly transforming data sets into powerful, opinionated dashboards using SQL.
  • Ibis Project - A DataFrame API for interacting with DuckDB (and other compute engines).
  • MotherDuck - Serverless data warehouse powered by DuckDB.
  • Boiling Data - Serverless data analytics overlay on top of S3 Data Lakes.
  • Hex Dataframe SQL - Hex's Dataframe SQL cells are powered by DuckDB.
  • Mode - Mode uses DuckDB for their in-memory data engine.
  • VulcanSQL - DuckDB can be used as a caching layer or a data connector in VulcanSQL, a Data API framework for data folks to create REST APIs by writing SQL templates.
  • Tad - A fast, free, cross-platform tabular data viewer application powered by DuckDB.
  • Honeycomb Maps - A browser-based geospatial analysis tool leveraging DuckDB Wasm.
  • Bauplan - A serverless data transformation platform for data lakes.
  • Excalichart.com - A fast, free dashboard for exploring your data.
  • Malloy - Malloy is an experimental language for describing data relationships and transformations. Malloy connects to BigQuery and Postgres, and natively supports DuckDB.
  • Evidence - Generate reports using SQL and markdown. The DuckDB connector allows querying across DuckDB, csv, parquet and json.
  • Latitude - Latitude uses DuckDB to power data snapshots. Drop a CSV file and query it with SQL at the speed of light.
  • Census - Census's dataset diffing for incremental syncs is powered by DuckDB.
  • Huey - Blazing-fast & intuitive pivot tables on .parquet, .csv, .json files and .duckdb tables in the browser based on DuckDB WASM. Open source (MIT). Zero install!
  • Parquet Explorer - Visual Studio Code extension for exploring Parquet files with SQL, powered by DuckDB.
  • Iceburst - The real-time data lake for monitoring & security.
  • DQOps - Data quality platform for data engineers, data quality teams and data operations.

Web Clients

  • Online DuckDB Shell - Online DuckDB shell powered by WebAssembly.
  • SQL Workbench - DuckDB-WASM based SQL Workbench for running queries on local or remote data, being able to show data as tables or visually as graphs, and sharing queries via URLs.
  • Sekuel Playground - Query your local parquet, csv, json. Your data will not be sent out of the device you are using.
  • CSVFiddle - Free tool to explore and share insights from CSV files using SQL. Import data, write SQL, then instantly share it with anyone.
  • Codapi - Embed executable code snippets directly into your product documentation, online course or blog post.
  • QuackDB - Open-source online DuckDB SQL playground and editor.

Libraries Powered by DuckDB

  • Mosaic - An extensible framework for linking databases and interactive views.
  • Fugue - A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.
  • Splink - A free Python library for fast, accurate data deduplication and record linkage.
  • Simple-data-analysis - Easy-to-use and high-performance JavaScript library for data analysis.
  • duckdb_fdw - DuckDB Foreign Data Wrapper for PostgreSQL.
  • @jetblack/duckdb-react - A context manager for React and duckdb-wasm.
  • QuackOSM - A Python library for downloading and transforming raw OpenStreetMap data into GeoParquet files.

SQL Clients and IDE that Support DuckDB

  • Harlequin - The DuckDB IDE for your terminal. (GitHub).
  • qStudio - A free SQL tool specialized for data analysts. It runs on every operating system and allows easy browsing of tables and charting of results.
  • DuckDB SQL Tools - Free DuckDB SQL Tools for VS Code IDE. Premium version available with advanced features.
  • VSCode SQLTools (Free) - Free open-source VSCode extension to query and explore your DuckDB databases with latest DuckDB support.
  • DBeaver - DBeaver is a universal database access and development tool that can be used to connect almost any type of database.
  • DataGrip - Paid SQL IDE by Jetbrains that supports many different database technologies, including DuckDB.
  • Duckling - A fast viewer for CSV/Parquet files and DuckDB/SQLite, based on Tauri.

Projects Powered by DuckDB

  • NBA Monte Carlo - Monte Carlo simulation of the NBA season, leveraging Meltano, dbt, DuckDB and Evidence.
  • Datadex - Open source and local friendly data platform to collaborate on Open Data using DuckDB, Dagster, dbt, and Quarto.
  • endoflife.date database

Integrations

Extensions

  • Official Extensions - Official DuckDB extensions.
    • arrow - A zero-copy data integration between Apache Arrow and DuckDB.
    • duckdb_iceberg - For reading Iceberg tables.
    • duckdb_aws - For handling AWS credentials.
    • duckdb_azure - For using the Azure Blob storage.
    • mysql - To read from and write to MySQL databases.
    • postgres - To read from and write to PostgreSQL databases.
    • spatial - Enables geospatial processing.
    • sqlite - To read from and write to SQLite databases.
  • DuckDB Extension Radar - Repository that contains DuckDB extensions on GitHub. Refreshed daily.
  • Deltatable - DuckDB extension for reading Delta tables.
  • Lance - Integrate Lance (modern columnar data format for ML implemented in Rust) with DuckDB.
  • PRQL - Run PRQL commands directly within DuckDB.
  • duckdb-engine - SQLAlchemy driver for DuckDB.
  • Scrooge-McDuck - DuckDB extension to support a set of aggregation functions and data scanners on financial data.
  • duckdb-pytables - DuckDB extension to allow running SQL on arbitrary data sources.
  • ODBC Scanner DuckDB Extension - DuckDB extension to read data directly from databases supporting the ODBC interface.
  • duckdb-jfr-extension - DuckDB extension to read JFR (Java Flight Recorder) files directly.
  • QDuckDB - Plugin for reading DuckDB spatial tables in QGIS software.
  • ERPL - DuckDB SAP connector using RFC, ODP, or BICS.
  • duckdb-extension-template-zig - A Zig & Nix toolkit template for building extensions against multiple versions of DuckDB using Zig, C or C++.
  • DuckPGQ - DuckDB extension for graph workloads that supports the SQL/PGQ standard.

Media

Talks

Podcasts

Blog Posts

Contribute

Contributions welcome! Read the contribution guidelines first.

awesome-duckdb's People

Contributors

davidgasquez avatar szarnyasg avatar anna-geller avatar edmundmiller avatar jacopotagliabue avatar adriens avatar tobilg avatar warrenbhw avatar randomfractals avatar ygulas avatar mehd-io avatar sipemu avatar archiewood avatar ocadaruma avatar pdet avatar robinl avatar rpbouman avatar alitrack avatar tconbeer avatar tydunn avatar tylerhillery avatar vojay-dev avatar dtenwolde avatar l1xnan avatar prasburst avatar tc-cole avatar ak-coram avatar nickcrews avatar nshiab avatar mihaibojin avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.