OpenDAL is a data access layer that allows users to easily and efficiently retrieve data from various storage services in a unified way.
Major components of the project include:
Libraries
- Rust Core
- C Binding not released
- Cpp Binding not released
- Haskell Binding not released
- Java Binding
- Lua Binding not released
- Node.js Binding
- Python Binding
- Ruby Binding not released
- Swift Binding not released
- Zig Binding not released
Applications
Services
Standard Storage Protocols (like ftp, webdav)
Object Storage Services (like s3, gcs, azblob)
- azblob: Azure Storage Blob services
- cos: Tencent Cloud Object Storage services
- gcs: Google Cloud Storage Service
- obs: Huawei Cloud Object Storage Service (OBS)
- oss: Aliyun Object Storage Service (OSS)
- s3: AWS S3 alike services
- supabase: Supabase Storage Service being worked on
- wasabi: Wasabi Cloud Storage
File Storage Services (like fs, azdls, hdfs)
- fs: POSIX alike file system
- azdls: Azure Data Lake Storage Gen2 services (As known as ABFS)
- hdfs: Hadoop Distributed File System(HDFS)
- ipfs: InterPlanetary File System HTTP Gateway
- ipmfs: InterPlanetary File System MFS API being worked on
- webhdfs: WebHDFS Service
Consumer Cloud Storage Service (like gdrive, onedrive)
- gdrive: Google Drive being worked on
- onedrive: OneDrive being worked on
Key-Value Storage Service (like rocksdb, sled)
Cache Storage Service (like memcached, moka)
- ghac: GitHub Action Cache Service
- memcached: Memcached service
- mini_moka: Mini Moka backend
- moka: Moka backend
- vercel_artifacts: Vercel Remote Caching Service being worked on
Welcome to add any services that are not currently supported here.
The examples are available at here.
The documentation is available at https://opendal.apache.org.
OpenDAL is an active open-source project. We are always open to people who want to use it or contribute to it. Here are some ways to go.
- Start with Contributing Guide.
- Submit Issues for bug report or feature requests.
- Discuss at dev mailing list (subscribe / unsubscribe / archives)
- Asking questions in the Discussions.
- Talk to community directly at Discord.
- Databend: A modern Elasticity and Performance cloud data warehouse.
- GreptimeDB: An open-source, cloud-native, distributed time-series database.
- deepeth/mars: The powerful analysis platform to explore and visualize data from blockchain.
- mozilla/sccache:
sccacheisccachewith cloud storage - RisingWave: A Distributed SQL Database for Stream Processing
- Vector: A high-performance observability data pipeline.
- OctoBase: the open-source database behind AFFiNE, local-first, yet collaborative.
- Pants: A fast, scalable, user-friendly build system for codebases of all sizes.
- QuestDB: An open-source time-series database for high throughput ingestion and fast SQL queries with operational simplicity.
- Milvus: A cloud-native vector database, storage for next generation AI applications
- QuestDB: An open-source time-series database for high throughput ingestion and fast SQL queries with operational simplicity.
Licensed under the Apache License, Version 2.0
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent