Giter VIP home page Giter VIP logo

datafusion-wasm-bindings's Introduction

datafusion-wasm-bindings's People

Contributors

waynexia avatar alamb avatar

Stargazers

Dejan Simic avatar Jon Mease avatar Trent Hauck avatar avatar Xuanwo avatar Tim O'Guin avatar

Watchers

Daniël Heres avatar avatar Jiayu Liu avatar QP Hou avatar Yijie Shen avatar Wakahisa avatar avatar

Forkers

tshauck

datafusion-wasm-bindings's Issues

Question about OpenDAL path dependence

Hi, @waynexia, this project is so cool! I'm so happy that opendal's work on object_store integration and wasm target paid back.

However, I noticed that opendal is path dependence:

opendal = { version = "0.45.0", path = "../opendal/core", default-features = false, features = [
    "services-s3",
    "services-http",
] }
object_store_opendal = { path = "../opendal/integrations/object_store", default-features = false }

Would you like to list the changes you made in opendal? I'm happy to get them upstream and released.

(Maybe @WenyXu and @tisonkun also interested in this issue?)

Can't fetch http query set

Originally posted by @alamb: waynexia/datafusion-wasm#1

Thanks @waynexia this is amazing 🙏 I found it from apache/datafusion#9834

I figured I would report an error I encountered in case that would help

I tried to setup https://waynexia.github.io/datafusion-playground/ to read from the clickbench data

https://github.com/ClickHouse/ClickBench/

However, when I ran this query

create external table hits stored as parquet location 'https://datasets.clickhouse.com/hits_compatible/hits.parquet';
datafusion error: Object Store error: Generic Unexpected error: Unexpected (temporary) at stat, context: { url: https://datasets.clickhouse.com:443/hits_compatible/hits.parquet, called: http_util::Client::send, service: http, path: hits_compatible/hits.parquet } => send http request, source: error sending request: JsValue(TypeError: Failed to fetch
TypeError: Failed to fetch
    at Dy (https://waynexia.github.io/datafusion-playground/assets/index-gHAd8CE7.js:9:2251)
    at https://waynexia.github.io/datafusion-playground/assets/datafusion_wasm_bg-X0TXhe2e.wasm:wasm-function[14334]:0x1883492
    at https://waynexia.github.io/datafusion-playground/assets/datafusion_wasm_bg-X0TXhe2e.wasm:wasm-function[199]:0x5388ce
    at https://waynexia.github.io/datafusion-playground/assets/datafusion_wasm_bg-X0TXhe2e.wasm:wasm-function[629]:0x95fbcf
    at https://waynexia.github.io/datafusion-playground/assets/datafusion_wasm_bg-X0TXhe2e.wasm:wasm-function[7872]:0x15833d4
    at https://waynexia.github.io/datafusion-playground/assets/datafusion_wasm_bg-X0TXhe2e.wasm:wasm-function[12086]:0x17c1ac2
    at https://waynexia.github.io/datafusion-playground/assets/datafusion_wasm_bg-X0TXhe2e.wasm:wasm-function[11177]:0x176f631
    at https://waynexia.github.io/datafusion-playground/assets/datafusion_wasm_bg-X0TXhe2e.wasm:wasm-function[10984]:0x175ba20
    at https://waynexia.github.io/datafusion-playground/assets/datafusion_wasm_bg-X0TXhe2e.wasm:wasm-function[580]:0x91e713
    at https://waynexia.github.io/datafusion-playground/assets/datafusion_wasm_bg-X0TXhe2e.wasm:wasm-function[11483]:0x178bd1a)

Screenshot 2024-04-01 at 3 50 10 PM

Short-term plan for this binding

Task list for the first milestone.

The first milestone should:

  • Be generally available for common SQL queries.
  • Have a fundamental docs API and can be released to npm.js automatically
  • Have the basic config API for essential configs.
  • Define a (unstable) data API (maybe on top of https://arrow.apache.org/docs/js/ ?)

Tasks:

  • Expose configuration APIs (SessionContext)
  • Define and expose RecordBatch
  • Support GCS and Azure Bolb
  • Support more kinds of queries (fix them in the "upstream" datafusion)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.