Comments (14)
any update to this?
from vscode-data-preview.
hoping to look more into this in fall.
from vscode-data-preview.
awesome!
here are some parquet files that you can test (they are from a sample database)
https://gbpublic.blob.core.windows.net/data/Delta/AdventureWorksDW2012/DimAccount/part-00000-c80d6666-85ea-4b0e-97b2-2d424292b0c1-c000.snappy.parquet
https://gbpublic.blob.core.windows.net/data/Delta/AdventureWorksDW2012/DimDate/part-00000-667f9b57-ecaf-452b-880d-8e1addd46354-c000.snappy.parquet
https://gbpublic.blob.core.windows.net/data/Delta/AdventureWorksDW2012/FactProductInventory/part-00000-9e3e5c91-e2f2-4ee6-aec4-acfd675f9881-c000.snappy.parquet
https://gbpublic.blob.core.windows.net/data/Delta/AdventureWorksDW2012/DimProduct/part-00000-0cb4eaf5-02fd-46ab-8019-dc90d0a9d6ff-c000.snappy.parquet
https://gbpublic.blob.core.windows.net/data/Delta/AdventureWorksDW2012/FactInternetSales_part_big.delta/SalesTerritoryKey%3D4/part-00001-6d55c0d8-ee35-44ec-8d1a-9af7653522c6.c000.snappy.parquet (this one is bigger, 50 MB)
let me know if I can support you further!
regards,
-gerhard
from vscode-data-preview.
One small favour - could you add .pq extension for parquet as well?
from vscode-data-preview.
@RandomFractals I think only .parquet is "standard" but you allow .parq which I've never seen while .pq you can find used by different people on the internet. Possibly because this is a standard import in Python:
import pyarrow.parquet as pq
from vscode-data-preview.
Any time frame when this will be available?
Would like to refer to your extension when it comes to viewing parquet files
from vscode-data-preview.
I have not looked into it in a while. Last time I checked nodejs parquet lib had some issues and required native modules which I can't support in vscode ...
from vscode-data-preview.
Looks like there is a new package to work with parquet in TypeScript
https://github.com/kbajalc/parquets
Would this work?
from vscode-data-preview.
that might work. I'll take it for a spin this weekend.
Thanks for linking me to it.
from vscode-data-preview.
@gbrueckl good news. I tried parquets
lib you suggested and looks like we can load one of the sample parquet data files now.
I'll see if I can find more parquet data files online and if it works with more examples, I'll package and ship this soon.
from vscode-data-preview.
@gbrueckl thanks for the test files. I hope they get loaded b/c I tried some from https://github.com/apache/parquet-testing and did not have much luck with the data files from that repo due to encoding or shortcomings of the parquets and the underlying parquetjs lib we are using.
@JacekPliszka I'll map .pq
extension to parquet data provider. Is that a standard short file extension for parquet files? I have not seen it before.
Let me know if you can get your parquet files to load. This release is super alpha and I am sure there is much more we'll need to tweak to make it work with snappy compression, etc.
from vscode-data-preview.
btw, @gbrueckl I tried parquet files you shared. None of them load. I get page dictionary errors. will have to dig into that more later.
from vscode-data-preview.
Any updates on whether you will support parquet?
from vscode-data-preview.
@peder1001 most likely in the new Tabular Data Viewer extension I started working on this year, which does include Tabulator and Perspective viewer. You can try it with CSV data files from marketplace for now:
https://marketplace.visualstudio.com/items?itemName=RandomFractalsInc.tabular-data-viewer
from vscode-data-preview.
Related Issues (20)
- Incorrect type inferencing for parquet files HOT 1
- Initial load of parquet file is clunky HOT 1
- how to preview a parquet file? HOT 3
- avro preview intermittently succeeds HOT 4
- Update to latest finos/perspective? HOT 1
- Implement Shift Horizontal Scrolling in Data Grid view
- Numeric column data type (int/float) is determined by the first row value and rounded accordingly
- copy table to clipboard HOT 1
- Lists do not display in "Data Table" or "Data Summary" Presentation modes HOT 2
- 为啥临时文件用不了 HOT 1
- ints being interpreted as floats HOT 5
- `*.parquet` not a selectable filetype when opening files in windows
- data preview cannot be found in vscode marketplace HOT 2
- Update extension badges in docs HOT 1
- Can't load nyc taxi parquet files
- Incorrect Type inference CSV
- Grid view bug for two JSON named arrays
- [Feature] [.xlsx] select sheet to load from excel file
- Pandas Parquet not shown ("loading data" forever) HOT 2
- It is not clear how to save changes to file. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vscode-data-preview.