abs-tudelft / fast-p2a Goto Github PK
View Code? Open in Web Editor NEWThis project forked from mythir/fast-p2a
An FPGA-based Parquet-to-Arrow converter
License: Apache License 2.0
This project forked from mythir/fast-p2a
An FPGA-based Parquet-to-Arrow converter
License: Apache License 2.0
This is a major issue especially when using multiple instances. We should consider removing the possibility of using this design altogether.
It has some more issues, such as needing to clear valid bits in BRAM after reset and each parquet page (it needs 512 cycles for that), and using much more BRAMs than conceptually needed (it uses a duplicate of the entire 64k data buffer instead of reconstructing data from a sliding window)
The delta designs have a limited data width because of timing difficulty in the parallel prefix adder. However, it should be possible to pipeline this. In the last cycle of the pipeline, the final value of the previous addition is known. So it can be added to all the partial sums. (see figure below or also fig. 17 here: http://users.encs.concordia.ca/~asim/COEN_6501/Lecture_Notes/L2_Notes.pdf)
.-. .-. .-. .-.
|1| |2| |3| |4| input
'-' '-' '-' '-'
| \ | | |
(r) (+) (r) (r)
| | \ | |
(r) (r) (+) (r)
| | | \ |
(r) (r) (r) (+)
| | | |
(+) (+) (+) (+) <- (ppp)
| | | |\ ^
| | | | \ | update ppp
| | | | '--'
v v v v
.-. .-. .-. .-.
|o| |o| |o| |o| output
'-' '-' '-' '-'
(r) = register
(+) = addition (and register)
(ppp) = previous partial prefix (initialize with the start value)
All the changes to support multiple instances have been done on the oc-accel platform. These still need to be propagated to the other platforms and the general testbenches.
A bit of circuitry has been added to AxiTop that allows 1 instance to be reset without taking down the whole AXI bus. But this only works for reads. It also stops instances from reading ahead much further after it finishes.
The reason this does not works for writes yet is that it is more complex (the circuit will need to keep track of an active bursts, so that it will generate the exact number of data beats that is expected due to the original write request), and because the design already waits for all the writes to finish before the done
flag is asserted (and the software normally waits for the done flag before resetting). However, if an instance was improperly configured or gets stuck for some other reason, we still want to be able to reset it (although note that you can always reset the entire design).
Right now, it is not possible to pass different generics down to the multiple instances. Maybe it is possible to convert the configuration string into an array of such config strings, so that each column decoder can get its own config.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.