haixuantao / bert-onnx-rs-pipeline Goto Github PK
View Code? Open in Web Editor NEWThis demo showcase the use of onnxruntime-rs with a GPU on CUDA 11 to run Bert in a data pipeline with Rust.
This demo showcase the use of onnxruntime-rs with a GPU on CUDA 11 to run Bert in a data pipeline with Rust.
Running host: linux
Since I cannot download data from train_October_9_2012.csv(because unknown error), I recreated small version of it:
| X | PostId | PostCreationDate | OwnerUserId | OwnerCreationDate | ReputationAtPostCreation | OwnerUndeletedAnswerCountAtPostTime | Title | BodyMarkdown | Tag1 | Tag2 | Tag3 | Tag4 | Tag5 | PostClosedDate | OpenStatus | |
| --- | -------- | ------------------- | ----------- | ------------------- | ------------------------ | ----------------------------------- | ------------------------------------------------- | ------------------------------------------------- | ---------- | --------------- | ----- | ---- | ---- | ------------------- | ------------- | --- |
| 1 | 10035653 | 2012-04-05 20:37:35 | 1159226 | 2012-01-19 18:46:16 | 1 | 0 | what is the best way to connect my | I know this question can be answered by | c++ | NA | NA | NA | NA | 2012-04-05 23:31:34 | too localized | |
| 2 | 6046168 | 05/18/2011 14:14:05 | 543315 | 09/17/2010 10:15:06 | 1 | 2 | For Mongodb is it better to reference an objec... | I am building a corpus of indexed sentences in... | mongodb | NA | NA | NA | NA | NA | open | |
| 3 | 4873911 | 02/02/2011 11:30:10 | 465076 | 10/03/2010 09:30:58 | 192 | 24 | How to insert schemalocation in a xml document... | i create a xml document with JAXP and search a... | dom | xsd | jaxp | NA | NA | NA | open | |
| 4 | 3311559 | 07/22/2010 17:21:54 | 406143 | 07/22/2010 16:58:20 | 1 | 0 | Too many lookup tables | What are the adverse effects of having too man... | sql-server | database-design | enums | NA | NA | NA | open | |
| 5 | 9990413 | 04/03/2012 09:18:39 | 851755 | 07/19/2011 10:22:40 | 4 | 1 | What is this PHP code in VB.net | I am looking for the vb.net equivalent of this... | php | vb.net | NA | NA | NA | 04/15/2012 21:12:48 | too localized | |
| 6 | 10421966 | 05/02/2012 21:25:01 | 603588 | 02/04/2011 18:05:34 | 334 | 14 | Spring-Data mongodb querying multiple classes ... | With Spring-Data, you can use the @Document an... | mongodb | spring-data | NA | NA | NA | NA | open | |
| | | | | | | | | | | | | | | | | |
After training with this data I cannot inference using python nor rust
If I use python, following error occurs:
Traceback (most recent call last):
File "src/python_alternative.py", line 39, in <module>
df_tiny = df_tiny[
File "/home/user/.local/lib/python3.8/site-packages/pandas/core/frame.py", line 3464, in getitem
indexer = self.loc._get_listlike_indexer(key, axis=1)[1]
File "/home/user/.local/lib/python3.8/site-packages/pandas/core/indexing.py", line 1314, in _get_listlike_indexer
self._validate_read_indexer(keyarr, indexer, axis)
File "/home/user/.local/lib/python3.8/site-packages/pandas/core/indexing.py", line 1374, in _validate_read_indexer
raise KeyError(f"None of [{key}] are in the [{axis_name}]")
KeyError: "None of [Index(['Title', 'BodyMarkdown', 'OpenStatus'], dtype='object')] are in the [columns]"
If I use rust:
thread 'main' panicked at 'called Option::unwrap() on a None value', src/main.rs:97:55
I tried to install your solution locally, and came up with 2 errors:
pip install -r requirements.txt
ERROR: Invalid requirement: 'version https://git-lfs.github.com/spec/v1' (from line 1 of requirement.txt)
Hint: It looks like a path. File 'version https://git-lfs.github.com/spec/v1' does not exist.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.