Comments (4)
Hi @wjliu,
Can you maybe try calling this free function to see if it helps (you might need to wrap result_list
in a WeldValue
object first though)?
Let me know if this doesn't fix your problem and I will take a closer look.
from weld.
Hi @deepakn94 ,
I can't use WeldValue to wrap result_list, because it needs a void pointer of C type. How should I use WeldValue ? I just started learning Python, please give me your advice, thanks.
from weld.
Hmm, I might need to do some work here to get this to work cleanly. (basically we want the WeldValue
returned by the Weld program to be stored somewhere in Python so that we can easily free it when required)
I will keep you posted, @wjliu.
from weld.
reproduced with this scripts:
examples/python/grizzly/data_cleaning_grizzly.py
#!/usr/bin/python
# The usual preamble
import pandas as pd
import grizzly.grizzly as gr
import time
# Get data (NYC 311 service request dataset) and start cleanup
na_values = ['NO CLUE', 'N/A', '0']
raw_requests = pd.read_csv('data/311-service-requests.csv',
na_values=na_values, dtype={'Incident Zip': str})
print "Done reading input file..."
for i in range(500):
requests = gr.DataFrameWeld(raw_requests)
start = time.time()
# Fix requests with extra digits
requests['Incident Zip'] = requests['Incident Zip'].str.slice(0, 5)
# Fix requests with 00000 zipcodes
zero_zips = requests['Incident Zip'] == '00000'
requests['Incident Zip'][zero_zips] = "nan"
# Display unique incident zips again (this time cleaned)
print requests['Incident Zip'].unique().evaluate()
end = time.time()
print "Total end-to-end time, including compilation: %.2f" % (end - start)
from weld.
Related Issues (20)
- License file in python sdists
- Weld as a Foundation for ML in Rust
- unable to import grizzly.grizzly HOT 1
- Is there any benchmark between pandas and grizzly?
- Incorrect website grizzly install command HOT 3
- "simplify_assignments" creates infinite loop with iterate
- Getting compile error: Invalid input token: 'name' HOT 3
- Build error while on Fedora HOT 2
- Error "ModuleNotFoundError: No module named 'weldarray'" when using python3. HOT 1
- Is weld still under research/development? HOT 4
- Standardizing Weld's IR code HOT 1
- An active weld community
- An IR which returns a wrong answer.
- Cannot build on aarch64 (Apple Silicon Chips)
- example udfs run error
- Using REPL for weld code using CUDFs
- Running Python UDFs in Weld.
- unique() function on weld-capi.
- movielens_grizzly.py code not working.
- Bump framework versions in Weld probe core
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from weld.