Comments (5)
Yup, seems to be dead... :/
from bubbles.
😞
from bubbles.
Why don't Python nerds need to ETL?
Maybe they have custom solutions, like me. The most important thing is parallelism, jobs scheduling and execution, a possibility to track issues (inspecting a staging state step-by-step), scheduler state inspection and management, and so on. Abstraction layer for transforms/calculations aren't so important, because they can be done in raw sql. For everyting else, i.e matrix operations, a numpy/scipy packages are just enough. Just my 5cents.
from bubbles.
@marcinn what do you use for the more important points that you mentioned? Something like Luigi?
from bubbles.
I didn't know about Luigi when I was starting work with ETLs. I created something similar to this, but way limited - without dependency handling, parallelism only within a run() method, just sequence of tasks per job, simple jobs scheduler and runner (2in1, just 1 runner at once).
My ETL was a simple Python script written in one module, at the beginning. It was changing slowly into something bigger, and that's how my custom library and scheduler was created.
Luigi looks more powerful and generic. I think I'll start using it, because maintaining a custom solution isn't cheap and requires a lot of time. Thank you for pointing Luigi here.
from bubbles.
Related Issues (20)
- Pip install bubbles fails HOT 1
- Example code not working HOT 2
- Import in example code fail
- Error trying to convert from PostgreSQL to Sqlite HOT 1
- Support for JSON table schema
- import bubbles fails HOT 2
- Examples not working
- How about mongodb example?
- Link to project page http://bubbles.databrewery.org is broken. HOT 1
- This package still supported? HOT 2
- Unable to find operation source_object HOT 1
- Proposing a PR to fix a few small typos
- Consumable retention policy
- Retry in nested operation should not replace parent
- Date to/from string conversions should use SQL format
- join_details should use same column names if no keys are specified
- Composed operations have no way of dealing with consumables
- Field part reference for compound field types
- Simplify handling of tuple list (ordering, aggregations)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bubbles.