Comments (9)
Anybody?
from bonobo.
For now, the recommended way to pass arguments at runtime is to use environment variables. It's universal, widely used, and ops-friendly.
There will be in the future some way to use arguments to set it, but I see it as a non priority knowing that env can do pretty much anything in this regard.
Do you have a particuliar use case where environment cannot do the job ?
INPUT_FILE=inventory.txt MAGIC_NUMBER=2773 bonobo run csvsanitizer
from bonobo.
Thanks for getting back to me!
Believe it or not I didn't know you could do that with bash. Unfortunately, my work environment is exclusively windows excepting our FTP and proxy servers themselves. I would need to be using one of our Windows Server 2012 production servers so I can't even get around it via the Linux subsystem for Windows (Ubuntu Bash on Windows).
Although I might just not be aware of how, I am not sure you can set env vars before the bonobo run command like you can in bash. I have seen suggestions for ways to get-around this issue on stack exchange though it looks to be pretty hacky and not ideal (https://superuser.com/questions/223104/setting-and-using-variable-within-same-command-line-in-windows-cmd-exe). Additionally, even setting them in separate steps wouldn't work well as, in my experience at least, unlike Linux environment variables they don't seem to be as easily set or changed, especially for individual shell instances.
I realize that docker is an option but it still seems like extra complexity for most of my use cases where I really just want to pass traditional kwargs. Additionally, Am I right in believing, especially after your answer, that you not implementing kwargs was a design decision, or do you just believe it unnecessary? It is your project so I completely understand if it is the former but I would love to do some work on it and submit a pull request if it is the latter and you are open to the idea.
Lastly, perhaps you know something about Windows which I don't and your suggestion would work, albeit with minor variations. Do you?
Once again, thanks for getting back to me!
from bonobo.
Ok, I'm not very familiar with windows (understand, I did not use a windows machine more than a couple times for the last 10 years), but I guess you have a very valid point, I tend to think bash is universal, but indeed it's not.
I'll think about an alternate solution, which will probably be an additional command line flag that sets an environment variable (for exemple, -e FOO=bar), so windows or any strange env will work. Also, support for a ".env" file will be present in the near future.
I created a draft "rfc" about that to summarize all said here and to draft the future features and documentation on this topic: https://github.com/python-bonobo/bonobo/wiki/RFC-Configuration-&-Environment
from bonobo.
Thanks Romain, I think that sounds great.
from bonobo.
If you want to submit a pull request in this direction like you suggested (using argparse, and using same syntax as docker does: https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e-env-env-file), I'll be very glad to integrate it in the next release.
from bonobo.
Romain,
Absolutely, I would be happy to work on that.
from bonobo.
Checklist:
- Open PR against python-bonobo/bonobo:develop
- Move
tests/util/get_passed_env.py
into examples - Documentation page about passing env. That can be a new
docs/guide/environment.rst
(or something else if you think of a better name) file, that will be linked from the main guide index. -
--env
argparse registration may want to go intoregister_generic_run_arguments
instead ofregister
, as environment will probably be usefull in more than justbonobo run
context
from bonobo.
Am I right in believing now that your merged my pull request into the develop branch you want me to pull down a new copy of the develop branch, go through the checklist you provided, and then submit a new pull request against the develop branch?
Additionally, I am fine with a new 'environment' doc page but when are you planning on fleshing-out http://docs.bonobo-project.org/en/master/reference/commands.html? I would be happy to do one or both depending on which you think is better. However, regarding the latter I would like to know what the -c cmd
is for, are you still planning on implementing it, if so, could you provided an example?
Thanks
from bonobo.
Related Issues (20)
- Documentation missing - Not clear how to check if there were any errors in transformation nodes HOT 1
- Allow packaging==20.3 HOT 6
- DEBUG cannot be enabled by env var, even though it is said to be possible HOT 3
- resolve start and end of named chain are identical as _input and _output HOT 4
- Filter Node error HOT 2
- in the absence of docs - need some real world, readable examples HOT 1
- How to configure logging in bonobo to always log to stdout
- incompatibility with pip==20.x
- bonobo_sqlalchemy.Select remove easy limit for oracle HOT 1
- Graceful termination hangs when filtering data
- AttributeError: Graph and get_cursor() HOT 1
- Documentation example broken HOT 3
- CsvReader does not accept files in another directory HOT 2
- CsvReader does not accept absolute path names in Windows HOT 7
- No output in docker container
- [FR] loop (sub-)chains HOT 1
- Example for using the RateLimited feature?
- processors.py need to be updated for Python 3.10+ for collections HOT 7
- Bug: Could not load 'init': cannot import name 'soft_unicode' from 'markupsafe' HOT 2
- Is the project being abondoned? HOT 10
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bonobo.