Comments (4)
Hi @glodus,
s3 can be locked pretty tightly so that it is only readable with an access_key and a secret_key and not to the rest of the world (You would have to define the appropriate security policy for the resource (bucket, or subdirectory)). This is the setup we use. Airflow is also equipped to handle IAM roles, if your setup permits.
Let me know if you have more questions about this!
from airflow.
I was more thinking about what I would put on the airflow server, about the "worklow", "pipeline" or dags. It could be very unfortunate that someone understood it.
Not really related, but still a question: Do you handle LDAP authentification or perhaps flask_login already handle it ?
from airflow.
From our perspective security is almost an all-or-nothing as far as executing pipelines. If you do have access to an Airflow box, you pretty much have full access to all the systems Airflow interfaces with. We spoke internally about using Airflow for more sensitive datasets that not all engineers should have access to, and the solution there would be to setup an alternate Airflow environment for that purpose.
The UI is mostly a read-only UI, and there are different level of access there. It's possible to create a limited access where people can just look at pipeline definitions and how they progress.
The flask_login backdoor should allow you to do pretty much anything in terms of authentication. Our setup has a reverse proxy that takes care of ssl and ldap authentication that squeezes http headers in the request, and our airflow_login
module just look for the headers to grant the right level of access. Using flask_login and ldap is probably fairly well documented. A temporary setup can be achieved with privileged users using an ssh tunnel.
from airflow.
Thanks.
from airflow.
Related Issues (20)
- Add/Edit Connection in REST API should reject non-JSON "extra" field HOT 7
- max_active_runs_per_dag is not respected HOT 1
- Adding max_consecutive_failed_dag_runs in UI for dag HOT 1
- Timezone information should be shown in Next Run details HOT 1
- DAGs scheduled on February 29th are not executed. HOT 4
- constraint "dag_run_dag_id_execution_date_key" not using run_id HOT 1
- Allow Pause on Task instance level HOT 1
- Mapping key "cluster-autoscaler.kubernetes.io/safe-to-evict" already defined HOT 8
- Cloud Composer (Google managed Airflow) sometimes does not retry tasks detected as zombies HOT 2
- Connection to Postgres failed due to special characters in a password HOT 4
- Show task_failed_deps on for scheduled tasks in Task Instance Details panel HOT 3
- Build correct SQLAlchemy URI in hooks based on `DbApiHook`
- Resolve mypy violations for `azure-cosmos` >= 4.6.0
- Calculate run duration with current time for running tasks in task and dag duration charts
- Getting dags from dagbag is failing HOT 1
- Triggering a inactive dag manually makes the dag active which leads to making the dag, and the subsequent run is also triggered at the next scheduled time HOT 3
- All jobs have transitioned from queue status to failed status. HOT 2
- Scheduler was dead when database service failover. HOT 3
- Airflow task fails when no permission to bash HOT 5
- Fix remaining `PT012` pytest checks: `pytest-raises-with-multiple-statements`
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from airflow.