Comments (13)
from prefect-dbt.
from prefect-dbt.
To be assigned to Bela
from prefect-dbt.
from prefect-dbt.
Hi, we were wondering if there's anything we can do to help add prefect-dbt to the docs?
from prefect-dbt.
@belasobral93 can you give input here?
from prefect-dbt.
@ahuang11 Hi there, can you validate for accuracy? (The formatting needs to be cleaned up, but I will ask our Docs team for their guidelines). Additionally I also am looking for a higher res version of the dag (image below) - can you help me find a copy?
Prefect
If your organization is using Prefect, the way you will run your jobs depends on the version you are on, and whether you are orchestrating dbt cloud or core jobs. A variety of options are described below.
- If you are using Prefect 2.0
-
Invoking dbt cloud jobs
- Use the [trigger_dbt_cloud_job_run_and_wait_for_completion task](https://prefecthq.github.io/prefect-dbt/#trigger-a-dbt-cloud-job-and-wait-for-completion. As jobs are executing, you can poll dbt to see whether or not the job completes without failures, through the prefect UI.
-
Invoking dbt through the cli
- Use the trigger_dbt_cli_command task
For details on both of these methods, see prefect-dbt docs.
- If you are using Prefect 1.0
- Invoking dbt cloud jobs
- Trigger dbt Cloud jobs with the DbtCloudRunJob task. Running this task will generate a markdown artifact viewable in the Prefect UI. The artifact will contain links to the dbt artifacts generate as a result of the job run. - Invoking dbt through the cli
- Use the DbtShellTask to schedule, execute and monitor your dbt runs.
- Use the supported ShellTask to execute dbt commands through the shell.
from prefect-dbt.
@ahuang11 wanted to make sure you saw my ask ^
from prefect-dbt.
Thanks for the reminder! Also cc @desertaxle
I fixed some hyperlinks and cleaned up the formatting a bit. Also maybe you can use this image?
https://user-images.githubusercontent.com/12350579/186269036-01f1090f-1887-433d-8331-ee7c8b315dd5.png
I think our marketing team may also want to comment on how we want to publish this.
Prefect
If your organization is using Prefect, the way you will run your jobs depends on the version you are on, and whether you are orchestrating dbt Cloud or core jobs. A variety of options are described below.
On Prefect 2
dbt Cloud
Use the trigger_dbt_cloud_job_run_and_wait_for_completion flow. As jobs are executing, you can poll dbt to see whether or not the job completes without failures, through the prefect UI.
dbt Core
Use the trigger_dbt_cli_command task.
For details on both of these methods, see prefect-dbt docs.
On Prefect 1
dbt Cloud
Trigger dbt Cloud jobs with the DbtCloudRunJob task. Running this task will generate a markdown artifact viewable in the Prefect UI. The artifact will contain links to the dbt artifacts generate as a result of the job run.
dbt Core
Use the DbtShellTask to schedule, execute and monitor your dbt runs. Use the supported ShellTask to execute dbt commands through the shell.
from prefect-dbt.
@ahuang11 thank you! If you want to give the language and images a run by your marketing team, let me know and I can get the approval from our docs/partnership team. If not, I'll go ahead and submit this draft tomorrow.
from prefect-dbt.
@belasobral93 thanks; I think it can be submitted as a draft! Curious if dbt team is open to updating this in the future (like in 3-6 months) as we rebrand off the migration period (deprecating Prefect 1 and updating Prefect 2 to simply Prefect)?
from prefect-dbt.
Thank you!
from prefect-dbt.
from prefect-dbt.
Related Issues (20)
- Prefect dbt 0.2.6 coroutine error HOT 26
- Saving `DbtCliProfile` with `prefect_sqlalchemy.DatabaseCredentials` raises `TypeError: Type is not JSON serializable: URL` HOT 2
- Using private key in SnowflakeTargetConfigs
- GlobalConfigs' "log_format" should be of type String instead of Bool. HOT 3
- TargetConfigs' credentials gets dropped upon loading a DbtCliProfile
- Support SqlAlchemyConnector for PostgresTargetConfigs
- Issues with logging and dbt 1.4 (protobuf) HOT 8
- Credentials missing when using SnowflakeTargetConfigs in CLIProfile HOT 4
- Postgres port set as string in profiles.yml HOT 1
- Typo on dbt prefect documentation side
- add `trigger_dbt_cli_command` example to docs
- DBT improvements: dbt_build_task
- Potential path parsing issue with Windows HOT 3
- Integrations test for forked repos HOT 1
- block register command for creds block is incorrect HOT 1
- Execute a dbt CLI command HOT 2
- Block registration states 4 new block types created, but UI shows 6 new block types HOT 1
- Manage Compute engine credentials in addition of oauth2 HOT 13
- Change poll log level
- Show polling status updates selectively by comparing previous status vs current status
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from prefect-dbt.