Comments (10)
@Aylr you have brought to an end WEEKS of frustration. THANK YOU so much for suggesting this. I hadn't played with github actions enough to realize that I could just grab the library and run it that way. I've modified your suggestion slightly to fix my needs (we just need to run one or two checks against files that are modified in a given branch) and we're off and running!
on:
pull_request:
branches: [master]
jobs:
PreCommitChecks:
name: Run pre-commit checks
runs-on: ubuntu-latest
steps:
- name: checkout
uses: actions/checkout@v3
- uses: actions/setup-python@v3
- id: file_changes
uses: trilom/[email protected]
with:
output: ' '
- name: Install pre-commit
run: pip install pre-commit
- name: Run pre-commit model checks
run: pre-commit run --files ${{ steps.file_changes.outputs.files}}
from dbt-checkpoint.
This is a year old, is this going to be fixed?
from dbt-checkpoint.
@KristoferNorwood for viz
from dbt-checkpoint.
Requesting suggestions here on above, since I have tried a multitude of things to handle this, including $ARGS in environment variables, checking out the work and comments in PR 48, and hacking my own fork to get around this, but it seems that the github action is escaping the args line as indicated both in @simon-tarr 's comment and in the original readme. By escaping, I mean that it is passing the entire args line as one command (not as a command + flag + argument). The result is that it doesn't recognize what is passed as a command. As an example, even if I pass nothing, it defaults to the "run --all-files" in the actions yaml, and returns this:
usage: pre-commit [-h] [-V]
{autoupdate,clean,hook-impl,gc,init-templatedir,install,install-hooks,migrate-config,run,sample-config,try-repo,uninstall,help}
...
pre-commit: error: argument command: invalid choice: 'run --all-files' (choose from 'autoupdate', 'clean', 'hook-impl', 'gc', 'init-templatedir', 'install', 'install-hooks', 'migrate-config', 'run', 'sample-config', 'try-repo', 'uninstall', 'help')
Notice the line invalid choice: 'run --all-files'
- it's not seeing it as run
with an added flag. @tomsej have you seen this in your development?
from dbt-checkpoint.
I have encountered the same issue as @grace-cityblock with:
pre-commit: error: argument command: invalid choice: 'run --files <FILE PATHS>'
While the solution by @Aylr circumvents the issue, it is frustrating as a new user that this still isn't fixed or documented as a potential problem.
from dbt-checkpoint.
Is there a workaround to this bug while a fix is implemented? Thanks!
EDIT - Turns out there is although I'm not sure this is a good work around for everyone. I changed the version of pre-commit-dbt from @v1.0.0 to the latest (as of 23rd Feb 2022) commit. So my main.yml file now looks like:
- name: Build Docs
uses: offbi/pre-commit-dbt@cc144b9a1a24112428d9a52a66905815990e4554
env:
DB_PASSWORD: ${{ secrets.DB_PASSWORD }}
with:
args: run --files ${{ steps.file_changes.outputs.files}}
from dbt-checkpoint.
Is there a workaround to this bug while a fix is implemented? Thanks! EDIT - Turns out there is although I'm not sure this is a good work around for everyone. I changed the version of pre-commit-dbt from @v1.0.0 to the latest (as of 23rd Feb 2022) commit. So my main.yml file now looks like:
- name: Build Docs uses: offbi/pre-commit-dbt@cc144b9a1a24112428d9a52a66905815990e4554 env: DB_PASSWORD: ${{ secrets.DB_PASSWORD }} with: args: run --files ${{ steps.file_changes.outputs.files}}
Thank you for this @simon-tarr! This got me past the reference format issue. I'm still trying to get around passing arguments but that helped a lot.
from dbt-checkpoint.
@grace-cityblock I've found a workaround you might find useful. I'm not using the dbt-pre-commit GitHub action at all, which doesn't bother me too much since it appears to be a thin wrapper anyway:
Note I'm using the excellent dbt-action pinned to the latest commit on main that fixes a bug that isn't released yet.
jobs:
PreCommitChecks:
name: Run pre-commit checks
runs-on: ubuntu-latest
steps:
- name: checkout
uses: actions/checkout@v2
- name: Install dbt dependencies
uses: mwhitaker/dbt-action@56f1f21c8026df783c17f65743b17855b5be22d9
with:
dbt_command: "dbt deps --profiles-dir . --target=dev"
- name: Build dbt docs # the dbt pre-commit checks use assets compiled by dbt docs generate
uses: mwhitaker/dbt-action@56f1f21c8026df783c17f65743b17855b5be22d9
with:
dbt_command: "dbt docs generate --profiles-dir . --target=dev"
- name: Install pre-commit
run: pip install pre-commit
- name: Run pre-commit model checks
run: pre-commit run --all-files
from dbt-checkpoint.
checking out the work and comments in PR 48, and hacking my own fork to get around this, but it seems that the github action is escaping the args line as indicated both in @simon-tarr 's comment and in the original readme. By escaping, I mean that it is passing the entire args line as one command (not as a command + flag + argument). The result is that it doesn't recognize what is passed as a command. As an example, even if I pass nothing, it defaults to the "run --all-files" in the actions yaml, and returns this:
I'm facing the exact same issues, it's weird no one has this issue, I was able to replicate it each and every time.
Is there a fix? @tomsej @Aylr @grace-cityblock
Thank you!
from dbt-checkpoint.
@grace-cityblock Did you not even need to pass environment variables or put a profiles.yml
file up on your dbt repo with the approach you ultimately went with?
from dbt-checkpoint.
Related Issues (20)
- run, compile, test should use same dbt profile.
- replace-script-table-names fails on EXTRACT(datepart FROM column)
- create check-exposure-has-meta-keys HOT 1
- allow extra keys in check-sources-has-meta-keys HOT 1
- Add skip functionality to "dbt-deps" hook
- Support for multiple data types for check-column-name-contract hook HOT 1
- `check-column-name-contract` doesn't output the name of the files HOT 1
- check-model-name-contract support for snapshots HOT 1
- Typo in docs for how to use --exclude
- Following the instructions for github actions failed (re: profile.yml)
- unable to exclude specific model from pre-commit
- check_source_has_all_columns reports failure due to use of lower() HOT 1
- Support multiple dbt project roots in a single repo HOT 2
- check-script-ref-and-source erroneously checks refs in comments
- Generate docs for only the staged/changed model(s) HOT 3
- `check-source-has-tests` has the wrong argument documented for test count HOT 1
- check-source-loaded-at-field-is-valid
- check-column-name-contract (and maybe other checkpoints based on Catalog) does not work with Versioned models HOT 1
- dbt-docs-generate speedup possibility with --no-compile? HOT 9
- check-script-ref-and-source hook not accounting for versioned models
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dbt-checkpoint.