[ADAP-1069] Custom batch_id prevents rebuilding the model on subsequent builds about dbt-bigquery HOT 6 OPEN

Looks like I missed that this was also found in ADAP-1016.

from dbt-bigquery.

Thanks for raising this @dlubawy, it's worth noting that the fix we have pushed out likely does not address this specific case as setting a static batch_id will be expected to fail on rerun (that's how dataproc batch Id's are supposed to work). Ultimately where we should go is probably having the config allow you to specify a batch_id prefix and we add the invocation_id as a suffix so that each run has a distinct batch_id.

from dbt-bigquery.

This was found after trying to bypass the ADAP-1067 issue by setting a custom batch_id.

from dbt-bigquery.

Thanks for raising this @dlubawy, it's worth noting that the fix we have pushed out likely does not address this specific case as setting a static batch_id will be expected to fail on rerun (that's how dataproc batch Id's are supposed to work). Ultimately where we should go is probably having the config allow you to specify a batch_id prefix and we add the invocation_id as a suffix so that each run has a distinct batch_id.

Prefix with an appended invocation_id sounds like a good idea. However, there would need to be some enforcement on the user input for the configuration such that this wouldn't break due to batch_id length constraints. This is the issue for #1050 when batch_id was being derived from node names in v1.7.

from dbt-bigquery.

This issue implies we have some opportunities to improve the UX around batch id.

from dbt-bigquery.

I've been running into this problem for a while now. For completeness on issue tracking, I'll link my recent comment on a related issue: #1006.

from dbt-bigquery.