[Feature] Allow `args` flag to be passed to `run-operation` about dbt-core HOT 3 CLOSED

Wanna give this a try?

{% macro cowsay(message="hi") %}

{% do log(" ________________________________________", info=True) %}
{% do log("   " ~ message, info=True) %}
{% do log(" ----------------------------------------", info=True) %}
{% do log("        \   ^__^", info=True) %}
{% do log("         \  (oo)\_______", info=True) %}
{% do log("            (__)\       )", info=True) %}
{% do log("                ||----w |", info=True) %}
{% do log("                ||     ||", info=True) %}

{% endmacro %}

dbt run-operation cowsay --args '{message: "Arg, I be a pirate"}'

16:08:08   ________________________________________
16:08:08     Arg, I be a pirate
16:08:08   ----------------------------------------
16:08:08          \   ^__^
16:08:08           \  (oo)\_______
16:08:08              (__)\       )
16:08:08                  ||----w |
16:08:08                  ||     ||

from dbt-core.

Since it looks like we already support this, I'm going to close this as "not planned". But just let me know if I overlooked something and we can re-open it.

from dbt-core.

@aranke I overlooked one key piece in your original message 😅

👉 the part where {{ some_macro() }} is within the args you are passing.

I'm interpreting that the desired behavior is for the --vars argument to accept Jinja that is rendered as a pre-step and passed to the macro after it is rendered. (I recognize that it could be interpreted the exact opposite, so please let me know if so.)

While we don't plan on adopting that behavior within run-operation, there are a couple ways to achieve this today:

1. Push the Jinja logic into the macro (recommended)
2. Use interactive compilation to render your values as a pre-step

Option 1 - push the Jinja logic into the macro

Suppose you have a cowsay macro like described in #9862 (comment).

We'd recommend refactoring the cowsay macro so that it does not need --args to contain any Jinja. That way, it can be called like this:

dbt run-operation cowsay
dbt run-operation cowsay --args "{suffix_type: 'standard'}"
dbt run-operation cowsay --args "{suffix_type: 'alternate'}"

{% macro cowsay(message="hi", suffix_type=none) %}

{% set timestamp = get_timestamp_suffix(suffix_type) %}

{% do log(" ________________________________________", info=True) %}
{% do log("   " ~ timestamp ~ " " ~ message          , info=True) %}
{% do log(" ----------------------------------------", info=True) %}
{% do log("        \   ^__^"                         , info=True) %}
{% do log("         \  (oo)\_______"                 , info=True) %}
{% do log("            (__)\       )"                , info=True) %}
{% do log("                ||----w |"                , info=True) %}
{% do log("                ||     ||"                , info=True) %}

{% endmacro %}


{% macro get_timestamp_suffix(suffix_type=none) %}
    {% if suffix_type == 'standard' %}
        {% do return(run_started_at.strftime('%d-%m-%Y')) %}
    {% elif suffix_type == 'alternate' %}
        {% do return(run_started_at.strftime('%Y%m%d')) %}
    {% else %}
        {% do return("") %}
    {% endif %}
{% endmacro %}

Option 2 - use interactive compilation

The following is a more complicated option. I'm not sure if there are valid use-cases for it or not, but providing it for the sake of completeness.

The general pattern is this:

1. Use interactive compilation to render your Jinja
2. Store the result in an environment variable
3. Pass the value of the environment variable as an argument

It is a multi-step process that is dependent upon the capabilities of the shell it is operating within (bash, zsh, fsh, PowerShell, etc.), so your mileage may vary.

Here's an example that works for me in zsh (and should work with bash as well) as long as jq is installed:

export message=$(dbt --log-format json compile --inline "{{ get_timestamp_suffix(suffix_type='standard') }}" --output text | jq -r 'select(.data.compiled != null) | .data.compiled')
dbt run-operation cowsay --args "{message: '$message'}"
unset message

If #9840 is implemented, then the command in the first step would be a bit more simple:

export message=$(dbt compile --inline "{{ get_timestamp_suffix(suffix_type='standard') }}" --quiet)
dbt run-operation cowsay --args "{message: '$message'}"
unset message

from dbt-core.