Is this a new bug in dbt-core? <li class="task-l

Not sure if it is related or not, but in <a class="issue-link js-issue-link" data-erro

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

Notes from estimation: could be related to <a class="issue-lin

[Bug] unit tests' comparisons are sometimes sensitive to the order of records returned about dbt-core HOT 5 CLOSED

joellabes commented on June 12, 2024

[Bug] unit tests' comparisons are sometimes sensitive to the order of records returned

from dbt-core.

Comments (5)

dbeatty10 commented on June 12, 2024 3

Here's some minimal reproducible examples using dbt-postgres:

Reprex

models/my_input_model.sql

select 1 as id, 'some string' as status

models/my_model.sql

select * from {{ ref("my_input_model") }}

models/unit_tests.yml

unit_tests:
  # ❌
  - name: unexpected_test_1
    model: my_model
    given:
      - input: ref("my_input_model")
        rows:
          - {"id":  , "status": 'B'}
          - {"id":  , "status": 'B'}
          - {"id": 3, "status": 'A'}
    expect:
        rows:
          - {"id": 3, "status": 'A'}
          - {"id":  , "status": 'B'}
          - {"id":  , "status": 'B'}

  # ❌
  - name: unexpected_test_2
    model: my_model
    given:
      - input: ref("my_input_model")
        rows:
          - {"id": 3, "status": 'A'}
          - {"id":  , "status": 'B'}
          - {"id":  , "status": 'B'}
    expect:
        rows:
          - {"id":  , "status": 'B'}
          - {"id":  , "status": 'B'}
          - {"id": 3, "status": 'A'}

  # ✅
  - name: expected_test_0
    model: my_model
    given:
      - input: ref("my_input_model")
        rows:
          - {"id": 3, "status": 'A'}
          - {"id":  , "status": 'B'}
    expect:
        rows:
          - {"id":  , "status": 'B'}
          - {"id": 3, "status": 'A'}

  # ✅
  - name: expected_test_1
    model: my_model
    given:
      - input: ref("my_input_model")
        rows:
          - {"id": 3, "status": 'A'}
          - {"id":  , "status": 'B'}
          - {"id":  , "status": 'B'}
    expect:
        rows:
          - {"id": 3, "status": 'A'}
          - {"id":  , "status": 'B'}
          - {"id":  , "status": 'B'}

  # ✅
  - name: expected_test_2
    model: my_model
    given:
      - input: ref("my_input_model")
        rows:
          - {"id":  , "status": 'B'}
          - {"id":  , "status": 'B'}
          - {"id": 3, "status": 'A'}
    expect:
        rows:
          - {"id":  , "status": 'B'}
          - {"id":  , "status": 'B'}
          - {"id": 3, "status": 'A'}

Run this command:

dbt build -s +my_model

from dbt-core.

joellabes commented on June 12, 2024 1

Amusingly, I also believe that each adapter has different order needs! This test fails on Redshift but passes locally on Snowflake: https://app.circleci.com/pipelines/github/dbt-labs/dbt-audit-helper/416/workflows/64b0d0f6-8cd8-48cf-bac2-e462987d69fa/jobs/409 (and presumably vice versa, but I haven't 100% tested that)

from dbt-core.

dbeatty10 commented on June 12, 2024

Not sure if it is related or not, but in dbt-labs/dbt-redshift#821, the rows with non-null values also needed to come prior to the rows with nulls.

from dbt-core.

graciegoheen commented on June 12, 2024

@joellabes Which database are you seeing this error in?

from dbt-core.

graciegoheen commented on June 12, 2024

Notes from estimation:

could be related to #9881 (but that's for csv, this one is independent of format)
this is happening when there are multiple explicit NULL values in the expected output of a unit test
different adapters have different order requirements
alignment in daff ob/main/core/dbt/task/test.py#L338-L353

from dbt-core.

Recommend Projects