Skip to content

Commit

Permalink
restore testing setup
Browse files Browse the repository at this point in the history
  • Loading branch information
MichelleArk committed Sep 25, 2024
1 parent 3a9d7ce commit 07544fe
Show file tree
Hide file tree
Showing 3 changed files with 3 additions and 5 deletions.
3 changes: 2 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,8 @@ To run functional tests we rely on [dagger](https://dagger.io/). This launches a

```sh
pip install -r dagger/requirements.txt
python dagger/run_dbt_spark_tests.py --profile apache_spark --test-path tests/functional/adapter/incremental_strategies/test_microbatch.py
python dagger/run_dbt_spark_tests.py --profile databricks_sql_endpoint --test-path tests/functional/adapter/test_basic.py::TestSimpleMaterializationsSpark::test_base
```

`--profile`: required, this is the kind of spark connection to test against

Expand Down
2 changes: 0 additions & 2 deletions dagger/run_dbt_spark_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -150,8 +150,6 @@ async def test_spark(test_args):

tst_container = tst_container.with_(env_variables(TESTING_ENV_VARS))
test_path = test_args.test_path if test_args.test_path else "tests/functional/adapter"
# TODO: remove before merging!
test_path = "tests/functional/adapter/incremental_strategies/test_microbatch.py"
result = await tst_container.with_exec(
["pytest", "-v", "--profile", test_profile, "-n", "auto", test_path]
).stdout()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -79,8 +79,7 @@
{#-- microbatch wraps insert_overwrite, and requires a partition_by config #}
{% set missing_partition_key_microbatch_msg -%}
dbt-spark 'microbatch' incremental strategy requires a `partition_by` config.
Ensure you are using a `partition_by` column that is of grain {{ config.get('batch_size') }}
for microbatch model {{ model.name }}.
Ensure you are using a `partition_by` column that is of grain {{ config.get('batch_size') }}.
{%- endset %}

{%- if not config.get('partition_by') -%}
Expand Down

0 comments on commit 07544fe

Please sign in to comment.