feat(dbt): add dataQualityAssertions coverage to csv_to_postgres scenario#277
Open
roller100 wants to merge 3 commits intoOpenLineage:mainfrom
Open
feat(dbt): add dataQualityAssertions coverage to csv_to_postgres scenario#277roller100 wants to merge 3 commits intoOpenLineage:mainfrom
roller100 wants to merge 3 commits intoOpenLineage:mainfrom
Conversation
added 3 commits
March 18, 2026 13:04
Add `dbt-ol test` to test/run.sh so test-phase events are captured alongside seed and run events. Add three expected event files for the dataQualityAssertions facet: - events/customers/data_quality_event.json (not_null + unique on customer_id) - events/orders/data_quality_event.json (not_null on customer_id, not_null + unique on order_id) - events/analytics/data_quality_event.json (not_null on customer_id + total_revenue, unique on customer_id) Register all three as test cases in config.json with the dataQualityAssertions facet tag. Expected event content derived from actual dbt-ol 1.23.0 output against a live PostgreSQL 15 instance. Signed-off-by: roller100 (BearingNode) <contact@bearingnode.com>
The local debugging section referenced run_dbt_tests.sh, which was removed during the PR OpenLineage#211 cleanup. Replace with accurate instructions using docker compose + run.sh directly. Update the workflow description, test structure layout, and validation scope (add dataQualityAssertions) to match the current architecture. Signed-off-by: roller100 (BearingNode) <contact@bearingnode.com>
Add **/events/*.jsonl to the existing local-testing ignore block. dbt-ol writes events.jsonl before extraction; this prevents it from being accidentally staged in any producer's events directory. Signed-off-by: roller100 (BearingNode) <contact@bearingnode.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on the dbt producer compatibility test merged in #211.
Adds
dbt-ol testtotest/run.shso test-phase events are captured alongside seed and run. Includes three expecteddata_quality_event.jsonfiles forstg_customers,stg_orders, andcustomer_analytics, each registered inconfig.jsonunder thedataQualityAssertionsfacet tag. Event content derived from actualdbt-ol 1.23.0output against PostgreSQL 15.Also fixes the local debugging section of
producer/dbt/README.md, which still referencedrun_dbt_tests.sh— a script removed in #211 — replacing it with the currentdocker compose+run.shworkflow.