Skip to content

fix: wrap _check_dynamic_streams_availability in try/except#990

Draft
devin-ai-integration[bot] wants to merge 1 commit intomainfrom
devin/1776378187-fix-dynamic-streams-check-exception
Draft

fix: wrap _check_dynamic_streams_availability in try/except#990
devin-ai-integration[bot] wants to merge 1 commit intomainfrom
devin/1776378187-fix-dynamic-streams-check-exception

Conversation

@devin-ai-integration
Copy link
Copy Markdown
Contributor

Resolves https://github.com/airbytehq/airbyte-internal-issues/issues/16199:

Summary

CheckStream._check_dynamic_streams_availability in airbyte_cdk/sources/declarative/checks/check_stream.py previously accessed source.resolved_manifest and source.dynamic_streams without any error handling. When those accesses raised — e.g., when a ConfigComponentsResolver fails during manifest resolution — the exception propagated all the way up through check_connection and surfaced to users as a raw Python traceback instead of a clean (False, message)AirbyteConnectionStatus.FAILED.

This PR wraps the dynamic-stream resolution lines in try/except, funneling errors through self._log_error(...), matching the existing pattern already used in the sibling _check_stream_availability method in the same file.

Changes

  • airbyte_cdk/sources/declarative/checks/check_stream.py: wrap the source.resolved_manifest.get(...) / source.dynamic_streams / _map_generated_streams(...) block in try/except Exception as error and return self._log_error(logger, "resolving dynamic streams for check", error) on failure. The # type: ignore[...] comments are preserved and the post-try/except loop is unchanged.
  • unit_tests/sources/declarative/checks/test_check_stream.py: add a parametrized unit test that constructs a CheckStream with a non-empty dynamic_streams_check_configs list, injects a fake Source that raises during dynamic stream resolution (once via resolved_manifest.get(...) and once via the dynamic_streams property), and asserts that check_connection returns (False, <error message>) instead of raising.

Context

The triggering discovery is airbytehq/airbyte#76373 (source-google-ads), which activates this previously-inert code path by adding dynamic_streams_check_configs to the Google Ads manifest.

Declarative-First Evaluation

Not applicable — this is a defensive error-handling fix inside the Python CDK itself (not a connector manifest change).

Breaking Change Evaluation

Not a breaking change. No schema, spec, PK, cursor, stream set, or state format changes. Purely defensive error handling — happy-path behavior is unchanged, and the only observable difference is that previously-unhandled exceptions now return a clean (False, message) pair instead of a raw traceback.

Review & Testing Checklist for Human

  • Confirm the sibling-method pattern match is appropriate here (i.e., that catching broad Exception inside _check_dynamic_streams_availability is acceptable for consistency with _check_stream_availability, even though the repo's general Python standards discourage broad except Exception blocks).
  • Verify the new unit tests exercise the intended failure paths (resolved_manifest.get raising and the dynamic_streams property raising on subsequent access).

Suggested end-to-end verification: run a manifest-based source whose resolved_manifest or dynamic_streams resolution fails (e.g., a manifest with a misconfigured ConfigComponentsResolver) and confirm check returns a FAILED status with the resolving dynamic streams for check error message rather than an unhandled traceback.

Notes

Keeping this as a draft per repo convention. The fix scope matches the triage report exactly (lines 122-149). There is a related but separate concern at check_stream.py:94-95 where hasattr(source, "dynamic_streams") can itself propagate a non-AttributeError raised from the dynamic_streams property — the new test's second parametrization covers that via a two-call counter (first access succeeds through hasattr, second raises inside the method body).

Link to Devin session: https://app.devin.ai/sessions/95e2723a3e234d2d9d161792bc94d75b

Match the sibling _check_stream_availability pattern by funneling
resolution errors through self._log_error so that failures raised
while accessing source.resolved_manifest or source.dynamic_streams
surface as (False, message) instead of a raw Python traceback.

Co-Authored-By: bot_apk <apk@cognition.ai>
@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@github-actions
Copy link
Copy Markdown

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1776378187-fix-dynamic-streams-check-exception#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1776378187-fix-dynamic-streams-check-exception

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /autofix - Fixes most formatting and linting issues
  • /poetry-lock - Updates poetry.lock file
  • /test - Runs connector tests with the updated CDK
  • /prerelease - Triggers a prerelease publish with default arguments
  • /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
  • /poe <command> - Runs any poe command in the CDK environment
📚 Show Repo Guidance

Helpful Resources

📝 Edit this welcome message.

@github-actions
Copy link
Copy Markdown

PyTest Results (Fast)

4 020 tests  +2   4 009 ✅ +2   7m 6s ⏱️ -34s
    1 suites ±0      11 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit 8b3e3d3. ± Comparison against base commit 1256a1f.

@github-actions
Copy link
Copy Markdown

PyTest Results (Full)

4 023 tests  +2   4 011 ✅ +2   8m 54s ⏱️ - 2m 39s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit 8b3e3d3. ± Comparison against base commit 1256a1f.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants