Fix interactive environment clean up failure at atexit. by shunping · Pull Request #38526 · apache/beam

shunping · 2026-05-18T15:13:36Z

The exception occurs in test clean up only and is harmless. Changing it to a warning message and make the test log cleaner.

https://github.com/apache/beam/actions/runs/26003104622/job/76429819073

Exception in thread run_worker_job-202[job]_ref_Environment_default_environment_1:
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/threading.py", line 1075, in _bootstrap_inner
    self.run()
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/threading.py", line 1012, in run
    self._target(*self._args, **self._kwargs)
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/apache_beam/runners/worker/sdk_worker.py", line 265, in run
    for work_request in self._control_stub.Control(get_responses()):
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/grpc/_channel.py", line 538, in __next__
    return self._next()
           ^^^^^^^^^^^^
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/grpc/_channel.py", line 956, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "UNKNOWN:Error received from peer ipv4:127.0.0.1:45545 {grpc_message:"Socket closed", grpc_status:14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/apache_beam/runners/worker/data_plane.py", line 719, in _read_inputs
    for elements in elements_iterator:
                    ^^^^^^^^^^^^^^^^^
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/grpc/_channel.py", line 538, in __next__
    return self._next()
           ^^^^^^^^^^^^
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/grpc/_channel.py", line 956, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "UNKNOWN:Error received from peer ipv4:127.0.0.1:45545 {grpc_message:"Socket closed", grpc_status:14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/threading.py", line 1075, in _bootstrap_inner
    self.run()
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/threading.py", line 1012, in run
    self._target(*self._args, **self._kwargs)
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/apache_beam/runners/worker/data_plane.py", line 736, in <lambda>
    target=lambda: self._read_inputs(elements_iterator),
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/apache_beam/runners/worker/data_plane.py", line 719, in _read_inputs
    for elements in elements_iterator:
                    ^^^^^^^^^^^^^^^^^
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/grpc/_channel.py", line 538, in __next__
    return self._next()
           ^^^^^^^^^^^^
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/grpc/_channel.py", line 956, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Socket closed"
	debug_error_string = "UNKNOWN:Error received from peer ipv4:127.0.0.1:45545 {grpc_message:"Socket closed", grpc_status:14}"
>
Exception ignored in atexit callback: <bound method InteractiveEnvironment.cleanup of <apache_beam.runners.interactive.interactive_environment.InteractiveEnvironment object at 0x787b14391670>>
Traceback (most recent call last):
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/apache_beam/runners/interactive/interactive_environment.py", line 298, in cleanup
    self.cleanup_environment()
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/apache_beam/runners/interactive/interactive_environment.py", line 288, in cleanup_environment
    cache_manager.cleanup()
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/apache_beam/runners/interactive/cache_manager.py", line 293, in cleanup
    fs.delete([self._cache_dir + '/full/'])
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/apache_beam/io/gcp/gcsfilesystem.py", line 357, in delete
    self._gcsIO().delete(path, recursive=True)
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/apache_beam/io/gcp/gcsio.py", line 272, in delete
    f'gs://{bucket_name}/{blob.name}' for blob in blobs_to_delete
                                                  ^^^^^^^^^^^^^^^
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/google/api_core/page_iterator.py", line 208, in _items_iter
    for page in self._page_iter(increment=False):
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/google/api_core/page_iterator.py", line 244, in _page_iter
    page = self._next_page()
           ^^^^^^^^^^^^^^^^^
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/google/api_core/page_iterator.py", line 373, in _next_page
    response = self._get_next_page_response()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/google/api_core/page_iterator.py", line 432, in _get_next_page_response
    return self.api_request(
           ^^^^^^^^^^^^^^^^^
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/google/cloud/storage/_http.py", line 90, in api_request
    return call()
           ^^^^^^
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/google/api_core/retry/retry_unary.py", line 294, in retry_wrapped_func
    return retry_target(
           ^^^^^^^^^^^^^
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/google/api_core/retry/retry_unary.py", line 156, in retry_target
    next_sleep = _retry_error_helper(
                 ^^^^^^^^^^^^^^^^^^^^
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/google/api_core/retry/retry_base.py", line 216, in _retry_error_helper
    raise final_exc from source_exc
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/google/api_core/retry/retry_unary.py", line 147, in retry_target
    result = target()
             ^^^^^^^^
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py312/build/srcs/sdks/python/target/.tox-py312-cloud/py312-cloud/lib/python3.12/site-packages/google/cloud/_http/__init__.py", line 494, in api_request
    raise exceptions.from_http_response(response)
google.api_core.exceptions.Forbidden: 403 GET https://storage.googleapis.com/storage/v1/b/test-bucket/o?projection=noAcl&prefix=132468897523968%2Ffull%2F&prettyPrint=false: beam-github-actions@apache-beam-testing.iam.gserviceaccount.com does not have storage.objects.list access to the Google Cloud Storage bucket. Permission 'storage.objects.list' denied on resource (or it may not exist).

gemini-code-assist · 2026-05-18T15:14:27Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses an issue where interactive environment cleanup failures during the atexit phase caused noisy and unnecessary exceptions in test logs. By catching these exceptions and logging them as warnings, the cleanup process becomes more resilient and the test output is significantly cleaner.

Highlights

Error Handling: Wrapped the cache cleanup logic in a try-except block to prevent uncaught exceptions during the atexit process.
Logging: Replaced the fatal exception with a warning log message when cache cleanup fails, improving test log cleanliness.

New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces error handling and logging to the cleanup method of the CacheManager to ensure that failures during cache deletion are caught and logged rather than causing a crash. While the logging implementation is sound, the reviewer identified that moving GCP-related imports to the top level introduces a hard dependency on the google-cloud-storage library, which would cause ImportError for users without GCP dependencies installed. The feedback recommends reverting these to local imports within the cleanup method to maintain the optional nature of GCP support in the Beam SDK.

github-actions · 2026-05-18T17:04:34Z

Assigning reviewers:

R: @claudevdm for label python.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

shunping · 2026-05-18T19:28:45Z

The failure of "PreCommit Python PVR Flink" exists in master branch as well, so it is irrelevant to the change made here.

Fix interactive environment clean up failure at atexit.

932d459

github-actions Bot added python runners labels May 18, 2026

shunping marked this pull request as ready for review May 18, 2026 15:14

gemini-code-assist Bot reviewed May 18, 2026

View reviewed changes

Comment thread sdks/python/apache_beam/runners/interactive/cache_manager.py Outdated

Comment thread sdks/python/apache_beam/runners/interactive/cache_manager.py

github-actions Bot added the Next Action: Reviewers label May 18, 2026

shunping added 2 commits May 18, 2026 14:03

Fix failed tests.

fbe67fb

Formatting.

bd8fa41

claudevdm approved these changes May 18, 2026

View reviewed changes

shunping merged commit 55eb624 into apache:master May 18, 2026
97 of 101 checks passed

shunping deleted the fix-interative-environment-warning branch May 18, 2026 19:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix interactive environment clean up failure at atexit.#38526

Fix interactive environment clean up failure at atexit.#38526
shunping merged 3 commits into
apache:masterfrom
shunping:fix-interative-environment-warning

shunping commented May 18, 2026 •

edited

Loading

Uh oh!

gemini-code-assist Bot commented May 18, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented May 18, 2026

Uh oh!

shunping commented May 18, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

shunping commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist Bot commented May 18, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented May 18, 2026

Uh oh!

shunping commented May 18, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

shunping commented May 18, 2026 •

edited

Loading