Skip to content

Conversation

@zhanghaitao3
Copy link

Fix multiple test failures

This commit addresses the following test failures:

  1. Connection pool and timeout issues:

    • Fix timeout error in test_pool_04
    • Fix timeout error in test_command_timeout_01
  2. Execution-related failures:

    • Fix error message matching in test_execute_exceptions_1
    • Fix result order assertion in test_executemany_client_failure_in_transaction
  3. Database feature support:

    • Handle unsupported CREATE TABLE INHERITS feature
    • Fix column not found error message format matching
  4. Codec-related issues:

    • Fix extension security warnings by proper enable_extension handling
    • Fix syntax error in void type handling

These fixes improve test suite stability and code robustness.

This commit addresses the following test failures:

1. Connection pool and timeout issues:
   - Fix timeout error in test_pool_04
   - Fix timeout error in test_command_timeout_01

2. Execution-related failures:
   - Fix error message matching in test_execute_exceptions_1
   - Fix result order assertion in test_executemany_client_failure_in_transaction

3. Database feature support:
   - Handle unsupported CREATE TABLE INHERITS feature
   - Fix column not found error message format matching

4. Codec-related issues:
   - Fix extension security warnings by proper enable_extension handling
   - Fix syntax error in void type handling
…fic)

This commit addresses critical concurrency issues in the `COPY OUT` protocol implementation. These issues were specifically observed in Python 3.9.9 environments (likely due to differences in asyncio event loop scheduling compared to newer versions like 3.13), leading to deadlocks and data corruption.

Changes:

1. Fix Data Loss (Backpressure Handling):
   - Issue: In Python 3.9, `_dispatch_result` could be triggered multiple times while the sink was paused, overwriting `_pending_result`.
   - Fix: Implemented data merging logic. New incoming data is now appended to the existing pending buffer instead of replacing it.

2. Fix State Pollution on Cancel:
   - Issue: Cancelling a `COPY` task left residual data in `_pending_result`. Subsequent queries (e.g., `SELECT 1`) would incorrectly consume this stale data, causing `AttributeError: 'tuple' object has no attribute '_init_types'`.
   - Fix: Added a `try...finally` block in `copy_out` to unconditionally clear `_pending_result` and reset protocol state upon exit.

3. Fix Deadlocks:
   - Issue: The connection could get stuck in a paused state if execution was interrupted or if `resume_reading` was missed.
   - Fix: Ensured `self.resume_reading()` is called immediately after consuming buffered data in `_new_waiter` and in the cleanup phase.

4. Fix Logic Errors:
   - corrected `_new_waiter` to prevent assigning completed Futures to `self.waiter`, avoiding `InternalClientError`.

Fixes HuaweiCloudDeveloper#8
@zhanghaitao3
Copy link
Author

Fix COPY OUT race conditions and state corruption (Python 3.9.9 specific)

This commit addresses critical concurrency issues in the COPY OUT protocol implementation. These issues were specifically observed in Python 3.9.9 environments (likely due to differences in asyncio event loop scheduling compared to newer versions like 3.13), leading to deadlocks and data corruption.

Changes:

  1. Fix Data Loss (Backpressure Handling):

    • Issue: In Python 3.9, _dispatch_result could be triggered multiple times while the sink was paused, overwriting _pending_result.
    • Fix: Implemented data merging logic. New incoming data is now appended to the existing pending buffer instead of replacing it.
  2. Fix State Pollution on Cancel:

    • Issue: Cancelling a COPY task left residual data in _pending_result. Subsequent queries (e.g., SELECT 1) would incorrectly consume this stale data, causing AttributeError: 'tuple' object has no attribute '_init_types'.
    • Fix: Added a try...finally block in copy_out to unconditionally clear _pending_result and reset protocol state upon exit.
  3. Fix Deadlocks:

    • Issue: The connection could get stuck in a paused state if execution was interrupted or if resume_reading was missed.
    • Fix: Ensured self.resume_reading() is called immediately after consuming buffered data in _new_waiter and in the cleanup phase.
  4. Fix Logic Errors:

    • corrected _new_waiter to prevent assigning completed Futures to self.waiter, avoiding InternalClientError.

Fixes #8
Fixes #6
Fixes #7
Fixes #9
Fixes #10
Fixes #11
Fixes #12

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment