Skip to content

why-async-wrapper.md

Latest commit

 

History

History
533 lines (401 loc) · 19.6 KB

why-async-wrapper.md

File metadata and controls

533 lines (401 loc) · 19.6 KB

Why an Async Wrapper is Necessary

This document provides a comprehensive technical analysis of why the async-cassandra wrapper is necessary for modern async Python applications using Cassandra.

Table of Contents

Executive Summary

While the cassandra-driver provides execute_async() for non-blocking query execution, it was designed before Python's async/await became standard. This creates several fundamental incompatibilities with modern async Python applications.

1. The Synchronous Paging Problem

The Issue

The fetch_next_page() method is synchronous and blocks the event loop:

# From cassandra-driver documentation
result = session.execute_async(query).result()
while result.has_more_pages:
    result.fetch_next_page()  # BLOCKS the event loop!

Evidence

Impact

  • Blocks event loop for the duration of network round-trip to Cassandra
  • Prevents processing other requests during this time
  • Performance impact depends on latency and query patterns

2. Thread Pool Bottlenecks

The Issue

The cassandra-driver uses a thread pool (Session.executor) for I/O operations:

# From cassandra/cluster.py
self.executor = ThreadPoolExecutor(max_workers=executor_threads)

Problems with Thread Pools in Async Apps

  1. Thread Pool Exhaustion

    • Default pool size from driver source: min(2, get_cpu_count() // 2) (cluster.py line 1048)
    • Each blocking operation consumes a thread
    • High concurrency quickly exhausts the pool

  2. Context Switching Overhead

    • Threads require OS-level context switches
    • Much heavier than async task switches
    • Increases latency and CPU usage

  3. GIL Contention

    • Python's Global Interpreter Lock creates contention
    • Threads can't truly run in parallel for Python code
    • Reduces effectiveness of threading

Evidence

From the driver source (cassandra/cluster.py):

# Line 2087 in Session.__init__
self.executor = ThreadPoolExecutor(max_workers=executor_threads)

# Line 2718 in execute_async
future = self.executor.submit(self._execute, *args, **kwargs)
# This goes through thread pool, not event loop

3. Missing Async Context Manager Support

The Issue

The driver doesn't provide async context managers:

# This doesn't work with cassandra-driver
async with cluster.connect() as session:  # Not supported
    await session.execute(query)

# You can't do this either
async with cluster:  # Not supported
    pass

Why This Matters

  • No automatic async cleanup
  • Risk of resource leaks in async applications
  • Inconsistent with Python async conventions

What async-cassandra Provides

# Async context manager support (use carefully - see note below)
async with AsyncCluster(['localhost']) as cluster:
    async with await cluster.connect() as session:
        result = await session.execute(query)
# Automatic cleanup on exit

# IMPORTANT: For production applications, create cluster/session once:
cluster = AsyncCluster(['localhost'])
session = await cluster.connect()
# Reuse session for all requests - DO NOT close after each use

Note: While async-cassandra provides context manager support for convenience in scripts and tests, production applications should create clusters and sessions once at startup and reuse them throughout the application lifetime. See the FastAPI example for the correct pattern.

4. Lack of Async-First API Design

The Issue

The driver's API wasn't designed for async/await:

  1. Futures vs Coroutines

    # cassandra-driver returns ResponseFuture
future = session.execute_async(query)
result = future.result()  # Not awaitable

# async-cassandra returns coroutines
result = await session.execute(query)  # Natural async/await

    See ResponseFuture API

  2. No Async Iteration

    # Can't do this with cassandra-driver
async for row in result:
    await process(row)

  3. Mixed Sync/Async APIs

    • Some operations only sync (prepare, set_keyspace)
    • Others only async through callbacks
    • No consistent async interface

5. Event Loop Integration Issues

The Issue

The driver doesn't integrate with asyncio's event loop:

# Driver uses its own event loop in libev
self._io_event_loop = EventLoop()

# This runs in a separate thread, not asyncio's loop

Problems

  1. Two Event Loops: Driver's libev loop + asyncio loop
  2. Cross-Thread Communication: Overhead and complexity
  3. No Backpressure: Can't use asyncio's flow control

Evidence

From driver architecture (connection.py):

  • Uses libev/libuv for I/O loop (io/libevreactor.py)
  • Runs in separate thread from asyncio
  • Requires thread-safe communication

6. Callback-Based vs Async/Await Patterns

The Issue

The driver uses callbacks, not async/await:

# cassandra-driver pattern
def callback(response):
    # Handle response
    pass

def err_callback(error):
    # Handle error
    pass

future = session.execute_async(query)
future.add_callback(callback)
future.add_errback(err_callback)

Problems

  1. Callback Hell: Nested callbacks become unreadable
  2. Error Handling: Try/except doesn't work naturally
  3. No Stack Traces: Lost context in callbacks

async-cassandra Solution

from cassandra import InvalidRequest

# Clean async/await pattern
try:
    result = await session.execute(query)
    processed = await process(result)
    await save(processed)
except InvalidRequest as e:
    # Natural error handling with Cassandra exceptions
    logger.error(f"Invalid query: {e}")

7. Resource Management in Async Contexts

Connection Lifecycle Issues

  1. Synchronous Shutdown

    # cassandra-driver - shutdown methods are synchronous
cluster.shutdown()  # Synchronous method
session.shutdown()  # Synchronous method

    See Cluster.shutdown() API

  2. No Graceful Async Cleanup

    • Can't await pending operations
    • No async connection draining
    • Risk of resource leaks

Statement Preparation

# cassandra-driver - prepare() is synchronous
prepared = session.prepare(query)  # Synchronous call

# async-cassandra provides async version
prepared = await session.prepare(query)  # Non-blocking

The synchronous prepare() method is documented in the Session API

8. Performance Implications

Performance Considerations

  1. Thread Pool Overhead

    • Each query requires thread allocation from limited pool
    • Context switching between threads has OS overhead
    • Python's GIL prevents true parallelism

  2. Memory Usage

    • Each thread requires its own stack space
    • Coroutines share the same stack
    • Memory difference becomes significant with many concurrent operations

  3. Scheduling Overhead

    • Thread scheduling is handled by the OS
    • Coroutine scheduling is handled by Python's event loop
    • Event loop scheduling is more efficient for I/O-bound operations

Real-World Impact

Performance improvements vary based on:

  • Workload characteristics (query complexity, result size)
  • Network latency to Cassandra cluster
  • Concurrency level of the application
  • Hardware resources available

The async approach shows the most benefit in high-concurrency scenarios where the thread pool becomes a bottleneck.

Additional Benefits of async-cassandra

1. Streaming API

# Memory-efficient large result processing
async for row in await session.execute_stream(query):
    await process(row)  # Processes without loading all in memory

2. Metrics and Monitoring

  • Built-in async metrics collection
  • Integration with async monitoring systems
  • No thread-safety concerns

3. Retry Policies

  • Async-aware retry logic
  • Idempotency checking for safety
  • Non-blocking retries

4. Type Safety

  • Full type hints for async operations
  • Better IDE support
  • Catch errors at development time

What This Wrapper Actually Solves (And What It Doesn't)

The Reality Check

Let's be completely honest: the cassandra-driver is fundamentally a synchronous, thread-based driver. No wrapper can change this underlying architecture. Here's what we're really dealing with:

The Driver's Architecture

From analyzing the source code:

  1. All I/O is blocking (connection.py):

    # Actual socket operations block the thread
self._socket.send(data)  # Blocks
data = self._socket.recv(self.in_buffer_size)  # Blocks

  2. "Async" means thread pool (cluster.py):

    def execute_async(self, query, ...):
    # Just submits to thread pool
    future = self._create_response_future(query, ...)
    self.submit(self._connection_holder.get_connection, ...)
    return future  # Not an asyncio.Future!

  3. ResponseFuture is not asyncio-compatible:

    • Custom future implementation
    • Callbacks run in thread pool threads
    • No native event loop integration

What async-cassandra Actually Does

The wrapper provides a bridge between the thread-based driver and asyncio:

# What happens under the hood
def _asyncio_future_from_response_future(response_future):
    asyncio_future = asyncio.Future()

    def callback(result):
        # Bridge from thread to event loop
        loop.call_soon_threadsafe(
            asyncio_future.set_result, result
        )

    response_future.add_callback(callback)
    return asyncio_future

What This Solves

  1. Developer Experience:

    • Clean async/await syntax
    • Natural error handling with try/except
    • Integration with async frameworks (FastAPI, aiohttp)
    • See our FastAPI example for a complete implementation

  2. Event Loop Compatibility:

    • Prevents blocking the event loop with synchronous calls
    • Allows other coroutines to run while waiting for Cassandra

  3. Async Paging (with caveats):

    • Wraps page fetching to not block event loop
    • But still uses threads under the hood

What This CANNOT Solve

  1. Thread Pool Overhead:

    • Every query still goes through the thread pool
    • Limited by thread pool size (default 2-4 threads) - see configuration guide
    • Thread creation/switching overhead remains

  2. True Async Performance:

    • Not comparable to truly async drivers (like aioredis)
    • Still limited by GIL and thread contention
    • Cannot handle thousands of concurrent connections

  3. Resource Usage:

    • Threads still consume ~2MB each
    • Thread pool must be sized appropriately
    • Not as lightweight as pure coroutines

  4. Fundamental Blocking Operations:

    • Connection establishment blocks threads
    • DNS resolution blocks threads
    • SSL handshakes block threads

Performance Reality

Instead of claiming "25x improvement", here's the reality:

  • Best case: Prevents event loop blocking, allowing better concurrency
  • Typical case: Similar throughput to sync driver, better latency distribution
  • Worst case: Added overhead from thread-to-event-loop bridging

🎯 The Most Important Point

The real benefit is not blocking other operations, not magically making Cassandra faster.

Think of it this way: Without this wrapper, when your app queries Cassandra, it's like a cashier who stops serving all customers while waiting for a price check. With this wrapper, the cashier can help other customers while waiting. The price check doesn't get faster, but the line keeps moving!

When Should You Use This?

Use async-cassandra when:

  • You have an async application (FastAPI, aiohttp)
  • You need to prevent blocking the event loop
  • You want cleaner async/await syntax
  • You're already committed to the cassandra-driver

Don't use it expecting:

  • True async performance like aioredis or asyncpg
  • To handle thousands of concurrent queries
  • To eliminate thread overhead
  • Magic performance improvements

The Honest Comparison

Aspect True Async Driver (e.g., asyncpg) async-cassandra
I/O Model Non-blocking sockets Blocking sockets in threads
Concurrency Thousands of connections Limited by thread pool
Memory ~2KB per connection ~2MB per thread*
CPU Usage Single thread Multiple threads + GIL
Performance High Moderate
Integration Native async Adapted via callbacks

*Memory calculation: Each thread in Python requires approximately 1.5-2MB of memory. This is an empirical observation that can be verified:

# Measure actual thread memory overhead
import os
import psutil
import threading
import time

def worker():
    # Keep thread alive
    while True:
        time.sleep(1)

# Measure baseline memory
process = psutil.Process(os.getpid())
baseline_mb = process.memory_info().rss / 1024 / 1024

# Create threads and measure memory increase
threads = []
for i in range(10):
    t = threading.Thread(target=worker, daemon=True)
    t.start()
    threads.append(t)
    time.sleep(0.1)  # Let thread fully initialize

final_mb = process.memory_info().rss / 1024 / 1024
per_thread_mb = (final_mb - baseline_mb) / 10

print(f"Memory per thread: {per_thread_mb:.2f} MB")
# Typically shows 1.5-2.0 MB per thread

The memory includes:

  • Thread stack space: Reserved virtual memory (not all used immediately)
  • Thread-local storage: Python interpreter state per thread
  • OS thread structures: Kernel data structures for thread management
  • Python objects: Thread objects, locks, and synchronization primitives

Future Possibilities

A truly async Cassandra driver would require:

  • Complete rewrite using non-blocking I/O
  • Native asyncio protocol implementation
  • Elimination of thread pools
  • Direct event loop integration

This is a massive undertaking that the cassandra-driver team hasn't prioritized, likely because:

  • Significant engineering effort required
  • Need to maintain compatibility
  • Current solution works for most use cases

Conclusion

The async-cassandra wrapper exists to solve a specific problem:

Modern async Python applications need to use Cassandra without blocking the event loop.

This wrapper provides:

  • Event loop compatibility - Prevents blocking other operations
  • Framework integration - Works with FastAPI, aiohttp, etc.
  • Better developer experience - async/await instead of callbacks
  • Streaming without blocking - Page fetching doesn't freeze your app

But let's be clear about what it is:

  • A compatibility layer, not a performance enhancement
  • A bridge between threads and asyncio, not true async I/O
  • A practical solution, not a perfect one

The Bottom Line

If you need to use Cassandra from an async Python application, you have three choices:

  1. Use synchronous calls - Blocks your event loop, kills concurrency
  2. Use execute_async() directly - Callbacks, no async/await, still blocks on paging
  3. Use this wrapper - Clean async/await, no event loop blocking, but still threads underneath

Until someone writes a true async Cassandra driver from scratch (massive undertaking), this wrapper provides the best available solution for async Python applications that need Cassandra.

While the underlying architecture remains thread-based, this wrapper provides essential compatibility for async applications. When you need to integrate Cassandra with FastAPI or any async framework, having an interface that doesn't block your event loop is crucial for maintaining application responsiveness.

References

  1. cassandra-driver source code

  2. DataStax Python Driver Documentation

  3. JIRA Issues (Note: Some JIRA links may require DataStax account access)

  4. Python Documentation

  5. Community Discussions

  6. Performance References

This analysis is based on cassandra-driver version 3.29.2. The driver team may address some of these issues in future versions.