Skip to content

Implement TDS cursor streaming (sp_cursoropen/sp_cursorfetch/sp_cursorclose) for Repo.stream support#184

Open
zerobatu wants to merge 3 commits into
elixir-ecto:masterfrom
zerobatu:master
Open

Implement TDS cursor streaming (sp_cursoropen/sp_cursorfetch/sp_cursorclose) for Repo.stream support#184
zerobatu wants to merge 3 commits into
elixir-ecto:masterfrom
zerobatu:master

Conversation

@zerobatu
Copy link
Copy Markdown
Contributor

Summary

Implements the DBConnection cursor callbacks (handle_declare/4, handle_fetch/4, handle_deallocate/4) in Tds.Protocol using SQL Server's cursor stored procedures, enabling Repo.stream/2 and DBConnection.stream/4 support.

Changes

Core Implementation

  • lib/tds/protocol.ex: Replaced stub implementations of handle_declare/4, handle_fetch/4, and handle_deallocate/4 with working cursor operations:

    • handle_declare/4 uses sp_cursoropen (ProcID 2) with FORWARD_ONLY / READ_ONLY cursor options. Automatically enables PARAMETERIZED_STMT mode (scrollopt 0x1004) when parameters are present.
    • handle_fetch/4 uses sp_cursorfetch (ProcID 7) with NEXT fetch type and configurable max_rows.
    • handle_deallocate/4 uses sp_cursorclose (ProcID 9) to close the cursor.
    • Updated message(:executing, ...) to include return value parameters in Tds.Result so the cursor handle can be extracted from sp_cursoropen's output.
  • lib/tds/messages.ex:

    • Uncommented cursor stored procedure IDs (@tds_sp_cursoropen, @tds_sp_cursorfetch, @tds_sp_cursorclose).
    • Added encode_rpc clauses for :sp_cursoropen, :sp_cursorfetch, :sp_cursorclose.
    • Added {:returnvalue, param} handling in parse(:executing, ...) to capture output parameters from cursor RPCs.
  • lib/tds/result.ex: Added params field to Tds.Result struct for storing RPC output/return value parameters.

  • lib/tds/tokens.ex: Added support for TABNAME (0xA4) and COLINFO (0xA5) tokens sent by SQL Server in cursor result sets.

  • lib/tds.ex: Added Tds.stream/4 public API function delegating to DBConnection.stream/4.

Tests

  • test/stream_test.exs: 12 tests covering:
    • Simple streaming queries
    • Chunked streaming with max_rows
    • Parameterized queries
    • Empty result sets
    • DBConnection.stream/4 direct usage
    • Lazy enumeration with Stream.map and Enum.take
    • Lazy batch processing with Stream.each and side effects (Agent)
    • Progressive transformation with Stream.flat_map
    • Streaming after a prior query on the same connection
    • Connection recovery after stream errors
    • Reusable prepared queries

Notes

  • SQL Server cursors add a ROWSTAT column to result sets. Users should be aware of this additional column when processing stream results.
  • Streaming requires an active transaction (SQL Server requirement).
  • The :max_rows option controls fetch batch size and defaults to 500, matching Ecto.Repo.stream/2 convention.

Claudio Alvarado added 2 commits May 18, 2026 22:33
@zerobatu zerobatu force-pushed the master branch 3 times, most recently from 4577ac2 to 2cfa0d6 Compare May 19, 2026 03:35
GitHub Actions runners have a pre-existing /etc/apt/sources.list.d/microsoft-prod.list
that references signed-by=/usr/share/keyrings/microsoft-prod.gpg. Adding the same repo
without signed-by causes a conflicting values error. Fix by removing the pre-existing
source list, using gpg --dearmor, and adding signed-by consistently.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant