[Blueprints] Support multiline SQL queries in the runSql step#2928
Merged
[Blueprints] Support multiline SQL queries in the runSql step#2928
Conversation
… support The runSql step now processes SQL files using proper streaming semantics instead of loading entire files into memory. This change enables handling large SQL dumps efficiently while adding support for multiline queries, SQL comments, and complex query structures. The implementation uses WP_MySQL_Naive_Query_Stream to parse SQL incrementally in 8KB chunks. As each chunk arrives, the parser identifies complete queries (those ending with semicolons) and executes them immediately. This approach keeps memory usage constant regardless of file size and provides better feedback during long-running imports. Query parsing leverages the MySQL lexer from the sqlite-database-integration plugin, which understands MySQL syntax including string literals, comments, and nested statements. The bundled WP_MySQL_Naive_Query_Stream class wraps the lexer to handle streaming scenarios where queries may span multiple chunks. Tests have been updated to verify multiline query handling, SQL comment preservation, and queries with subqueries. The streaming parser correctly handles edge cases like empty lines, semicolon-only lines, and queries split across chunk boundaries.
Removed loading of PHP classes from sqlite-database-integration repository in tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adapts WP_MySQL_Naive_Query_Stream to support multiline SQL queries in the
runSqlstep. With this PR, the following call works:Whereas before this PR, the
runSqlstep assumed every line of a SQL file is a separate query and would fail on the above call.Implementation details
See WordPress/sqlite-database-integration#264. Tl;dr we tokenize the query and treat
;and EOF tokens as query separators. The stream is only "naive" in that every query must be smaller than 15MB. It might fail for some very large WordPress posts, but should work most of the time. Once the lexer provides an explicit distinction between syntax errors and incomplete input, we'll be able to support arbitrarily large queries.Testing Instructions (or ideally a Blueprint)
Tests have been updated to verify multiline query handling, SQL comment preservation, and queries with subqueries. The streaming parser correctly handles edge cases like empty lines, semicolon-only lines, and queries split across chunk boundaries.
cc @JanJakes