[SPARK-58738][SQL] Add parse_command function for SQL statement JSON analysis - #57962
Open
srielau wants to merge 5 commits into
Open
[SPARK-58738][SQL] Add parse_command function for SQL statement JSON analysis#57962srielau wants to merge 5 commits into
srielau wants to merge 5 commits into
Conversation
…rences UnresolvedWith keeps CTE definitions in innerChildren, so collectWithSubqueries missed tables/functions/params inside WITH. Add foreachPlanDeep and broader CTE and nested-subquery tests.
Classify CompoundBody as BEGIN END (-22) and deep-walk SingleStatement roots, exception handlers, and simple CASE else bodies so nested refs are not dropped.
Cover SELECT/DML/DDL, CTEs, scripts, errors, and batch evaluation via sql-tests inputs with regenerated result and analyzer golden files.
Keep only statement_identifier/code on success, expose error line/position, and broaden complex SQL, scripting, and non-syntax parser error tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Add a built-in scalar SQL function
parse_command(sqlStmt)that parse-only analyzes a SQL statement string and returns a compact JSON description:parse_successand, on failure, a nested STANDARD-format error object (errorClass,messageTemplate,sqlState,messageParameters) so batch evaluation never fails a query on bad SQLstatement_identifier,statement_code,statement_type,statement_class); Spark-only statements use product-specific identifiers with negative implementation-defined codestable_referencesandfunction_references(identifier part sequences)select_listentries with name parts and expression textparameter_markers(namedandunnamed_count)as_subqueryforCREATE TABLE ... AS SELECT(still Table 39CREATE TABLE/ code 77)Why are the changes needed?
Dialect-migration and SQL analysis tooling need an efficient, batchable way to verify that translated SQL parses and to extract statement kind / references without requiring catalog objects to exist.
Does this PR introduce any user-facing change?
Yes. New SQL function:
How was this patch tested?
Added
ParseCommandResultSuiteandParseCommandSuite(catalyst). Ran:All 13 tests passed.
Was this patch authored or co-authored using generative AI tooling?
Generated-by: Cursor Grok 4.5