[SPARK-58723][SQL] Support the ANSI SQL/JSON_QUERY function

[s-ganesha]

What changes were proposed in this pull request?

Add the ANSI SQL:2016 JSON_QUERY function (feature T828), which extracts the JSON value located by a SQL/JSON path from a JSON input and returns it as JSON text.

JSON_QUERY(jsonExpr, path
           [ RETURNING type ]
           [ (WITHOUT | WITH [CONDITIONAL | UNCONDITIONAL]) [ARRAY] WRAPPER ]
           [ (KEEP | OMIT) QUOTES ]
           [ (NULL | ERROR | EMPTY ARRAY | EMPTY OBJECT) ON EMPTY ]
           [ (NULL | ERROR | EMPTY ARRAY | EMPTY OBJECT) ON ERROR ])

It is implemented as a parser-only production (no FunctionRegistry entry), mirroring the merged JSON_VALUE (SPARK-58685) and reusing the JSON path evaluator from JSON_TABLE (SPARK-58366). The matched object/array/scalar is serialized verbatim; a scalar is emitted as JSON text under the default WITHOUT ARRAY WRAPPER, OMIT QUOTES unquotes a scalar string, the array wrapper wraps per the standard (CONDITIONAL wraps only a scalar), and ON EMPTY / ON ERROR default to NULL. RETURNING is STRING-only for now (VARIANT deferred); paths are wildcard-free.

Example (j = {"id":7,"name":"Ada","tags":["x","y"],"addr":{"city":"NYC"}}):

SELECT json_query(j, '$.addr');                        -- {"city":"NYC"}
SELECT json_query(j, '$.tags');                        -- ["x","y"]
SELECT json_query(j, '$.id');                          -- 7        (scalar as JSON text)
SELECT json_query(j, '$.name' OMIT QUOTES);            -- Ada
SELECT json_query(j, '$.tags[0]' WITH ARRAY WRAPPER);  -- ["x"]
SELECT json_query(j, '$.missing' EMPTY ARRAY ON EMPTY);-- []
SELECT json_query('not json', '$.a');                  -- NULL     (NULL ON ERROR default)

Follow-ups (shared with JSON_VALUE, out of scope here): default-collation rewrite of the result, whole-stage codegen, and cross-expression parse sharing for sibling projections.

Why are the changes needed?

JSON_QUERY is the SQL-standard way to extract an object/array/scalar fragment from JSON, supported by Oracle, SQL Server, PostgreSQL, Trino, Flink, and others. Spark had no equivalent, forcing migrated queries onto get_json_object, whose NULL/error semantics differ (it conflates "missing" and "parse error", always returns STRING, and offers no wrapper/quotes control). This is the natural follow-on to JSON_VALUE.

Does this PR introduce any user-facing change?

Yes. It adds the new JSON_QUERY SQL function and its documentation, and adds JSON_QUERY, WRAPPER, QUOTES, KEEP, OMIT, CONDITIONAL, UNCONDITIONAL, and OBJECT as non-reserved keywords (usable as identifiers in both modes). No change to existing behavior.

How was this patch tested?

• New JsonQuerySuite (end-to-end): object/array/scalar extraction, JSON null, all wrapper modes, KEEP/OMIT QUOTES (incl. escapes), EMPTY ARRAY/OBJECT, ON EMPTY/ON ERROR, malformed/trailing input, RETURNING VARCHAR/CHAR normalization, direct-construction validation, and keyword-as-identifier.
• Golden files in json-functions.sql; regenerated keywords*.sql.out.
• ExpressionParserSuite for every clause spelling; SQLKeywordSuite; SparkThrowableSuite; and TableIdentifierParserSuite / JsonExpressionsSuite for regression.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 4.8)

[uros-b]

Thank you @ganeshashree!