[FLINK-37925][table] Support casting from VARIANT to primitive types by raminqaf · Pull Request #28733 · apache/flink

raminqaf · 2026-07-13T12:18:02Z

What is the purpose of the change

Add CAST and TRY_CAST from VARIANT to primitive scalar types (FLIP-521, sub-task FLINK-37925). Previously a VARIANT could only be read as another variant or rendered to a JSON string, with no way to extract a typed scalar out of it.

Brief change log

Add VariantToPrimitiveCastRule (registered in CastRuleProvider) covering BOOLEAN, all integer/floating/decimal numerics,
BINARY/VARBINARY, DATE, TIMESTAMP and TIMESTAMP_LTZ.
Numeric targets are lenient: any numeric variant converts following the regular numeric cast semantics. Non-numeric targets require the stored value to match the target kind.
On a type mismatch CAST fails the job and TRY_CAST returns NULL.
Permit the cast at planner validation time via a VARIANT source branch in LogicalTypeCasts (explicit only).
Out of scope: CHARACTER_STRING (already served by the JSON display rule and JSON_STRING), TIME (no counterpart in the variant type model), and constructed types ARRAY/ROW/STRUCTURED (tracked in FLINK-37926).

Verifying this change

This change added tests and can be verified as follows:

CastRulesTest: variant → each supported scalar, covering success, type mismatch (CAST throws), and null inputs.
CastRuleProviderTest: the rule resolves for supported targets.
LogicalTypeCastsTest: explicit-cast support matrix, including that TIME/STRING/ARRAY and all implicit casts are rejected.
CastFunctionITCase: end-to-end CAST/TRY_CAST over PARSE_JSON(...).

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? yes
If yes, how is the feature documented? docs (VARIANT casting section in data-types.md) and JavaDocs

Was generative AI tooling used to co-author this PR?

Yes

Opus 4.8

Add CAST/TRY_CAST from a VARIANT to primitive scalar types. Numeric targets are lenient: a variant holding any numeric kind converts to the requested numeric type with regular numeric cast semantics, so a JSON integer stored in the smallest fitting type still widens (e.g. CAST(PARSE_JSON('42') AS INT)). Non-numeric targets require the stored value to be of the matching kind. On a type mismatch CAST fails the job and TRY_CAST returns NULL. Character strings and TIME are out of scope: strings are handled by the display-oriented VariantToStringCastRule and via JSON_STRING, and the variant type model has no TIME kind. Constructed types (ARRAY, ROW, STRUCTURED) are left to FLINK-37926.

…types Cover cast validation (LogicalTypeCastsTest), rule resolution (CastRuleProviderTest), the generated cast executor including numeric widening/narrowing and type-mismatch failures (CastRulesTest), and an end-to-end CAST/TRY_CAST over PARSE_JSON (CastFunctionITCase).

Mark VARIANT as an explicit cast source for scalar targets in the casting matrix and describe the lenient numeric behavior. Note that TIME and character-string targets are unsupported.

flinkbot · 2026-07-13T12:23:25Z

CI report:

0ebbdb1 Azure: PENDING

Bot commands

The @flinkbot bot supports the following commands:

@flinkbot run azure re-run the last Azure build

davidradl · 2026-07-13T12:50:12Z

+            case TIMESTAMP_WITH_LOCAL_TIME_ZONE:
+                return true;
+            default:
+                // TIME has no counterpart in the Variant type model. CHARACTER_STRING is handled by


I wonder if the cast is to a string then we could issue a helpful exception message indicating which SQL functions the user should use.

Even though this is a good idea and helpful for the user. Implementing it is a bit messy. Still I kept the changes for you to review. Since it adds lots of nose in the method. But let me know what you think. See getUnsupportedCastHint and CastInputTypeStrategy.

davidradl · 2026-07-13T12:55:14Z

+        final CodeWriter writer = new CastRuleUtils.CodeWriter();
+        switch (targetLogicalType.getTypeRoot()) {
+            case BOOLEAN:
+                writer.assignStmt(returnVariable, methodCall(inputTerm, "getBoolean"));


Do variants support nullable and non nullable concepts - should a cast of of boolean when the value is null work?

As far as I can see in the implementation of VARIANT type the get methods (here getBoolean) return a boolean and not an nullable Boolean.

Whereas NULL type is supported in VARIANT type. I have added logic for this and more tests to cover NULL casts and nullability.

snuyanzin · 2026-07-13T14:12:03Z

+                Arguments.of(new VariantType(), new DateType(), false, true),
+                Arguments.of(new VariantType(), new TimestampType(), false, true),
+                Arguments.of(new VariantType(), new LocalZonedTimestampType(), false, true),
+                Arguments.of(


is interval intentionally skipped?

Variant.Type has no interval kind. Please refer to: https://cwiki.apache.org/confluence/spaces/FLINK/pages/349637099/FLIP-521+Integrating+Variant+Type+into+Flink+Enabling+Efficient+Semi-Structured+Data+Processing#FLIP521:IntegratingVariantTypeintoFlink:EnablingEfficientSemiStructuredDataProcessing-Casting

snuyanzin · 2026-07-13T14:41:51Z

+                        .testResult(
+                                call("PARSE_JSON", "42").cast(BIGINT()),
+                                "CAST(PARSE_JSON('42') AS BIGINT)",
+                                42L,
+                                BIGINT().notNull())
+                        .testResult(


dummy question: what will happen with Double.NaN, Double.NEGATIVE_INFINITY, Double.POSITIVE_INFINITY while transforming to VARIANT and vice versa?

I got an exception on the Double.NEGATIVE_INFINITY

Caused by: org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonParseException: Non-standard token '-Infinity': enable `JsonReadFeature.ALLOW_NON_NUMERIC_NUMBERS` to allow at [...]

I like it in every places it fails differently 🙈

…_STRING for VARIANT-to-string casts A variant that stores a JSON null now casts to SQL NULL instead of failing in the type-specific accessor, when the cast target is nullable (a nullable variant source, or TRY_CAST which forces the target nullable). A NOT NULL target still fails on a null-valued variant because the result cannot carry NULL; making VARIANT casts unconditionally nullable would require changes to Calcite return-type inference and is left out of scope. Casting a VARIANT to a character string now reports a helpful validation error from the Table API type inference that points to JSON_STRING. The SQL path keeps Calcite's generic cast error.

raminqaf added 3 commits July 13, 2026 14:03

[FLINK-37925][docs] Document casting from VARIANT to primitive types

cca9d90

Mark VARIANT as an explicit cast source for scalar targets in the casting matrix and describe the lenient numeric behavior. Note that TIME and character-string targets are unsupported.

davidradl reviewed Jul 13, 2026

View reviewed changes

snuyanzin reviewed Jul 13, 2026

View reviewed changes

Comment thread ...able/flink-table-common/src/test/java/org/apache/flink/table/types/LogicalTypeCastsTest.java

snuyanzin reviewed Jul 13, 2026

View reviewed changes

Comment thread ...able/flink-table-common/src/test/java/org/apache/flink/table/types/LogicalTypeCastsTest.java Outdated

snuyanzin reviewed Jul 13, 2026

View reviewed changes

Comment thread ...table-planner/src/test/java/org/apache/flink/table/planner/functions/CastFunctionITCase.java

snuyanzin reviewed Jul 13, 2026

View reviewed changes

Comment thread ...table-planner/src/test/java/org/apache/flink/table/planner/functions/CastFunctionITCase.java Outdated

raminqaf force-pushed the FLINK-37925 branch from 2ce0f39 to 82a7b47 Compare July 14, 2026 10:18

[FLINK-37925] Add more tests in LogicalTypeCastsTest

0ebbdb1

raminqaf force-pushed the FLINK-37925 branch from 82a7b47 to 0ebbdb1 Compare July 14, 2026 13:00

Uh oh!

Conversation

raminqaf commented Jul 13, 2026

What is the purpose of the change

Brief change log

Verifying this change

Does this pull request potentially affect one of the following parts:

Documentation

Was generative AI tooling used to co-author this PR?

Uh oh!

flinkbot commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CI report:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

raminqaf Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

flinkbot commented Jul 13, 2026 •

edited

Loading

raminqaf Jul 14, 2026 •

edited

Loading