Skip to content

[FLINK-40240][runtime] Support TRY_CAST, IFNULL and NULLIF in YAML Transform - #4492

Open
haruki-830 wants to merge 1 commit into
apache:masterfrom
haruki-830:FLINK-40240
Open

[FLINK-40240][runtime] Support TRY_CAST, IFNULL and NULLIF in YAML Transform#4492
haruki-830 wants to merge 1 commit into
apache:masterfrom
haruki-830:FLINK-40240

Conversation

@haruki-830

Copy link
Copy Markdown
Contributor

Summary

This commit adds TRY_CAST, IFNULL, and NULLIF support to YAML Transform projections and filters. It aligns their behavior with Flink SQL while preserving the existing CAST failure semantics.

Key Changes

  1. Function and Syntax Support
  • Added operator and type metadata for TRY_CAST, IFNULL, and NULLIF.
  • Added preprocessing for TRY_CAST(expr AS type) because Calcite 1.32 does not support this syntax directly.
  • Handles nested expressions, different letter cases, quoted text, identifiers, and SQL comments.
  1. Code Generation and Runtime Semantics
  • Added Janino code generation for all three functions.
  • TRY_CAST supports the same target types as the existing CAST implementation and returns NULL only for data conversion failures.
  • Unsupported conversion paths fail during pipeline validation, while invalid configuration, UDF, and internal errors continue to fail the task.
  • Added null-safe IFNULL and numeric-aware/deep-equality NULLIF evaluation without changing existing CAST behavior.
  1. Test Coverage and Documentation
  • Added parser, code generation, runtime, and YAML specification coverage for projections and filters.
  • Covered successful and failed casts, nested expressions, nullability, invalid syntax, and function combinations.
  • Verified the changes with both default Flink 1.20 and the Flink 2 profile.
  • Updated the English and Chinese Transform documentation.

JIRA Reference

https://issues.apache.org/jira/browse/FLINK-40240

@github-actions github-actions Bot added docs Improvements or additions to documentation composer runtime labels Jul 28, 2026
@haruki-830
haruki-830 marked this pull request as ready for review July 28, 2026 06:31
@haruki-830

Copy link
Copy Markdown
Contributor Author

@lvyanquan @yuxiqian PTAL, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

composer docs Improvements or additions to documentation runtime

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant