Skip to content

Register the public MEOS functions alone on the Spark surface - #136

Merged
estebanzimanyi merged 2 commits into
MobilityDB:mainfrom
estebanzimanyi:codegen/register-public-functions-only
Oct 3, 2026
Merged

estebanzimanyi merged 2 commits into
MobilityDB:mainfrom
estebanzimanyi:codegen/register-public-functions-only

Conversation

@estebanzimanyi

Copy link
Copy Markdown
Member

Read an enum argument from its text through the public parser

An argument of a C enum travels in SQL as the text its name is, and both
JVM SQL surfaces turn that text into the enum through the catalog function
returning that enum from one string. The rule is one: SqlModel._enum_parsers
in codegen_jvm.py and ENUM_PARSER in codegen_spark_udfs.py both admit a
public parser alone (null_handle_type_from_string, interptype_from_string,
raquet_pixtype_from_string). In the Spark arm an enum argument is a string
whose conversion the four call builders now apply, as they pass every other
scalar argument through its expression.

Witness. Against MobilityDB d29d4ab293 and the catalog of MEOS-API
5ba0cd8492, the Spark arm refuses tjsonb_to_tbigint, public since
MobilityDB 21a15203ba, and every other function taking a nullHandleType or
an interpType, as an argument it cannot read, which the gaps ledger records;
the Flink arm reads an interpType through interptype_from_string, which the
catalog stated internal until MobilityDB 671ed23326.

Why. A binding calls the public API alone, and a public parser tests its
argument at entry, so a null or unknown name is reported to the engine
rather than read. Flink already passed an enum as its text; Spark refused
every function taking one, so the two surfaces differed on the constructors
and conversions that take an interpolation or a null handling.

Measured. From the same catalog and jar, against the generator that
answers every overload of a Spark name whose argument classes differ, the
Flink surface this branch generates is byte-identical, and Spark registers
3,724 names where that generator registers 3,646: 78 added and none removed, among them tintSeq,
tfloatSeqSet and the other Seq and SeqSet constructors,
setInterp, tsample, appendInstant, the tjsonb_to_* conversions and the
JSON functions taking a null handling; tnpoint gains its sequence
constructors from a base value and a time span. The gaps ledger loses 33
functions and lists 711, none new; the two parsers leave it as the
conversions the surface calls.

Register the public MEOS functions alone on the Spark surface

The Spark arm admits a function by the catalog's api field, as the Flink
arm does: a function is registered only when api is public. It tested the
name of the Doxygen group instead, and a function stating no group, which
the catalog states internal, passed that test.

Witness. Against MobilityDB 91efcc4721 and the catalog of MEOS-API
5ba0cd8492, the Spark surface JMEOS main generates calls 139 functions the
catalog states internal and registers 137 of them under their C names,
among them geo_clip_subject, geo_is_planar_areal, interptype_name,
pc_compression_name and the pg_date_, pg_interval_ and pg_numeric_*
families, while the Flink surface calls none.

Why. A binding projects the public API, the functions whose @InGroup is a
public group; an internal function carries no contract a binding may rely
on, asserting its preconditions where a public one tests them. MobilityDuck
reads the same field for the same reason.

Measured. From the same catalog and jar, Spark registers 3,587 names where
main registers 3,724: 137 removed, none added, and no remaining name
changes the functions behind it. The Flink surface is unchanged and the
gaps ledger lists the same 711 functions. The one internal function the
Spark surface still calls is numeric_out, the text writer the catalog's
typeEncodings names for NumericData, through which the public function
returning a numeric is serialized.

An argument of a C enum travels in SQL as the text its name is, and both
JVM SQL surfaces turn that text into the enum through the catalog function
returning that enum from one string. The rule is one: SqlModel._enum_parsers
in codegen_jvm.py and ENUM_PARSER in codegen_spark_udfs.py both admit a
public parser alone (null_handle_type_from_string, interptype_from_string,
raquet_pixtype_from_string). In the Spark arm an enum argument is a string
whose conversion the four call builders now apply, as they pass every other
scalar argument through its expression.

Witness. Against MobilityDB d29d4ab293 and the catalog of MEOS-API
5ba0cd8492, the Spark arm refuses tjsonb_to_tbigint, public since
MobilityDB 21a15203ba, and every other function taking a nullHandleType or
an interpType, as an argument it cannot read, which the gaps ledger records;
the Flink arm reads an interpType through interptype_from_string, which the
catalog stated internal until MobilityDB 671ed23326.

Why. A binding calls the public API alone, and a public parser tests its
argument at entry, so a null or unknown name is reported to the engine
rather than read. Flink already passed an enum as its text; Spark refused
every function taking one, so the two surfaces differed on the constructors
and conversions that take an interpolation or a null handling.

Measured. From the same catalog and jar, against the generator that
answers every overload of a Spark name whose argument classes differ, the
Flink surface this branch generates is byte-identical, and Spark registers
3,724 names where that generator registers 3,646: 78 added and none removed, among them tintSeq,
tfloatSeqSet and the other <type>Seq and <type>SeqSet constructors,
setInterp, tsample, appendInstant, the tjsonb_to_* conversions and the
JSON functions taking a null handling; tnpoint gains its sequence
constructors from a base value and a time span. The gaps ledger loses 33
functions and lists 711, none new; the two parsers leave it as the
conversions the surface calls.
The Spark arm admits a function by the catalog's api field, as the Flink
arm does: a function is registered only when api is public. It tested the
name of the Doxygen group instead, and a function stating no group, which
the catalog states internal, passed that test.

Witness. Against MobilityDB 91efcc4721 and the catalog of MEOS-API
5ba0cd8492, the Spark surface JMEOS main generates calls 139 functions the
catalog states internal and registers 137 of them under their C names,
among them geo_clip_subject, geo_is_planar_areal, interptype_name,
pc_compression_name and the pg_date_*, pg_interval_* and pg_numeric_*
families, while the Flink surface calls none.

Why. A binding projects the public API, the functions whose @InGroup is a
public group; an internal function carries no contract a binding may rely
on, asserting its preconditions where a public one tests them. MobilityDuck
reads the same field for the same reason.

Measured. From the same catalog and jar, Spark registers 3,587 names where
main registers 3,724: 137 removed, none added, and no remaining name
changes the functions behind it. The Flink surface is unchanged and the
gaps ledger lists the same 711 functions. The one internal function the
Spark surface still calls is numeric_out, the text writer the catalog's
typeEncodings names for NumericData, through which the public function
returning a numeric is serialized.
@estebanzimanyi
estebanzimanyi merged commit 22c400d into MobilityDB:main Oct 3, 2026
2 checks passed
@estebanzimanyi
estebanzimanyi deleted the codegen/register-public-functions-only branch October 3, 2026 07:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant