[CALCITE-7487] ProjectJoinTransposeRule throws ArrayIndexOutOfBoundsException in PushProjector when a Join input has a zero-column row type

[microbluey]

PushProjector.locateAllRefs contains a workaround, originally added for Fennel, that arbitrarily projects the first column of a Join/SetOp input when nothing else is projected from it:

if (nProject == 0 && childPreserveExprs.isEmpty()) {
  projRefs.set(0);
  nProject = 1;
}

It assumes the input has a first column. That fails for a zero-column input — canonically DEE, the Values with an empty row type and a single empty tuple that is the identity for cross join (it arises when an Aggregate with GROUP BY () has its output pruned to zero columns). The workaround then sets a bit past the input's fields, and createProjectRefsAndExprs uses it to index into an empty field list:

java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
    at org.apache.calcite.rel.rules.PushProjector.createProjectRefsAndExprs(PushProjector.java:523)
    at org.apache.calcite.rel.rules.ProjectJoinTransposeRule.onMatch(ProjectJoinTransposeRule.java:117)

Fix

Guard each workaround on the input it protects having at least one field: nFields > 0 for the left, nFieldsRight > 0 for the right. When there is genuinely no first column, the workaround is skipped instead of producing an out-of-range reference.

Two deviations from the JIRA description, both verified

1. The left-side case is not merely latent — it crashes too. The ticket notes the symmetric nProject == 0 path "has the same latent issue" but that only the right-side variant had been seen in practice. Putting DEE on the left of the cross join throws the identical exception, so there is a regression test for each direction.

2. The suggested left-side guard (nSysFields + nFields) > 0 would not be correct. Given the field layout documented on nFields:

| nSysFields | nFields | nFieldsRight |

the left input has nFields fields, so folding nSysFields into the condition lets the workaround through when nSysFields > 0 && nFields == 0. createProjectRefsAndExprs uses offset = nSysFields for the left side, so refIdx - offset would then be negative — a different crash rather than a fix. Hence nFields > 0.

Note on the resulting plan

Skipping the workaround means the pushed-down projection over the zero-column input is itself zero-column:

LogicalJoin(condition=[true], joinType=[inner])
  LogicalProject(col1=[CAST($0):VARCHAR NOT NULL])
    LogicalValues(tuples=[[{ 0 }]])
  LogicalProject
    LogicalValues(tuples=[[{  }]])

That is precisely what the Fennel workaround existed to avoid. Fennel has long been removed, and zero-column relational expressions are legal in Calcite today — DEE itself is one — so this seems right. Flagging it explicitly in case anyone disagrees, since it is the behavioural part of this change.

Out of scope, but worth recording

projRefs.set(0) and projRefs.set(nFields) look like they should be set(nSysFields) and set(nSysFields + nFields), since the left input's fields start after the system fields. Demonstrating that needs a Join that actually has system fields, and fixing it would change which column the workaround picks, so I have deliberately kept it out of this crash fix. Happy to file it separately if it is a real issue.

The reporter also wondered whether ProjectCorrelateTransposeRule has the same latent issue. It does construct a PushProjector (ProjectCorrelateTransposeRule.java:79-80), but the workaround is gated on childRel instanceof Join || childRel instanceof SetOp, and Correlate extends BiRel — it is neither. So that rule never reaches this code and is unaffected.

Tests

Both new tests fail on current main with the ArrayIndexOutOfBoundsException above and pass with the fix. RelOptRulesTest is green (923 tests). Golden plans were produced via the standard _actual.xml mechanism.

[patientstreetlight]

Thank you for taking the time to look at this and create a fix!

I'm not a calcite committer, so I don't have any real authority to approve the change. But I still wanted to say I took a look and I think this change looks good.

You also asked to get a second opinion about whether pushing down a zero-column project is OK (i.e. whether it's fine to undo the Fennel fix for zero-column input) - this sounds fine to me. We only reach this case when the plan already has a 0-column section, so it seems fine to me to transform it to a different plan which has a different 0-column section. That said, I should mention that I am not a calcite expert, and I don't have a good understanding of the context around Fennel, so you may want to seek another opinion if you're looking for a more authoritative answer.

[mihaibudiu]

What is DEE?

[mihaibudiu]

Can this be reproduced with a quidem test?
Not all plans that the builder can build can surface from a real program.

[microbluey]

Thanks for looking. Both answered, and I've pushed a commit for the first.

"What is DEE?" — fair challenge: the abbreviation appears nowhere else in the codebase, so it shouldn't be in a comment. It's the standard relational-algebra name for the relation with no attributes and one tuple (the identity for cross join; DUM is its empty counterpart), but I've replaced it with a plain description — "a Values with an empty row type and a single empty tuple, which returns one row with zero columns" — in 269e562.

"Can this be reproduced with a quidem test?" — I tried, and I don't think it can, which I should have said in the description rather than leaving you to ask.

You're right that not every builder-constructible plan surfaces from a real program. I attempted the SQL routes to a zero-column join input:

-- (select) as u  -> parse error, not valid syntax
select cast(a.x as varchar) from (values (1)) as a(x)
cross join (select count(*) from (values (1),(2))) as b;     -- runs fine
select cast(a.x as varchar) as c from (values (1)) as a(x)
cross join (select 1 from (values (1),(2)) group by ()) as b;  -- runs fine

Both execute correctly. The aggregate keeps a column in the plan the validator produces, so the workaround always has a first column to fall back on and the crash never triggers. RelFieldTrimmer deliberately avoids handing anything a zero-column row type for the same historical reason as this workaround (see the "Fennel abhors an empty row type" comment at RelFieldTrimmer.java:1031), which is presumably why no SQL path reaches it.

So the reachable route is via RelBuilder or an equivalent frontend rather than Calcite's own SQL: the reporter hit it through substrait-java, where an Aggregate(GROUP BY (), measures) with Rel.Emit{outputMapping=[]} yields exactly this shape on one side of a cross join. Since PushProjector guards defensively elsewhere and the fix is two bounds checks, guarding here seemed worth it even though Calcite's own parser can't get there — but if you'd rather this be fixed at the frontend, or declined as unreachable, that's a reasonable call and I'm happy to close it.

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
92.9% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[mihaibudiu]

I think you can squash the commits for merging