Generate a native ORCA plan for a replicated CTE in scalar subqueries#384
Open
Alena0704 wants to merge 2 commits into
Open
Generate a native ORCA plan for a replicated CTE in scalar subqueries#384Alena0704 wants to merge 2 commits into
Alena0704 wants to merge 2 commits into
Conversation
Contributor
Author
|
I decided to create about new pull request because we fixed that problem before with fall back walker and now I am developing approach making orca builds the query plan for cross slice scalar subqueries without breaking the semantics of checking cross slice pattern for other components with replicated tables (the regression test proved it). FYI, the previous PR to fix this topic and discussion #370 |
Alena0704
force-pushed
the
bugfix-cross-slice-shared-scan-cte
branch
from
82f0b49 to
26c2063
Compare
Alena0704
changed the title
Alena0704
changed the title
Alena0704
force-pushed
the
bugfix-cross-slice-shared-scan-cte
branch
from
26c2063 to
8eb4869
Compare
Alena0704
force-pushed
the
bugfix-cross-slice-shared-scan-cte
branch
from
5fd1d75 to
ac442fe
Compare
A CTE over a DISTRIBUTED REPLICATED table referenced from multiple scalar subqueries makes orca place the SharedScan Producer and Consumer on different slices. The cross-slice SharedScan hang itself is already prevented: FHasCrossSliceReplicatedCTEConsumer detects this topology before DXL translation and forces a fallback to the Postgres planner. This change extends orca to produce a correct native plan for the scalar-subquery case, so it no longer needs the Postgres-planner fallback for it: orca materializes the replicated CTE once and shares it across all references within its own plan. Repair in apply_shareinput_xslice (src/backend/cdb/cdbmutate.c): when a cross-slice Consumer is found inside a SubPlan and the CTE source is a replicated table, give the Consumer a local copy of the Producer's subtree (Materialize + base Scan) with a fresh share_id and SHARE_MATERIAL, co-located in the Consumer's slice. Because the table is replicated, every segment already holds the full data, so the local copy is equivalent and removes the cross-slice coordination. Other Consumers of the same CTE in the same slice reuse this copy (tracked by (orig_share_id, motId) -> new_share_id), so the CTE is materialized once and read by all references; cleanup_orphaned_producers then drops the now-unconsumed original Producers. New ApplyShareInputContext fields in src/include/nodes/relation.h hold the reuse map and consumer counts. The pre-DXL check that forces the fallback (CUtils::FHasCrossSliceReplicatedCTEConsumer) was too broad: it fired for every cross-slice replicated CTE Consumer. Narrow it to fire only for the join case -- a CTE Consumer under a duplicate-hazard / broadcast Motion (greengage 51fe92e) -- which still cannot be handled natively. The scalar-subquery case is no longer caught here, so it reaches the new materialization above instead of falling back.
Alena0704
force-pushed
the
bugfix-cross-slice-shared-scan-cte
branch
from
ac442fe to
92486db
Compare
Alena0704
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a CTE over a DISTRIBUTED REPLICATED table is referenced from several
scalar subqueries, ORCA puts the SharedScan Producer and Consumer on
different slices. That cross-slice SharedScan used to hang.
Until now we just avoided the hang: FHasCrossSliceReplicatedCTEConsumer
detected this shape before DXL translation and fell back to the Postgres
planner. This change lets ORCA handle the scalar-subquery case natively,
so no fallback is needed: the replicated CTE is materialized once and
shared by all references inside ORCA's own plan.
The fix is in apply_shareinput_xslice (src/backend/cdb/cdbmutate.c). When
a cross-slice Consumer is found inside a SubPlan and the CTE source is a
replicated table, the Consumer gets its own local copy of the Producer's
subtree (Materialize + base Scan) with a fresh share_id, marked
SHARE_MATERIAL and placed in the Consumer's slice. The table is
replicated, so every segment already has the full data and the local copy
is equivalent -- this removes the cross-slice coordination. Other
Consumers of the same CTE in the same slice reuse this copy (tracked by
(orig_share_id, motId) -> new_share_id), so the CTE is materialized once
and read by all references. cleanup_orphaned_producers then drops the
original Producers that are no longer used. The reuse map and consumer
counts live in new ApplyShareInputContext fields in
src/include/nodes/relation.h.
The pre-DXL fallback check (CUtils::FHasCrossSliceReplicatedCTEConsumer)
was too broad -- it fired for every cross-slice replicated CTE Consumer.
Narrow it to the join case only: a CTE Consumer under a
duplicate-hazard / broadcast Motion (greengage 51fe92e), which still
can't be handled natively. The scalar-subquery case no longer matches
here and reaches the new materialization path instead.