Skip to content

[BugFix] Fix FILTER(WHERE) dropped on aggregates in unified SQL path#5523

Merged
Swiddis merged 1 commit into
opensearch-project:mainfrom
dai-chen:fix/sql-filter-aggregate-calcite
Jun 8, 2026
Merged

[BugFix] Fix FILTER(WHERE) dropped on aggregates in unified SQL path#5523
Swiddis merged 1 commit into
opensearch-project:mainfrom
dai-chen:fix/sql-filter-aggregate-calcite

Conversation

@dai-chen

@dai-chen dai-chen commented Jun 8, 2026

Copy link
Copy Markdown
Collaborator

Description

This PR fixes aggregate FILTER(WHERE ...) clauses being silently dropped in the unified SQL path, where the predicate was ignored when building the Calcite AggCall. The filter is now applied to the aggregate call and its referenced columns are preserved through the pre-aggregation trimming projection, with no impact on PPL.

Related Issues

Part of #5248

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • New functionality has javadoc added.
  • New functionality has a user manual doc added.
  • New PPL command checklist all confirmed.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff or -s.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@dai-chen dai-chen self-assigned this Jun 8, 2026
@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

PR Reviewer Guide 🔍

(Review updated until commit 874dff8)

Here are some key observations to aid the review process:

🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ No major issues detected

@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
General
Prevent duplicate column references

The method getAggregateFilterInputRefs may add duplicate RexInputRef entries when
multiple aggregates reference the same column in their filter predicates. Consider
using a Set or deduplicating the results to avoid redundant column references in the
trimming projection.

core/src/main/java/org/opensearch/sql/calcite/CalciteRelNodeVisitor.java [1808-1818]

 private List<RexInputRef> getAggregateFilterInputRefs(
     List<UnresolvedExpression> aggExprList, CalcitePlanContext context) {
-  List<RexInputRef> refs = new ArrayList<>();
+  Set<RexInputRef> refs = new LinkedHashSet<>();
   for (UnresolvedExpression aggExpr : aggExprList) {
     AggregateFunction aggFunc = extractAggregateFunction(aggExpr);
     if (aggFunc != null && aggFunc.condition() != null) {
       refs.addAll(PlanUtils.getInputRefs(rexVisitor.analyze(aggFunc.condition(), context)));
     }
   }
-  return refs;
+  return new ArrayList<>(refs);
 }
Suggestion importance[1-10]: 6

__

Why: The suggestion correctly identifies that duplicate RexInputRef entries could be added when multiple aggregates reference the same column in their filter predicates. Using a Set to deduplicate is a valid optimization that improves efficiency, though the code would still function correctly with duplicates. The impact is moderate as it's an optimization rather than a bug fix.

Low

CalciteAggCallVisitor ignored AggregateFunction.condition, so
COUNT(*) FILTER(WHERE age > 30) degraded to COUNT(). Apply the
predicate via AggCall.filter() (wrapped in IS TRUE), and retain its
input columns in the pre-aggregation trimming projection so they
re-resolve. PPL is unaffected (it never sets condition).

Signed-off-by: Chen Dai <daichen@amazon.com>
@dai-chen dai-chen force-pushed the fix/sql-filter-aggregate-calcite branch from 48ba7d9 to 874dff8 Compare June 8, 2026 19:54
@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 874dff8

@Swiddis Swiddis merged commit f12e4c3 into opensearch-project:main Jun 8, 2026
62 of 63 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants