Skip to content

[Chore] Replace the multi-group Measure row merger with a batch merger #14068

Description

@hanahmily

Summary

Merge vectorized Measure results without first converting them into individual InternalDataPoint objects. Follow-up to #14066 under #13998.

Size audit and dependencies

Classification: tracking parent, not an executable leaf or automated implementation task.
Boundary: the production multi-group Measure caller of MergeGroupMIterators and its sortableDataPoints / sortedMIterator stack.
This spans batch consumption, caller/egress migration, schema alignment, ordering/version deduplication, and resource ownership. Decompose after the legacy removal merges.
Blocked by: #14066 via BanyanDB PR apache/skywalking-banyandb#1326.

Requirements

R1. Consume and merge per-group batches directly; do not reconstruct the row merger internally.
R2. Preserve requested ordering, version-based deduplication, schema alignment, and pagination.
R3. Materialize protobuf only where the response boundary requires it; preserve tracing and response compatibility.
R4. Release consumed batches correctly on completion, cancellation, and errors.

Concrete oracle

Two groups supply these records, with duplicate versions split across batches:

timestamp=10, SID=7, version=1, value=100
timestamp=10, SID=7, version=3, value=300
timestamp=20, SID=8, version=1, value=200

Ascending output is (10,7,3,300) followed by (20,8,1,200). Descending reverses that order. Ascending with offset 1 and limit 1 returns only (20,8,1,200).

Completion criteria and RED-test direction

  • The real processor multi-group caller consumes the batch-native merger.
  • Tests assert the exact oracle across batch boundaries, plus index ordering and supported schema alignment.
  • A path-sensitive production-caller regression fails if row iteration/materialization is restored. Output assertions alone are insufficient: the existing merger already produces these answers.
  • Each executable child identifies its public boundary and failing test before implementation; no helper-only foundation PR counts as completion.
  • Cancellation/error tests verify resource release; benchmark comparisons report allocations without claiming a new resource bound that has not been proved.

Packages: banyand/query, pkg/query/logical/measure, pkg/query/vectorized/measure.

go test ./banyand/query ./pkg/query/logical/measure ./pkg/query/vectorized/measure/...
go test ./test/integration/standalone/query/... ./test/integration/distributed/query/...

Out of scope

Changing ordinary Top or GroupBy semantics; the internal TopN pre-aggregation RPC; schema-evolved aggregate partial reduction (separate workstream).

Parent: #13998.

Compatibility and review policy

  • Preserve existing on-disk and wire formats and mixed-version compatibility. Any necessary protocol change requires a separately reviewed compatibility contract.
  • Use explicit, hand-calculated expected results; do not depend on the removed row engine as the test oracle.
  • Human review enforces the no-new-row-path policy. A row-call-site lint guard is not required.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    databaseBanyanDB - SkyWalking native database

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions