Skip to content

docs: Add more documentation about PartialSortExec operator - #24048

Merged
alamb merged 5 commits into
apache:mainfrom
alamb:alamb/partial-sort-exec-docs
Aug 3, 2026
Merged

docs: Add more documentation about PartialSortExec operator#24048
alamb merged 5 commits into
apache:mainfrom
alamb:alamb/partial-sort-exec-docs

Conversation

@alamb

@alamb alamb commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

N/A

Rationale for this change

I was trying to explain to someone recently how PartialSortExec works, and I found the
datafusion docs weren't quite as explicit about the befits as I wanted

What changes are included in this PR?

  1. Update the PartialSortExec documentation to explain the streaming and memory implications better, and add a worked buffering example

Are these changes tested?

By CI

Are there any user-facing changes?

Better docs

@alamb alamb added the documentation Improvements or additions to documentation label Aug 1, 2026
@alamb

alamb commented Aug 1, 2026

Copy link
Copy Markdown
Contributor Author

FYI @NGA-TRAN per our disucssion

@github-actions github-actions Bot added physical-plan Changes to the physical-plan crate and removed documentation Improvements or additions to documentation labels Aug 1, 2026
@codecov-commenter

codecov-commenter commented Aug 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.87%. Comparing base (f9dde71) to head (2d3fa42).

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #24048      +/-   ##
==========================================
- Coverage   80.88%   80.87%   -0.01%     
==========================================
  Files        1102     1102              
  Lines      375813   375813              
  Branches   375813   375813              
==========================================
- Hits       303966   303953      -13     
- Misses      53745    53752       +7     
- Partials    18102    18108       +6     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@2010YOUY01 2010YOUY01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, this is a great doc improvement.

Comment thread datafusion/physical-plan/src/sorts/partial_sort.rs Outdated

@NGA-TRAN NGA-TRAN left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sense. Thanks Andrew

Co-authored-by: Yongting You <2010youy01@gmail.com>
@alamb alamb added the documentation Improvements or additions to documentation label Aug 3, 2026
Comment thread datafusion/physical-plan/src/sorts/partial_sort.rs Outdated
@github-actions github-actions Bot removed the documentation Improvements or additions to documentation label Aug 3, 2026
@alamb
alamb enabled auto-merge August 3, 2026 20:40
@alamb

alamb commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

Thank you for the review @2010YOUY01 and @NGA-TRAN

@alamb alamb added the documentation Improvements or additions to documentation label Aug 3, 2026
@github-actions github-actions Bot removed the documentation Improvements or additions to documentation label Aug 3, 2026
@alamb
alamb added this pull request to the merge queue Aug 3, 2026
Merged via the queue into apache:main with commit 882d906 Aug 3, 2026
40 checks passed
@alamb
alamb deleted the alamb/partial-sort-exec-docs branch August 3, 2026 21:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

physical-plan Changes to the physical-plan crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants