Skip to content

refactor: make apply_expression_roots more ergonomic - #24226

Draft
jayshrivastava wants to merge 1 commit into
apache:mainfrom
jayshrivastava:js/as-ref-refactor
Draft

refactor: make apply_expression_roots more ergonomic#24226
jayshrivastava wants to merge 1 commit into
apache:mainfrom
jayshrivastava:js/as-ref-refactor

Conversation

@jayshrivastava

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

  • Closes #.

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions Bot added physical-expr Changes to the physical-expr crates proto Related to proto crate datasource Changes to the datasource crate physical-plan Changes to the physical-plan crate labels Aug 10, 2026
/// [`TreeNodeRecursion::Stop`] stops iteration and is returned immediately.
/// [`TreeNodeRecursion::Jump`] is normalized to [`TreeNodeRecursion::Continue`]
/// because this function does not visit expression children.
pub fn apply_expression_roots<I>(

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally what we want is this

pub fn apply_expression_roots<I>(
    roots: I,
    f: &mut dyn FnMut(&Arc<dyn PhysicalExpr>) -> Result<TreeNodeRecursion>,
) -> Result<TreeNodeRecursion>
where
    I: IntoIterator,
    I::Item: AsRef<Arc<dyn PhysicalExpr>>,
{
    for root in roots {
        match f(root.as_ref())? {
            TreeNodeRecursion::Stop => return Ok(TreeNodeRecursion::Stop),
            TreeNodeRecursion::Continue | TreeNodeRecursion::Jump => {}
        }
    }
    Ok(TreeNodeRecursion::Continue)
}

However, AsRef<Arc<dyn PhysicalExpr>> is surprisingly implemented for Arc<dyn PhysicalExpr>. We cannot add this implementation due to the orphan rule. AsRef is foreign and Arc is foreign.

This PR gets around the problem by adding a new type, PhysicalExprRoot but that new type makes this refactor seem less worthwhile.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is AsRef<dyn PhysicalExpr> for Arc<dyn PhysicalExpr> though but that requires changing the apply_expressions API to traverse over &dyn PhysicalExpr instead of &Arc<dyn PhysicalExpr>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think as long as it is clear what the traits are for and it makes downstream code easier to copy/paste/ work it would be ok

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.00000% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.99%. Comparing base (f4c8ba1) to head (f7203ae).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
datafusion/datasource-avro/src/source.rs 0.00% 1 Missing ⚠️
datafusion/datasource-json/src/source.rs 0.00% 1 Missing ⚠️
datafusion/physical-plan/src/execution_plan.rs 90.90% 0 Missing and 1 partial ⚠️
datafusion/proto/src/physical_plan/mod.rs 0.00% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main   #24226   +/-   ##
=======================================
  Coverage   80.98%   80.99%           
=======================================
  Files        1106     1106           
  Lines      383232   383215   -17     
  Branches   383232   383215   -17     
=======================================
- Hits       310372   310368    -4     
+ Misses      54543    54525   -18     
- Partials    18317    18322    +5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.


impl Eq for ProjectionExpr {}

impl AsRef<Arc<dyn PhysicalExpr>> for ProjectionExpr {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe worth adding a comment here explaining what this does

/// [`TreeNodeRecursion::Stop`] stops iteration and is returned immediately.
/// [`TreeNodeRecursion::Jump`] is normalized to [`TreeNodeRecursion::Continue`]
/// because this function does not visit expression children.
pub fn apply_expression_roots<I>(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think as long as it is clear what the traits are for and it makes downstream code easier to copy/paste/ work it would be ok

.map(|proj_expr| &proj_expr.expr),
f,
)
crate::apply_expression_roots(self.projector.projection().as_ref().iter(), f)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it does look nicer

Though I admit perhaps the magic required to make it work reduces some of its value

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

datasource Changes to the datasource crate physical-expr Changes to the physical-expr crates physical-plan Changes to the physical-plan crate proto Related to proto crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants