Byrow-aware literal handling for @subset / @rsubset (#259)#420
Open
kimjune01 wants to merge 4 commits into
Open
Byrow-aware literal handling for @subset / @rsubset (#259)#420kimjune01 wants to merge 4 commits into
kimjune01 wants to merge 4 commits into
Conversation
Add support for literal boolean values (and other literals) in @subset by implementing a catch-all fun_to_vec method that wraps literals in Returns. This allows expressions like @subset(df, true) or @subset(df, false) to work correctly. The fix uses Returns to create a constant function from literal values, which is only allowed in no_dest contexts (@subset, @rsubset, @with). Tests added to verify: - @subset(df, true) returns all rows - @subset(df, false) returns empty dataframe - Literals combine correctly with other conditions
The original fix used `[] => Returns(true)` which returns a scalar Bool. DataFrames.subset wraps non-ByRow functions with assert_bool_vec, which throws ArgumentError when the function returns a non-vector value. Fix: use `[] => ByRow(Returns(ex))` instead. ByRow bypasses the assert_bool_vec check (assert_bool_vec(::ByRow) = identity) and DataFrames' _empty_selector_helper calls the inner function once per row, producing a proper boolean vector. Also use $(Base.Returns) interpolation to avoid depending on caller scope resolution. Adds tests for @subset!, grouped DataFrames with literals.
Collaborator
|
Thanks! But I'm not 100% sure this is right. |
A bare literal like `@subset(df, true)` lowers to a scalar predicate, which `DataFrames.subset` rejects exactly as `subset(df, [] => Returns(true))` does. Accepting it (via `ByRow(Returns(...))`) made the macro diverge from the function and would mean different things across macros (`@select(df, 1)` vs `@subset(df, true)`). Replace the prior MethodError from JuliaData#259 with a clear ArgumentError pointing at the column-expression form. Tests assert the expansion-time error.
kimjune01
changed the title
Author
|
Good call. Reworked it to raise a clear ArgumentError instead of broadcasting, so the macro stays consistent with subset(df, [] => Returns(true)); if you want literals as a real feature later, that's the scoped allow_literals / ^(true) path from #259. |
pdeffebach
reviewed
May 22, 2026
| outer_flags::Union{NamedTuple, Nothing}=nothing, | ||
| allow_multicol::Bool = false) | ||
| if no_dest | ||
| return :([] => $ByRow($(Base.Returns)($ex))) |
Collaborator
There was a problem hiding this comment.
Don't you need to have ByRow here only if byrow = true?
Author
There was a problem hiding this comment.
seems like my fix was overly permissive. I was hoping tests passing would be good enough, but I was wrong. Added tests for both modes so to increase coverage.
ByRow only applies under the @byrow flag, so behaviour mirrors the underlying subset in both modes: @subset(df, true) -> errors, like subset(df, [] => Returns(true)) @rsubset(df, true) -> all rows, like subset(df, [] => ByRow(Returns(true))) Applying ByRow unconditionally (prior approach) made @subset accept what the function rejects. Tests now cover both modes; master had none for literal predicates either way.
kimjune01
changed the title
Collaborator
|
Can we allow literals for non-by-row subsetting as well? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
@subset(df, true)previously threw an opaqueMethodError(nofun_to_vecdispatch for literals), which is the bug in #259.This handles literals in a way that mirrors the underlying
DataFrames.subsetexactly, keyed on the@byrowflag:@subset(df, true)(not byrow): a bare literal is a scalar predicate, whichsubsetrejects just assubset(df, [] => Returns(true))does. It now raises a clearArgumentErrorin place of theMethodError.@rsubset(df, true)(byrow): a per-row literal is well defined and matchessubset(df, [] => ByRow(Returns(true))), which works, so it returns the expected rows.ByRowis applied only when the@byrowflag is set. Applying it unconditionally (the first approach here) made@subsetaccept what the function rejects, which is the divergence raised in review.Test plan
@subset(df, true)/false/ grouped /@subset!raise at macro expansion (asserted via@test_throws LoadError @eval ..., matching the suite's convention for invalid expansions).@rsubset(df, true)returns all rows,@rsubset(df, false)returns none, including the grouped case.@subset(df, :A .> 1)) unaffected.There was no test coverage for literal predicates in either mode before, so this adds it for both.