fix(linting): don't suggest an inflected form for "more <adj> <noun>"#3793
fix(linting): don't suggest an inflected form for "more <adj> <noun>"#3793AnayGarodia wants to merge 5 commits into
Conversation
|
Updated the |
hippietrail
left a comment
There was a problem hiding this comment.
More care needs to be taken to deal with words that are both noun and adjective so that we don't block suggestions. Look at the methods starting with .is_kind - there should be one for "is but is not".
| .then_positive_adjective(), | ||
| ) as Box<dyn Expr>, | ||
| Box::new(|tok: &Token, _source: &[char]| tok.kind.is_hyphen()), | ||
| Box::new(SequenceExpr::whitespace().then_noun()), |
There was a problem hiding this comment.
This new branch's prefix overlaps with the branch's one since they both start with whitespace, which isn't necessarily wrong but I always avoid it since things like overlapping POSes are a common source of unexpected false positives and false negatives.
| 33 | as the more common plural noun. Grammatical context is one way to determine | ||
| | ^~~~~~~~~~~ This is not an error, but an inflected form of this adjective also exists | ||
| Suggest: | ||
| - Replace with: “commoner” |
There was a problem hiding this comment.
This is a problem even though it looks like a bonus. You and I might avoid using "commoner" as an adjective, but it's a legitimate adjective and noun. Hiding the suggestion this way is an unexpected side-effect that should probably be considered by using something other than .then_noun() that takes into account that it might be both a noun and an adjective.
| Lint: Style (127 priority) | ||
| Message: | | ||
| 142 | parsing (1997) that merely assigning the most common tag to each known word and | ||
| | ^~~~~~~~~~~ This is not an error, but an inflected form of this adjective also exists |
| Lint: Style (127 priority) | ||
| Message: | | ||
| 5 | **We the People** of the United States, in Order to form a more perfect Union, | ||
| | ^~~~~~~~~~~~ This is not an error, but an inflected form of this adjective also exists |
There was a problem hiding this comment.
Same again. I don't like the adjective "perfecter" but it's still a legit word some users might want.
| Message: | | ||
| 2073 | A dead man passed us in a hearse heaped with blooms, followed by two carriages | ||
| 2074 | with drawn blinds, and by more cheerful carriages for friends. The friends | ||
| | ^~~~~~~~~~~~~ This is not an error, but an inflected form of this adjective also exists |
| Message: | | ||
| 4883 | “Of course she might have loved him just for a minute, when they were first | ||
| 4884 | married—and loved me more even then, do you see?” | ||
| | ^~~~~~~~~ This is not an error, but an inflected form of this adjective also exists |
| 5659 | One of my most vivid memories is of coming back West from prep school and later | ||
| | ^~~~~~~~~~ This is not an error, but an inflected form of this adjective also exists | ||
| Suggest: | ||
| - Replace with: “vividest” |
There was a problem hiding this comment.
Same again but unlike the others where the suggested adjectives all seemed ugly, "vividest" is a suggestion I would definitely consider.
There may be even better adjective suggestions this change prevents that we haven't thought of.
|
switched to heads up on the corpus fallout: three style suggestions come back that |
|
follow-up — the refined it to head-detection: keep |
hippietrail
left a comment
There was a problem hiding this comment.
I think we need more tests to be sure this new logic does what it's intended to do rather than just seeming to work by coincidence. With a very strong preference for real-world examples in the tests rather than contrived ones made up by an AI. Made up by a human would lie somewhere between.
| Box::new( | ||
| SequenceExpr::whitespace() | ||
| .then_noun() | ||
| .then_unless(SequenceExpr::whitespace().then_noun()), |
There was a problem hiding this comment.
This seems confusing but it looks like it now matches "more common noun" but not "more common noun noun"? However, "more common noun" followed by something other than whitespace or "more common noun " followed by something other than a noun will still actually be matched?
When rules start to get confusing with exceptions to exceptions etc. you definitely will want more than a single test case so that you can chase down each one of these loose ends to make sure we're not getting the desired results by fluke.
Unfortunately UnlessStep, .unless() and .then_unless() all lack any tests to clarify what they are and are not intended to do.
|
added the tests. all three are real sentences pulled straight from the corpus (harper-core/tests/text/), not made up: "a more perfect Union" from the Constitution, "loved me more even then" from Gatsby, and "the more common plural noun" from the part-of-speech tagging article. grep for them if you want to confirm. the first two suppress (noun head, and "then" not being a noun), the third still suggests commoner since "plural" is modifying the following "noun" rather than being the head. that contrast is the whole point of the head check, so if it ever regresses one of these three will catch it. |
MoreAdjective flagged "more short videos" and suggested "shorter videos".
When the adjective modifies a following noun, "more" quantifies the noun
phrase ("more [short videos]") rather than intensifying the adjective, so
the inflected form changes the meaning. Add a following noun as a third
recognized false-positive shape in the rule's optional trailing clause,
alongside the existing "than <adj>" (Automattic#2925) and hyphen (Automattic#3568) cases.
Closes Automattic#3688.
The MoreAdjective fix removes awkward inflected-form suggestions where
"more/most" is followed by an adjective+noun ("more common plural noun"
-> commoner, "a more perfect Union" -> perfecter, "more cheerful
carriages" -> cheerfuller, etc.). Regenerate the affected corpus
snapshots to reflect the removed false positives.
…oun guard then_noun() also matched words tagged as both noun and adjective, which suppressed the inflection suggestion for those (e.g. more common plural). switch to then_kind_is_but_is_not(is_noun, is_adjective) so a genuine noun head still suppresses (more short videos) but noun+adjective homographs don't block the suggestion. refs Automattic#3688.
…se head
is-noun-but-not-adjective was too blunt: it un-suppressed every noun+adjective
word, which re-surfaced bad suggestions on non-gradable adjectives (perfecter,
evener). instead keep then_noun() but require the noun be the head - if another
noun follows it (then_unless), the word is adjectival ("more common plural
noun") and the comparative suggestion stays. "more short videos" still
suppresses; perfecter/evener no longer appear. refs Automattic#3688.
three sentences already in harper-core/tests/text: "a more perfect Union" (Constitution) and "more even then" (Gatsby) stay suppressed because the trailing word is the noun head; "the more common plural noun" (a POS article) still suggests "commoner" because "plural" modifies a following noun. the suppress-vs-suggest contrast shows the head check isn't coincidental. refs Automattic#3688.
b6f8b29 to
86ec915
Compare
The problem is that three is still a tiny tiny sample. Ideally we'd be looking outside our tiny internal corpus into GitHub, Stack Overflow, Hacker News, Slashdot, Medium, Reddit where bad English abounds as well as good English. I do this with Google to varying levels of success. I understand AIs can't do this due to |
What
Fixes #3688.
MoreAdjectiveflagged "more short videos" and suggested the inflected form "shorter videos".Why
When the adjective is immediately followed by a noun it modifies, "more" quantifies the whole noun phrase ("more [short videos]") rather than intensifying the adjective, so rewriting to the comparative changes the meaning. This is the same "look one token past the adjective to spot a false positive" approach the rule already uses for the
than <adjective>case (#2925) and the hyphen case (#3568).How
Add
whitespace + nounas a third alternative in the rule's existing optional trailing clause (FirstMatchOf). When it matches, the span is longer than three tokens and the existingtoks.len() != 3guard inmatch_to_lintalready aborts — no new control flow. Existing true positives are unaffected because they aren't followed by a modified noun ("more fast", "more cute", "more good", "more humane than …").Testing
Added
dont_flag_more_short_videos_3688(1 lint before, 0 after). Full rule suite green, no regressions:Compatibility
Purely narrows one Style-level false positive; no public API change.