Repository navigation
agent-core: the wrap-up at max_steps is the only behavior; delete mask-breakeven - #244
Merged
Merged
Conversation
jaredLunde
enabled auto-merge
October 12, 2026 11:41
github-merge-queue
Bot
removed this pull request from the merge queue due to a conflict with the base branch
Oct 12, 2026
…k-breakeven Wrap-up (#232's --max-steps-wrap-up, re-tested with a tighter text): at a max_steps ceiling the model now always gets WRAP_UP_STEPS (5) more steps, told to stop cleanly: no new code and no finishing a change in progress, fix or revert only a file that no longer parses or imports, run the test suite once, summarize. A model that stops within the grant ends the run normally; one that uses it up still gets Error::MaxSteps naming the configured ceiling. - Luna, sqlparse + marshmallow at --max-steps 60, 3 runs a side: 6/6 wrap-up runs ended with an honest summary (control: 6 silent cut-offs), 0 broken trees on either side, $0.298 vs $0.300. The grant made no source edit in 5 of 6 runs and one targeted fix in the sixth; the arms' mean fixed fraction (0.634 vs 0.679) differs by what happened before the ceiling. - The old text ("fix or revert anything broken, start no new work") let a run keep implementing stubs during the grant and end with 171 regressions; the new one forbids continuing the task. Deleted: StepBudget, Agent::with_step_budget, --max-steps-wrap-up (an unknown argument now). AgentEvent::StepBudget becomes WrapUp { steps }. mask-breakeven (#234): Haiku sqlparse + marshmallow, 4 runs a side, mean fixed fraction 0.971 -> 0.898 (a paired drop past the 0.03 bar), compactions 21 -> 27, masks 10 -> 2, $ -9%. Its counterfactual was wrong: a pass runs only at the trigger, so skipping it means summarizing now, not saving the rewrite; one run's post-compaction context collapsed and it quit at 20%. Deleted with growth_per_turn and mask_net_saving; mask-breakeven is an unknown candidate now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk
jaredLunde
enabled auto-merge
October 12, 2026 12:07
jaredLunde
force-pushed
the
jared/wrap-up-and-mask
branch
from
October 12, 2026 12:07
fda1e03 to
e60cf2c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two decisions from the re-tests of #232 and #234.
Wrap-up at
max_steps: shipped as the only behaviorAt a
max_stepsceiling the model now always getsWRAP_UP_STEPS(5) more steps, with a tighter instruction: no new code and no finishing a change in progress; fix or revert only a file that no longer parses or imports; run the test suite once; summarize. A model that stops within the grant ends the run normally; one that uses it up still getsError::MaxStepsnaming the configured ceiling.--max-steps-wrap-upandStepBudget/with_step_budgetare deleted;AgentEvent::StepBudgetbecomesWrapUp { steps }.Why the tighter text: in round 1 one sqlparse run kept implementing stubs during the grant (
def process,__repr__) and ended with 171 regressions; those edits were new work, not repairs.Re-test, Luna, sqlparse + marshmallow at
--max-steps 60, 3 runs a side:The grant made no source edit in 5 of 6 runs (compile check and/or one full test run, then the summary) and one targeted fix in the sixth, so it did not lower any run's fixed fraction; the arms' mean gap comes from the 60 steps before the ceiling, where both arms run the same code.
mask-breakeven: deletedHaiku, sqlparse + marshmallow, 4 runs a side:
A paired drop past the 0.03 bar. The rule's counterfactual was wrong: a masking pass runs only at the trigger, so skipping it means summarizing now (a whole-prefix rewrite plus lost verbatim context), not saving the rewrite. One run's post-compaction context collapsed and it quit at 20%. Pricing the real alternative would converge on always masking, i.e. the control. Deleted with
growth_per_turnandmask_net_saving;mask-breakevenis an unknown candidate now.state/tailstay: the in-flight summary eval builds on them.🤖 Generated with Claude Code
https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk