Skip to content

Clarify expensive-run exploration and sweep uncertainty - #1

Merged
wassname2 merged 1 commit into
wassname:dev4from
wassname2:skill-uplift-expensive-runs
Sep 8, 2026
Merged

Clarify expensive-run exploration and sweep uncertainty#1
wassname2 merged 1 commit into
wassname:dev4from
wassname2:skill-uplift-expensive-runs

Conversation

@wassname2

Copy link
Copy Markdown
Collaborator

Changes

  • Keep the full anti-skimming form; distinguish it from the brief end message the user reads.
  • Explain batching changes in expensive runs using distinct predicted effects, possible interactions, and conditional updates to the Bayesian mental model.
  • Replace the sweep appendix's fixed t-statistic verdicts with raw paired comparisons and uncertainty. Distinguish exploratory decisions from claims of reliable superiority.

Targets dev4. Only SKILL.md and references/sweeps.md change.

Checks

git diff --check passed. A fresh-agent scenario check allowed batched changes, rejected artificial certainty from within-seed normalization, and retained the full form. It placed the form in the linked run log, so whether it must also appear in chat remains unresolved.

No actual training run or measured skill-uplift evaluation was performed.

Authored by Pi from wassname's requested changes.

Preserve the full anti-skimming form and distinguish it from the brief end message. Describe batched changes with distinct predicted effects and conditional Bayesian updates. Replace normalized-score reliability verdicts with raw paired comparisons and explicit uncertainty.

Validation: diff --check; fresh-agent design scenarios exercised batching, normalization and full-form requirements. No actual training run or measured uplift test. Scenario agent placed the full form in the linked run log, leaving chat-display interpretation unresolved.

Co-Authored-By: Pi <288921227+claudypoo@users.noreply.github.com>
@wassname2
wassname2 merged commit 76d93f2 into wassname:dev4 Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant