Skip to content

Add reverberation-trust agent - #147

Open
jeffreyhaole wants to merge 2 commits into
microsoft:mainfrom
jeffreyhaole:add-reverberation-trust
Open

Add reverberation-trust agent#147
jeffreyhaole wants to merge 2 commits into
microsoft:mainfrom
jeffreyhaole:add-reverberation-trust

Conversation

@jeffreyhaole

Copy link
Copy Markdown

Summary

Adds a new catalog agent, reverberation-trust, under agents/reverberation-trust/. It is a trust-triage reviewer for AGN reverberation-mapping (RM) continuum-to-line time lags: given a lag and its light curves it returns a calibrated trust verdict — the lag, a confidence tier (refuse/low/medium/high), and reviewable reasoning — by cross-checking independent estimators, arbitrating their disagreement with sampling facts (not more estimators), and corroborating with independent Look and Channel witnesses. It does not compute lags better than JAVELIN/PyCCF; it answers the question those codes omit: should you believe this number?

Type of change

  • New agent (under agents/<name>/)
  • Update to an existing agent
  • New starter kit (under starter-kits/<name>/)
  • Update to an existing starter kit
  • Schema change (docs/schemas/**)
  • Workflow / script change (.github/**)
  • Documentation only
  • Other (describe below)

Related issue / tracking

Executive Challenge: Hack for Science submission.

Schema impact

  • No schema changes
  • Schema changes are backward compatible
  • Schema changes are breaking — add the breaking-change label and link the rollout plan below

Validation checklist

  • Documentation (README.md, schemas) is updated to match the change.
  • No hand-edits to .auto-registry/** (it is generated post-merge).
  • No OS / editor artefacts committed (.DS_Store, .env, *.swp, .idea/, .vs/, etc.).
  • If model-weight files are added, they are Git-LFS tracked, ≤5 GB each, and in an allowed format.
  • No secrets, customer data, or internal-only URLs are committed.
  • All Markdown links resolve.

Reviewer notes (optional)

  • Party: 1p — publisher Microsoft Discovery (matches sibling 1p agents).
  • Passes the repo's own validator (.github/scripts/validate_pr.py) locally — all checks green.
  • Tests: the compute library ships 22 dual-runnable tests (pytest and python <file>.py); 21 pass, 1 intentional xfail (physical-window slack).
  • Estimator honesty: the DRW and JAVELIN-style estimators are independent, from-scratch re-implementations of the published methods (numpy + scipy) used as cross-checking witnesses — not the original authors' software. This is called out in Known Limitations and the References section, which credits the method lineage (Blandford & McKee; Gaskell & Peterson; Edelson & Krolik; Chelouche; Kelly/MacLeod; Zu et al.; Sun et al.; Timmer & König). A maintained/compiled JAVELIN is the intended production replacement at survey scale.
  • Excluded on purpose: a local MCP dev-harness (reverbtrust_mcp.py) and Discovery-App wiring are not in the PR — the catalog runs the tool via its container per tool.yaml, whose contract is light-curves-in → verdict-out. A local harness that fetches real SDSS-RM data (astroquery→VizieR) and reproduces the RMID 781 recovery end-to-end is available on request.
  • Live-validated end-to-end in the Discovery App on real SDSS-RM data (VizieR): the flagship RMID 781 false-confidence case is caught and downgraded to low by an independent witness.

One-page brief

Field Value
Scientific user AGN reverberation-mapping & disk-RM teams (SDSS-RM, OzDES, Swift / AGN STORM), and Rubin/LSST-era AGN RM at scale.
Bottleneck Trustworthy lags are vetted by hand, object-by-object, with inconsistent rigor — which does not scale to the Rubin flood of thousands of candidates.
Hypothesis A calibrated trust verdict, assembled from independent witnesses, can auto-certify the trustworthy lags and route the rest to humans with explained reasons.
Evidence / tool Five estimators (ICCF, DCF, von Neumann, DRW, JAVELIN-style) + a witness/trust layer (sampling-window fact-arbiter, red-noise & aliasing gates, cross-band/line/instrument Look witnesses, the X-ray-timing Channel cascade).
Baseline Raw CCF peak / single-method / current manual per-paper vetting.
Result Grier 2012: 5/5 Hβ lags at mean |Δ| = 1.3 d. SDSS-RM (44 AGN): median |Δ| = 5.3 d with confidence flags that separate accuracy; RMID 781 false-confidence caught and corrected by an independent witness; RMID 707 floor reported honestly. Channel axis: X-ray→UV→optical cascade on 5 AGN, X-ray coupling graded per object. Two orthogonal witnesses (M–σ, X-ray flux) fail for the predicted reason — the model is validated by its negatives.
Expected value Expert-grade, uncertainty-honest trust triage applied consistently at scale and usable below the resident-expert tier.
Evaluation Known-answer regression suite (documented same-data reversals), leakage-ablated; executed SDSS-RM scale run (accuracy-separating flags, trust recovery); expert-confirmed novel catch as stretch.

@github-actions github-actions Bot added needs-human-review Awaiting human approval 1p-contribution Authored by Microsoft (party='1p' in metadata) pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. labels Sep 3, 2026
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

✅ Automated Check Results — All checks passed

All structural, schema, content, documentation, and secret scan checks passed.

This PR is ready for human review. The maintainers have been automatically requested.

Reminder: 1 approval from a CODEOWNERS reviewer is required before this PR can be merged.

@jeffreyhaole

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree company="Microsoft"

@jeffreyhaole

Copy link
Copy Markdown
Author

Honest note because I forgot to ask GHCP to add it, but these are the only words I've actually typed in this PR. Everything submitted is 100% generated by GHCP running on Claude Opus 4.8, often referencing earlier work I've done with GHCP running Claude Sonnet 4.6. The ideas, expansions, connections, redirections, the successes and failures on both our parts was real, we each brought something unique.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1p-contribution Authored by Microsoft (party='1p' in metadata) needs-human-review Awaiting human approval pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant