Skip to content

Recall: selection, search, pinning, and a stated message budget - #10

Merged
senamakel merged 9 commits into
mainfrom
recall
Sep 1, 2026
Merged

Recall: selection, search, pinning, and a stated message budget#10
senamakel merged 9 commits into
mainfrom
recall

Conversation

@senamakel

@senamakel senamakel commented Sep 1, 2026

Copy link
Copy Markdown
Member

What and why

A turn reads a bounded window of a desk whose log is unbounded, so the decision
settled two hundred messages ago is invisible and gets re-litigated. Enlarging
the window moves the cliff and charges everyone for it. This lands the other
answer: make the transcript queryable, keep a small pinned working set
arriving regardless, and state the budget each message spends.

Five pieces, no new port and no new stored state:

  • tinyhivemind_core::select — one fixed-point ranking used by every picker,
    so an agent search and a desk search cannot disagree about what a better match
    is. Tiers (Exact 1000 → Subsequence 200) plus a density term worth at most
    100, so density orders within a tier and never promotes past one.
  • tinyhivemind_core::findagents, people, desks over the snapshots
    a turn already holds. Retired agents are never offered; a desk description is
    scored at half weight.
  • tinyhivemind::searchsearch_messages and search_threads, a bounded
    backward walk through the existing SessionLog port. A desk-scoped search
    reads the desk's whole interior, thread replies included: the projection is
    narrow so a turn stays readable, and search exists to reach the reply buried
    three deep.
  • tinyhivemind::pins!pin [^N] [#label] [note] / !unpin ^N, folded
    out of the log rather than stored beside it, so there is no second journal.
    The board rides into every turn through SessionContext.
  • BrevityPolicy — a stated per-message budget (600 characters against the
    30-message window), rendered into the briefing next to the pin and search
    spellings. Reported, never enforced: nothing here rewrites an authored
    message.

Regular expressions are supported behind an off-by-default regex feature. A
query written /…/ is an expression; an expression hit is read onto the same
tiers from the span the engine matched, so literal and expression hits rank in
one comparable list. Without the feature a Regex pattern is
Error::RegexUnsupported, and one that does not compile is
Error::InvalidPattern — never a silent literal fallback.

Spec: docs/specs/recall.md. Roadmap: P14, and the
entry says why it lands out of order.

Public API and behavior changes

Additive, but not zero:

  • New modules tinyhivemind_core::{select, find} and tinyhivemind::{search, pins},
    re-exported from each crate root.
  • New error variants Error::InvalidPattern and Error::RegexUnsupported.
  • TeamBriefing gains brevity: BrevityPolicy and SessionContext gains
    pins: Vec<Pin>. Both are #[serde(default)], so an older stored record
    still decodes; struct-literal constructions in host code need the new field.
  • TeamBriefing::system_text() gains two rule lines (the budget, and the pin
    spelling). SessionContext::system_text() renders the board between the
    thread index and host notes.
  • initialize_session_with_context now performs one additional bounded read
    (PIN_SCAN) for the pinboard, and passes the query's before so the board
    is bounded to the same snapshot as the transcript.
  • initialize_session and initialize_session_with_context take briefing by
    value and overwrite briefing.brevity.window with query.window: the budget
    the briefing states must match the window the query actually reads.
  • read_pinboard(log, conversation, limit, before) carries the exclusive
    sequence bound as its fourth argument.
  • New optional regex feature on both crates, off by default.

Validation

command outcome
cargo fmt --all -- --check pass
cargo clippy --all-targets --all-features -- -D warnings pass
cargo build --all-targets --all-features pass
cargo test --all-features pass — 331 tests, 0 failures
cargo test -p tinyhivemind (no features) pass — the feature-off RegexUnsupported path
cargo clippy --all-targets -- -D warnings pass
RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features pass
.github/scripts/assert-pure.sh clean
cargo llvm-cov --all-features 99.09% total; new files 96.7–100%

Deliberately untested edges: the whole.max(1) guard in select::ratio is
unreachable through any public path (a matched text is never empty) and exists
so a future caller cannot divide by zero.

Docs

docs/specs/recall.md, module READMEs for search/ and pins/, a Recall
wiki page linked from Home and the sidebar (pushed to the wiki repo; the
pointer bump is in this branch), and a row in README.md's read-more table.

Review round

Five findings from the Codex reviewer, all confirmed and fixed, each with a
regression test written to fail before the fix:

  • pins were read from the head of the log rather than the query's snapshot
    (before now threaded through read_pinboard / read_desk_rows);
  • a search excerpt could land off a match when lowercasing changed a
    character count (offsets are now mapped back to the original string);
  • two pins deposited in one message could reorder (the fold now tracks a
    directive ordinal, not pinned_at alone);
  • the briefing stated the default window rather than the query's;
  • a code fence was treated as closed by a shorter run of the same character.

Also merged origin/main into the branch and resolved the resulting conflict
(wiki pointer, README.md).

senamakel and others added 4 commits September 1, 2026 15:29
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Co-authored-by: Medulla <medulla@tinyhumans.ai>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-01T13:02:21.176959Z ba85128 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Sep 1, 2026

Copy link
Copy Markdown

Warning

Review limit reached

  • Run on-demand review

On-demand reviews are free for the next 19 days. After that, they cost $0.25 per reviewed file.

Or wait 6 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: dbcb527f-5fbe-4ddb-826d-5290b6333ac2

📥 Commits

Reviewing files that changed from the base of the PR and between 2aaaf02 and ba85128.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (32)
  • Cargo.toml
  • README.md
  • ROADMAP.md
  • crates/tinyhivemind-core/Cargo.toml
  • crates/tinyhivemind-core/src/find/mod.rs
  • crates/tinyhivemind-core/src/find/test.rs
  • crates/tinyhivemind-core/src/lib.rs
  • crates/tinyhivemind-core/src/select/mod.rs
  • crates/tinyhivemind-core/src/select/test.rs
  • crates/tinyhivemind-core/src/select/types.rs
  • crates/tinyhivemind-core/tests/public_api.rs
  • crates/tinyhivemind/Cargo.toml
  • crates/tinyhivemind/src/briefing/mod.rs
  • crates/tinyhivemind/src/briefing/test.rs
  • crates/tinyhivemind/src/briefing/types.rs
  • crates/tinyhivemind/src/error/mod.rs
  • crates/tinyhivemind/src/lib.rs
  • crates/tinyhivemind/src/pins/README.md
  • crates/tinyhivemind/src/pins/mod.rs
  • crates/tinyhivemind/src/pins/test.rs
  • crates/tinyhivemind/src/pins/types.rs
  • crates/tinyhivemind/src/search/README.md
  • crates/tinyhivemind/src/search/mod.rs
  • crates/tinyhivemind/src/search/test.rs
  • crates/tinyhivemind/src/search/types.rs
  • crates/tinyhivemind/src/threads/mod.rs
  • crates/tinyhivemind/tests/public_api.rs
  • docs/research/README.md
  • docs/research/long-context.md
  • docs/specs/README.md
  • docs/specs/recall.md
  • wiki

Warning

Your free Security trial is over. An organization admin can upgrade to Advanced for continuous pull request security review or dismiss this notice.


Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 39137959d9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/tinyhivemind/src/briefing/mod.rs Outdated
Comment thread crates/tinyhivemind/src/search/mod.rs
Comment thread crates/tinyhivemind/src/pins/mod.rs Outdated
senamakel and others added 2 commits September 1, 2026 15:41
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Co-authored-by: Medulla <medulla@tinyhumans.ai>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 07cdbc9a82

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/tinyhivemind/src/briefing/types.rs
Comment thread crates/tinyhivemind/src/pins/mod.rs Outdated
@senamakel

Copy link
Copy Markdown
Member Author

PR Babysitter status

Head: ba85128196151b00d14ee5e2de227c56a0465839 (merge of main into recall)

CI: all required checks green — Rust, Docs, Minimum supported Rust version, Supply chain, CodeRabbit.

Mergeable: MERGEABLE / CLEAN (merge conflict with main resolved via a normal merge commit, no rebase/force-push).

Fixes pushed this cycle

  • 07cdbc9 — clippy question_mark lint fix in tinyhivemind_core::select::score (CI toolchain 1.98.0 flags this; repo's default local toolchain did not).
  • a00102c — bound the pinboard read to the query's snapshot (read_desk_rows/read_pinboard now take before: Option<Sequence>), fixed within-message pin ordering with a directive ordinal, and fixed a lowercase-offset-to-original-text mapping bug in search excerpting (original_offset). All three are Codex P1/P2 findings; each has a regression test verified to fail without the fix.
  • e5f86a7 — reconciled TeamBriefing.brevity.window with SessionQuery.window in both initialize_session and initialize_session_with_context, and fixed the pin-marker fence parser to require a closing fence at least as long as the opener (fence_run). Both are Codex P2 findings; each has a regression test.
  • ba85128 — merge commit resolving a conflict with main (main added only README.md/wiki-pointer changes since the branch point; no Rust conflicts).

Review threads

All 5 Codex-flagged threads replied to in-thread with the fixing commit + validating test, and resolved. 0 unresolved threads remain. No changes-requested reviews. CodeRabbit review-limited but check reports SUCCESS.

Test evidence

Contract commands (cargo fmt --all -- --check, cargo clippy --all-targets --all-features -- -D warnings on both the default toolchain and CI's 1.98.0, cargo build --all-targets --all-features, cargo test --all-features) all pass, plus cargo test -p tinyhivemind (no features), MSRV build (1.88), RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features, and .github/scripts/assert-pure.sh.

Next action

Ready for pr-approval-reviewer.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ba85128196

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chat_id: message.chat_id.clone(),
parent: message.parent,
author: message.author.clone(),
excerpt: excerpt(line, original_offset(line, matched.offset)),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve regex offsets when building excerpts

When the regex feature is enabled, score_regex reports an offset in the original text, but this call always treats it as an offset in lowercased text. For example, with 200 İ characters before a /needle/ match, the valid offset 200 is remapped to 100 and the 96-character excerpt ends before the match. Fresh evidence after the prior offset fix is that the new remapping is unconditional even though only literal matching lowercases its input; distinguish the pattern type before applying it.

Useful? React with 👍 / 👎.

let Some((char, len)) = fence else { continue };
match open {
None => open = Some((start, char, len)),
Some((from, opener, opener_len)) if opener == char && len >= opener_len => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Require a bare closing fence before parsing directives

When a fenced code block contains a same-length fence with an info suffix, such as a ```rust line inside a backtick block, CommonMark treats that line as content rather than a closing fence, but this branch closes solely on character and run length. A following !pin or !unpin example is consequently parsed as a live directive and can mutate the board. Fresh evidence after the earlier fence-length fix is that the closing predicate still never verifies that the remainder of the line contains only whitespace.

Useful? React with 👍 / 👎.

| `Pin` | one board entry: target, pinner, `pinned_at`, label, note, excerpt |
| `read_directives(body, author, sequence)` | the grammar |
| `fold_pins(rows, limit)` | the pure fold, over a chronological slice |
| `read_pinboard(log, conversation, limit)` | the fold plus its bounded read |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Add the snapshot bound to the documented pinboard API

The public-surface table still advertises read_pinboard(log, conversation, limit), but the exported function now requires a fourth before argument. A host implementing the integration from this module documentation gets a compile error and is not told how to preserve the query snapshot; update this table and the recall specification to show and explain the bound.

AGENTS.md reference: AGENTS.md:L286-L289

Useful? React with 👍 / 👎.

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes: 1 lane(s) blocking, worst finding is high.

Fix or reply to the findings below and push. The next review clears this automatically once they are gone — you should not need to dismiss anything by hand.

             $0.1260 · 879,855 in / 18,849 out · 132,008 cached (15%) · openrouter/openai/text-embedding-3-small, z-ai/glm-5.2, deepseek/deepseek-v4-flash · 792 embedded
critique:    $0.0608 · 440,228 in / 11,574 out · 48,594 cached (11%)  · z-ai/glm-5.2, deepseek/deepseek-v4-flash
security:    $0.0558 · 324,234 in / 7,070 out  · 83,414 cached (26%)  · deepseek/deepseek-v4-flash, z-ai/glm-5.2
tests:       $0.0050 · 61,833 in  / 109 out    · 0 cached (0%)        · deepseek/deepseek-v4-flash
description: $0.0043 · 53,560 in  / 96 out     · 0 cached (0%)        · deepseek/deepseek-v4-flash

@tinysweeper

tinysweeper Bot commented Sep 1, 2026

Copy link
Copy Markdown

How this change flows

0 changed behaviours across 12 relationships. 5 surrounding behaviours are shown (60 graph nodes walked). 42 further behaviours left out to keep the diagram readable.

flowchart LR
  n0["named_conversation"]:::impacted
  n1["...ide_a_thread_and_propagates_read_failures"]:::impacted
  n2["initialize_session_with_context"]:::impacted
  n3["...hread_index_and_host_notes_beside_history"]:::impacted
  n4["OnePage"]:::impacted
  n1 -->|calls| n0
  n1 -->|tests| n0
  n1 -->|calls| n2
  n1 -->|tests| n2
  n1 -->|calls| n4
  n1 -->|tests| n4
  n3 -->|calls| n0
  n3 -->|tests| n0
  n3 -->|calls| n2
  n3 -->|tests| n2
  n3 -->|calls| n4
  n3 -->|tests| n4
  classDef changed fill:#0d4429,stroke:#238636,color:#e6edf3
  classDef impacted fill:#161b22,stroke:#6e7681,color:#c9d1d9
  classDef flagged fill:#5a1e02,stroke:#d93f0b,color:#ffffff
  classDef blocking fill:#67060c,stroke:#f85149,color:#ffffff
Loading

Green: changed behaviour. Grey: surrounding behaviour. Arrows name the call, use, implementation, or test relationship. Orange: has findings. Red: has a finding that blocks the merge.

tinysweeper 0.1.0

@senamakel
senamakel merged commit 78b3ffc into main Sep 1, 2026
14 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant