feat(agent): curb concept/entity sprawl (strict entity types, name-length gate, pending buffer) - #248
Open
sebastianbraun25 wants to merge 5 commits into
Conversation
added 5 commits
September 7, 2026 10:29
Adds concept_update_mode (default "rewrite", unchanged) with a new "append" mode: instead of sending an existing concept/entity page's full body to the LLM for a rewrite, generates a short 1-2 sentence note (the LLM never sees the existing page) and appends it deterministically under a "## Notes" heading, keyed by source document so re-ingesting an updated document replaces its own line instead of duplicating it. Removal cleanup and config docs updated accordingly. Resolves VectifyAI#245
VectifyAI#245) Adds a new `openkb consolidate [PAGE_NAME] [--all] [--min-notes N] [--dry-run] [--yes]` command (mirrors `recompile`'s CLI shape) and a new `openkb/agent/consolidator.py` module. For concept/entity pages accumulating notes under `concept_update_mode: append`, this folds the pending "## Notes" section into curated prose with a single LLM call per page — no new source document, no concept/entity classification, since the page is already fixed. - Contradictions between notes (or notes vs. existing prose) are described directly in the rewritten text rather than silently resolved. - The full wikilink whitelist is sent here (once per page per run, not per ticket) so consolidated prose can properly cross-link to other pages. - The "## Notes" section is replaced entirely; `sources:`/`type:` are preserved, only `description:` may be refreshed. A later `append_*_note` call re-creates a fresh "## Notes" section, so the next consolidation run only ever sees what changed since the last one — no extra tracking state needed. - Manual, opt-in only (no automatic trigger during ingest).
… gate, and a pending-topics buffer Problem: the plan step creates a dedicated wiki page for almost anything proposed on its first mention, including a document's own ticket/case identifier and overly specific one-off names. entity_types only labeled the "type" field and never gated whether an entity got created at all. Changes: - config: opt-in `strict_entity_types` (default false) drops an entity whose type doesn't match entity_types instead of coercing it to "other". - compiler: hard cap of 3 words on brand-new concept/entity names (_count_words), applied to "create" items only. - compiler: __DOC_TOKEN_GUIDANCE__ prompt substitution — a concrete, per-document "propose at most N brand-new items" suggestion computed from the document's real token count (reused from the existing summary/overview call's usage, no extra API call). Purely textual guidance, never enforced in code. - new openkb/pending.py (PendingTopicsStore): a brand-new concept/entity now collects up to 2 short notes across documents before a real page is created on the 3rd mention. Works in both concept_update_mode values — "append" mode promotion reuses the existing note-append writers with no extra LLM call; "rewrite" mode does one enriched create call using the buffered notes as context, then merges all contributing sources. - pending topics are surfaced to the plan call as quasi-existing briefs (for dedup) but never added to the wikilink whitelist. Tests: strict_entity_types resolver, max-words filter (concepts+entities), _doc_token_guidance, PendingTopicsStore unit tests, and two end-to-end lifecycle tests (rewrite/append) proving 3 mentions across documents are needed before a page exists. Updated existing create-path tests for the new buffer-then-promote semantics — 0 regressions against the pre-existing Windows-environment baseline (18 known failures, unchanged). Resolves VectifyAI#247
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
This PR was created in collaboration between a human and AI: implementation, tests, and PR
text were created by an AI assistant under the guidance and review of the human author.
Problem
The concept/entity plan step creates a dedicated wiki page for almost anything the LLM proposes
on its very first mention — including a document's own ticket/case identifier and overly specific
one-off names (e.g. "person name + their role in this one case"). For ingest workloads with many
short, similar documents (support tickets, case records), this produces a large number of low-value
entity/concept pages that are never referenced again.
entity_types:(a configurable per-KBvocabulary for the entity
typefield) is currently only used to label entities, not to gatewhether they get created at all — a mismatched type silently falls back to
"other".Solution / Changes
strict_entity_types(new opt-in config key, defaultfalse): whentrue, an entity whoseLLM-returned type doesn't match the configured
entity_typesvocabulary is dropped instead ofcoerced to
"other". Backward compatible (default preserves today's behavior exactly).-/_/whitespace) is dropped — a lightweight, deterministic proxy for "too specific to bereusable knowledge" (unique keys, hashes, ticket numbers, and multi-part combinations all tend
to produce long names). Applies only to
createitems, never toupdate(an already-existing,already-vetted name/type).
_CONCEPTS_PLAN_USER): a reuse-first framing, a concrete per-document"propose at most N brand-new concepts+entities" suggestion computed from the document's actual
token count (reusing the existing summary/overview call's
usage, no extra API call), and aname-generality rule. Purely textual — never enforced in code.
openkb/pending.py,PendingTopicsStore): a brand-newconcept/entity now collects up to 2 short notes across documents before a dedicated page is
created on the 3rd mention, instead of getting a page on the very first mention. Works in both
concept_update_modevalues:"append"mode promotion reuses the existingappend_concept_note/append_entity_notewriters (from feat(agent): opt-in append-note concept_update_mode to avoid full-page rewrites on ingest #245/feat(agent): opt-in append-note concept_update_mode to avoid full-page rewrites on ingest #246) with no extra LLM call — they already create-if-absent.
"rewrite"mode promotion does one enriched create call, using the buffered notes as extracontext, then merges all contributing document sources into the new page's frontmatter.
update/related over a near-duplicate create") but are never added to the wikilink whitelist —
no page exists for them yet.
Dependencies
Depends on #246(fix/issue-245-append-note-concept-update-mode, not yet merged) — thepending-buffer's
"append"mode promotion reusescompiler_notes.append_concept_note/append_entity_noteintroduced there.Testing
New tests:
strict_entity_typesresolver, the max-words filter (concepts + entities,_parse_entities_planthreading),_doc_token_guidance,PendingTopicsStoreunit tests (add/promote/remove/persistence/brief formatting), and two end-to-end lifecycle tests (rewrite/append)
proving 3 mentions across separate documents are required before a page exists, with all
contributing sources merged. Existing create-path tests updated for the new buffer-then-promote
semantics.
Issues
Resolves #247