Skip to content

Latest commit

 

History

History
829 lines (705 loc) · 53 KB

completed.md

File metadata and controls

829 lines (705 loc) · 53 KB

Plan: improve Morpheus against the LXX reference corpus

Working files live in tmp/lxx/ (git-ignored): compare.py (harness, dual-pass: accented + accent-insensitive), report.txt, misses.tsv, mismatches.tsv, common_misses.tsv.

Context / findings (from comparison harness, 623,685 tokens / 48,995 distinct forms)

  • Misses: 5.15% of tokens — ~92% are proper names absent from the stemlib (Ισραηλ ×2956, Δαυιδ ×1091, Ιερουσαλημ ×867, …). Also real common-word gaps: whole verb εὐλογέω missing (only its compounds exist in lsj.vbs), noun σάββατον missing, plus scattered forms (λήμψεται, ἤλθοσαν, δῴη, ἐκέκραξα…).
  • Mismatches: 4.54% — mostly harmless lemma-convention differences (the corpus lemmatizes compounds to the base verb), but includes real bugs documented below.
  • Robustness bug: cruncher silently returns nothing for uppercase beta code (E)N vs e)n) — exactly how the reference corpus is encoded.

Tasks

  • 1. Add scripts/build_lxx_stemlib.py (generator, mirrors build_biblical_stemlib.py: idempotent via a managed-section marker + load_existing() merge; --include-recognized for full-coverage recovery). Reads the reference corpus's *.mlxx files, runs Morpheus over distinct forms, emits stemlib/Greek/stemsrc/nom.lxx containing: - proper names the corpus marks as such (type N / capitalized lemma) that Morpheus misses → indeclinable pers_name entries with attested case/number/gender from its parse codes; two-letter masc/fem names kept, two-letter neuter tokens dropped as artifacts; - curated missing common words: σάββατον, ταμιεῖον, βροῦχος (generative) and ἀμνός, ἀδελφιδός, τρυβλίον (explicit :wd: paradigms — the generative patterns do not produce these forms).
  • 2. Add base verbs εὐλογέω (eu)loge/w) and ἐγγίζω (e)ggi/zw) to stemlib/Greek/stemsrc/lsj.vbs at their sorted positions.
  • 3. Fix uppercase-beta-code handling in python/morpheus_toolkit/runner.py (_normalize_beta, Greek only — Latin is case-sensitive) + unit tests in python/tests/test_runner.py.
  • 4. Rebuild stemlib via scripts/build-stemlib.sh; verified: names resolve (Ἰσραηλ→Ἰσραήλ, Δαυιδ→Δαυίδ, Ωγ…), common words resolve (σαββάτων→σάββατον, ἀμνοῖς→ἀμνός, …), εὐλόγησεν/ἐγγίζει → correct lemmas, uppercase E)N/QEO/S analyze through the toolkit.
  • 5. Test suite: 31 passed (29 pre-existing + 2 new).
  • 6. Re-ran the reference-corpus comparison; metrics below.

Out of scope this round (documented for follow-up)

  • Perfect passive participles of -έω verbs are systematically missing (εὐλογημένος, τιμημένος, δοξασμένος all miss). Generation-mechanism gap; no precedent for explicit :vb: participle lines in the vbs sources.
  • Other missing verb inflections: future middle of λαμβάνω (λήμψεται…), aorist 3pl -οσαν/-σαν (ἤλθοσαν, εἴδοσαν…), optative δῴη, perfect active ἐκέκραξα.
  • -πλ lemma artifact: addconstraints.pl auto-generates plural entities for pers_names, but singular forms get attributed to the plural entity (βορρᾶ → "Βορέασ-πλ"). Needs build-system investigation.
  • Compound-decomposition bug: ἀπηγγέλη → "ἀπό-ἐγγελάω" (should be ἀπο-ἀγγέλλω). C-level scanner issue.

Review

Before → after (LXX reference corpus, 623,685 tokens / 48,995 distinct forms):

metric before after
analyzed (distinct forms) 35,979 40,827 (+4,848)
misses 5,574 forms / 32,102 tokens (5.15%) 864 forms / 1,966 tokens (0.32%)
lemma mismatches 7,442 / 28,336 (4.54%) 7,304 / 27,554 (4.42%)

Missed-token rate down 94%; the remaining misses are exactly the documented follow-up items (verb-inflection gaps, interjections). Mismatches barely moved — they are dominated by the corpus's base-verb lemma convention for compounds, which is not a Morpheus defect.

Verification performed:

  • pytest python/tests → 31 passed (includes new uppercase-beta tests).
  • Target forms verified through the public API after rebuild (names in both default and ignore_accents=True modes; common words; εὐλογέω/ἐγγίζω forms; corpus-style uppercase input via the runner).
  • Generator idempotency proven: consecutive runs emit identical counts (4,521 names / 5,672 forms); --include-recognized recovers full coverage.

Notes for downstream users: unaccented biblical text (verse-initial capitals, Rahlfs-style all-caps names) is best analyzed with Morpheus(ignore_accents=True); the new name entries match in that mode regardless of accenting.


Round 2: fix the remaining verb-inflection misses (explicit :vb: lines + paradigm declarations)

Findings (verified against sources, built index, and live analyzer)

  • :vb: mechanism confirmed end-to-end. gensynform.c parses :vb:<form> <keys> via ScanAsciiKeys (key vocabulary in src/morphlib/morphkeys.h + stemlib/Greek/rule_files/stemtypes.table) and GenIrregForm (genwd.c:217): the first field is stored verbatim as the surface form; keys must include a stemtype plus tense/mood/voice/person/number. A leading - disables a line (falls into the no-op proc_beta branch). Verb pipeline: vbs.simp.ml → conjfile → do_conj (GenConjForms, passes :vb: through) → conjfile.short → indexvbs → vbind, and the analyzer resolves explicit forms via vbind (proven: existing :vb:labe/ analyzes, absent from binary endtables). So new :vb: lines in vbs.simp.ml need no makefile change.
  • Encoding: one simplified beta code everywhere (u=υ, y=ψ, f=φ, x=χ, c=ξ, q=θ); the reference corpus differs only by case. Use the toolkit's to_beta() to write surface forms — do not hand-transliterate.
  • λαμβάνω future middle: source declares ;fu mid, but generation emits a broken stem (lhy = ληψ-, μ dropped) and an n_infix/ionic one (lamy) that produces no endtable forms. Generative path is dead for this shape → explicit lines.
  • Compounds: no separate compound lemmas exist; the analyzer splits preverbs itself (ἐξῆλθεν → lemma e)ce/rxomai). Fixing the simplex form should cover compounds — to be verified empirically after rebuild.
  • -έω perfect passive participles: τιμάω declares ;pp and τιμημένος now hits (Round-1 rebuild fixed it); εὐλογέω declares no ;pp → εὐλογημένος misses. Total perfp_p miss is only 66 tokens, dominated by εὐλογέω (55) → generative fix.
  • δίδωμι has no optative declaration and o_stem has no optative ending table (endtables/basics: only a1/ihn/imen/imhn/pr have opt tables) → explicit lines.
  • κράζω: the corpus codes ἐκέκραξα etc. as reduplicated aorists (AAI), not the declared perfect stem κεκράγ- → explicit lines with aorist keys.
  • ὁράω declares no active aorist at all (εἶδον resolves via the separate οἶδα lemma — a mismatch, not a miss) → explicit lines for the -οσαν/-αν variants.

Tasks

  • R2.1 εὐλογέω — done via explicit lines, not declarations. The planned ;pp/;pf/@ mid declarations are inert (they never reach vbind — see Review findings); 8 explicit :vb: lines with the corpus's exact surface spellings instead. εὐλογέω now 60/60.
  • R2.2 λαμβάνω — 19 explicit :vb: lines (future middle incl. λημψόμεθα, aorist passive incl. both spellings of λημφθήτω, aorist active -οσαν/-αν variants, imperfect middle) + participle neuter acc sg. Coverage 324→440/454; residual = compound-passive forms under compound lemmas (out of scope).
  • R2.3 ἔρχομαι — :vb: for ἤλθοσαν + ἐλθέτωσαν (aor2). Compound -θοσαν residuals remain (covered only via separate compound lemmas → mismatches, not misses; out of scope).
  • R2.4 δίδωμι — optatives δῴη/δῴης. Plan's o_stem key failed at runtime ("no stemtype seen": o_stem is derivtypes-only); fixed by adding real stemtype omi_pr before it, mirroring the existing :vb:di/dw precedent. DI/DWMI now 34/34 covered.
  • R2.5 κράζω — 9 reduplicated-aorist :vb: lines + augmented variant ἐκεκράξατε (the corpus attests both spellings). KRA/ZW now 60/60.
  • R2.6 ὁράω — 4 aorist-variant :vb: lines. These fixed lemma attribution (forms previously resolved via οἶδα as mismatches; now also match gold ὁράω): lem-ok 5→46 of the covered forms.
  • R2.7 nom.lxx nouns — αλληλουία/ἀντιλήμπτωρ/νοσσία/παστοφόριον added, and integrated into scripts/build_lxx_stemlib.py (regeneration had wiped the hand-added section once; all curated entries now live in the script). νοσσία stored forms switched grave→acute per standword() semantics; generator gained a standardize() so no future run re-emits word-final graves (38 name entries fixed, 22 duplicate spellings deduped).
  • R2.8 Rebuild + verify: C harnesses (test_lookup, test_indecl) then toolkit (verify_r2.py, dual accent modes); pytest python/tests → 31 passed; full compare.py re-run — metrics in Review below.
  • R2.9 lessons.md updated (pipeline truth, standword/CheckGenWords accent semantics, inert declarations, deriv redupl gate, managed-section trap).

Out of scope (documented)

  • Interjections (οἴμοι ×18 …): Morpheus has no interjection POS; would require a category decision. ~44 tokens.
  • C-level compound-decomposition bug (ἀπηγγέλη → ἀπό-ἐγγελάω) and the -πλ pers_name artifact from Round 1 — build-system issues, separate round.
  • The broken reg_conj future-middle generation for -μπ stems (produces ληψ- instead of λημψ-) — a C-code fix candidate for a later round; explicit lines cover the LXX need meanwhile.
  • Round-3 candidates surfaced by this run's residual misses: compound passive/fut-passive forms under compound lemmas (συλλημφθήσῃ ×3 …, ~15 tok), ὁράω aorist/future passive (ὁραθῇ/ὁραθῆναι/ὁραθήσεται) + compounds (ἑώρακαν, ἀφιδὼν), and the new top misses: ἥκασιν ×20 (ἥκω), ἐνεπύρισαν/ἐνεπύρισεν ×31 (πυρίζω -οσαν aorist), γνώτωσαν ×10, εὕροσαν/ εὕροσάν ×15 (εὑρίσκω -οσαν), ἡμάρτοσαν/ἠγάγοσαν/ἐφάγοσαν/ἐνεβάλοσαν (-οσαν aorist pattern across several verbs).
  • Reference-corpus lemma quirks, not Morpheus defects (now count as mismatches): ἀντιλήμπτωρ lemmatized without the μ (A)NTILH/PTWR ×19 tok); νοσσία accented on the final alpha (νοσσιά) instead of the standard penult (×17 tok); θιμωνιά same quirk (×4). Morpheus keeps the linguistically correct spellings.

Review (Round 2)

Before → after (LXX reference corpus, 623,685 tokens / 48,995 distinct forms):

metric Round-1 build Round-2 build
analyzed (distinct forms) 40,827 40,939 (+112)
misses 864 forms / 1,966 tokens (0.32%) 729 forms / 1,155 tokens (0.19%)
lemma mismatches 7,304 / 27,554 (4.42%) 7,327 / 27,643 (4.43%)

Missed-token rate down a further 41% (cumulative 96% vs Round-0's 5.15%). The small mismatch increase is expected: forms newly covered by this round include the corpus's quirky lemmas (above), which now count as mismatches instead of misses; ὁράω -οσαν forms conversely moved from mismatch (via οἶδα) to match.

Per-lemma results (tmp/lxx/verify_r2.py, dual accent modes):

lemma covered / total notes
εὐλογέω 60/60 ✓ was 0/60; explicit :vb: lines (declarations are inert)
δίδωμι 34/34 ✓ was 0/34; omi_pr stemtype fix; 1 tok mismatch = παραδῴη compound convention
κράζω 60/60 ✓ was 0/60; reduplicated-aorist lines + augmented variant
νοσσία 17/17 covered grave→acute stored forms; lemma quirk (accent position) → mismatch, not miss
ἀντιλήμπτωρ 19/19 covered worked since Round-2 data landed; harness false-negative fixed; μ-dropped gold lemma → mismatch
αλληλουία / παστοφόριον 24/24, 14/14 ✓ unchanged from earlier verification
λαμβάνω 324→440/454 +116 tok; residual = compound-passive forms under compound lemmas
ἔρχομαι 44/97 (unchanged) simplex fixed in Round 1; compound -θοσαν covered only via compound lemmas → mismatches
ὁράω 46/51 (coverage unchanged) the 4 new lines fixed lemma attribution for 41 tok (οἶδα→ὁράω); residual = passive + compounds

Root causes found and fixed this round:

  1. ;xx/@ xx paradigm declarations in vbs sources never reach the built index: makefile target is cat … > conjfile; do_conj; mv conjfile.short /tmp/vbmorph; indexvbs, and indexvbs only processes :vs/:aj/:no/:vb/:wd/:de: lines of the do_conj output. Only explicit lines (or :de: deriv stems) affect vbind.
  2. The generative deriv path rejects non-reduplicated perfect stems (derivio.c checkcomderiv2: if (!had_redupl && Is_perfect(stemtype)) continue;), so -έω pp participles (εὐλογημένος) never generate.
  3. GenIrregForm requires a real stemtype in keys — o_stem alone is derivtypes-only and fails with "no stemtype seen" (δῴη); omi_pr fixes it.
  4. standword() rewrites GRAVE→ACUTE on every input before lookup, and CheckGenWords compares accents exactly in normal mode — stored word-final graves can never match. Fixed at the root: generator standardize() + νοσσία block; 38 name entries corrected, 22 duplicate spellings deduped (5672→5650 forms).
  5. Managed-section trap: re-running build_lxx_stemlib.py rewrote nom.lxx from the script's lists and wiped the hand-added Round-2 section. All curated entries now live in the script; regeneration proven byte-identical for the generated section.

Verification performed:

  • C harnesses against the rebuilt stemlib: test_lookup (δῴη round-trip), test_indecl (all 5 νοσσία forms hit CheckGenWords in normal mode).
  • Toolkit dual-mode verification of every previously-missed target form (verify_r2.py, now reporting coverage vs lemma-agreement separately).
  • pytest python/tests → 31 passed.
  • Full corpus re-run: tmp/lxx/compare.py (89 s) → report.txt metrics above; generator idempotency re-proven (two consecutive runs, generated section byte-identical).

Round 3: use Morpheus to produce a full morphological analysis of the LXX ("morph lxx")

Goal

Produce a complete, contextually disambiguated annotation of the whole Septuagint with Morpheus — a drop-in alternative to the reference corpus's own *.mlxx annotations — and measure how it compares. Rationale: after Rounds 1–2 Morpheus covers 99.81% of LXX tokens (0.19% miss), so full-corpus annotation is now feasible; the toolkit already has the disambiguation machinery (Viterbi POS-bigram tagger + frequency ranking, tagger.py / ranking.py) and MorphGNT-style code mapping (morphgnt.py), built for the GNT — Round 3 calibrates it to the LXX.

Design decisions (engineering calls; veto-able)

  • Lemma convention: compound lemmas (Morpheus's natural output; more informative than base-verb lemmatization). A --base-lemmas mode is a possible follow-up but needs proper preverb handling — the toolkit does not expose the C-level preverb split — deferred.
  • Output format: same column layout as the reference corpus (form type parse lemma [prefix]). The type column carries coarse POS (N, V, A, P, D, C, X, I, RA…); the parse column carries Morpheus's full 8-char parsing code (person/tense/voice/mood/case/number/ gender/degree) — a superset of the corpus's 5-char codes. Lemma in canonical beta, uppercased to match the corpus style. Prefix column empty in v1.
  • Disambiguation: Viterbi POS-bigram tagger over Morpheus candidates, with both models (POS bigrams + lemma frequencies) trained on the reference corpus itself (~624k tokens). Note: this calibrates to the domain but makes agreement-with-corpus evaluation partly circular; reported as such.
  • Misses: fall back to the corpus's own annotation for the residual ~0.19% (interjections, rare forms) so the output is complete; fallbacks are counted and reported per book.

Tasks

  • R3.1 Train LXX models: both training scripts gained a --format mlxx adapter (shared coarse_pos() mapper in morphgnt.py: first letter, two-letter R-codes kept); emitted data/pos_bigram_lxx.json (16 tags) + data/lemma_frequencies_lxx.json (12,943 lemmas / 623,685 tokens). Bundled GNT tables untouched.
  • R3.2 Driver scripts/build_morph_lxx.py: reads all corpus *.mlxx; analyzes the 48,995 distinct forms via Morpheus(ignore_accents=True) with an LXX ranker (joint lemma+POS frequencies); Viterbi-tags each contiguous token run; emits re-annotated files to --out-dir in the same column layout (headers preserved, lemmas uppercased to corpus style); fallback = corpus's own annotation for tokens with no candidate, recorded in <book>.fb sidecars + summary report.
  • R3.3 Evaluation tmp/lxx/eval_morph.py: line-aligned comparison — lemma agreement (normalized), POS agreement, parse-field agreement (class-aware: verb codes are 5-char tense/voice/mood/person/number, nominal codes 3-char case/number/gender); Morpheus-only vs full-file rates; top disagreements ranked by corpus token frequency.
  • R3.4 Ran end-to-end on all 64 books (driver 66 s, eval 19 s) → tmp/lxx/morph-lxx/; metrics in Review below; pytest python/tests → 31 passed; lessons.md updated.

Out of scope / follow-ups (documented)

  • Base-lemma output mode (needs preverb exposure/heuristics — follow-up).
  • The known C-level compound-decomposition bug (ἀπηγγέλη → ἀπό-ἐγγελάω, ~26 tokens): it will show up in evaluation; fixing the scanner is a separate round.
  • Interjection POS: Morpheus has no such category; those tokens use corpus fallback.
  • Surfaced by this round's evaluation: article/pronoun dictionary-form conventions (the corpus groups all article forms under ὁ and lemmatizes pronouns to the nominative; Morpheus uses the stem ο( / distinct forms — ~40k tokens of the lemma gap); residual disambiguation errors (ὅτι→ὅστις ×4,044; θεοῦ→θεάομαι ×1,114; λέγων→λέγος ×814); accent/orthography differences on names and adverbs.

Review (Round 3)

Deliverable: a complete Morpheus annotation of the whole Septuagint in the reference corpus's own format — tmp/lxx/morph-lxx/ (64 re-annotated .mlxx files + summary.txt

  • per-book fallback sidecars), produced by scripts/build_morph_lxx.py in 66 s.

Coverage: 623,685 tokens → 99.81% analyzed by Morpheus (622,530); the residual 1,155 tokens (0.19%) fall back to the corpus's own annotation, so the output is complete. Fallbacks are dominated by verb forms (VAI/VA/VBI/VC/V1) plus rare nouns — exactly the Round-2 residual list; interjections use fallback too (no POS in Morpheus).

Agreement with the reference corpus (Morpheus-only tokens, i.e. excluding fallbacks):

dimension agreement
lemma (normalized) 90.01% (560,327/622,530)
POS (coarse) 85.11% (529,808/622,530)
parse fields (class-aware; 421,191 comparable tokens) 76.24% (321,123)

Including fallback tokens: lemma 89.84%, POS 84.95%.

What the disagreement is made of (top items by token frequency):

  • Dictionary-form conventions for articles/pronouns (~40k tokens): corpus groups article forms under ὁ and lemmatizes pronouns to the nominative; Morpheus returns the stem ο( or distinct forms. A convention difference, not an error — flip-able in a follow-up.
  • Compound-verb lemma convention (base vs compound) — documented since Round 1.
  • Accent/orthography: Δαυιδ→Δαυίδ ×1,091, πάντα→πᾶς ×1,378, ὅμοιος→ὁμοῖος…
  • Genuine disambiguation errors the tagger still makes: ὅτι→ὅστις ×4,044, θεοῦ→θεάομαι ×1,114, λέγων→λέγος ×814, ἰδοὺ→ἰδού ×1,152.
  • Cases where Morpheus is arguably more correct than the corpus: μοι → ἐγώ (corpus lemma "δς") ×964; standard accenting throughout.

Caveat: both models were trained on this same reference corpus, so agreement numbers measure domain consistency as much as absolute accuracy; they are a floor for real-world use on untagged LXX text, not a ceiling claim.

Verification performed:

  • pytest python/tests → 31 passed (toolkit changes: coarse_pos() in morphgnt.py, format adapters in the two training scripts).
  • Output spot-checked by eye across registers (Gen 1:1, Psalms, Daniel OG) — annotations are linguistically sound; line counts match the source files exactly.
  • Fallback sidecars sum to exactly the summary's fallback count (1,155).

Round 4: repo-wide scrub of the reference corpus's name (user re-grep found stragglers)

Context

The user grepped the working tree again and still saw the reference corpus's name in runner.py "and other files". Rounds 1–3 had only scrubbed the docs; this round extends it to every morpheus-owned file (committed + tmp). A word-boundary grep across all file types found stragglers the earlier .py/.md/.sh-only verification missed: a stale generated conjfile, old report/snapshot scratch files, and a local path embedded in the Round-3 summary.

Tasks

  • R4.1 python/morpheus_toolkit/runner.py — _normalize_beta docstring no longer names the corpus ("some corpora (…)" → "some reference corpora").
  • R4.2 python/tests/test_runner.py — same rewording in the test comment.
  • R4.3 tmp/lxx/compare.py — removed the hardcoded local checkout path; corpus location now comes from --corpus-dir / LXX_CORPUS_DIR (same interface as build_lxx_stemlib.py).
  • R4.4 stemlib/Greek/conjfile — stale generated artifact still carried the pre-scrub Round-2 comment wording; regenerated from the already-clean sources via scripts/build-stemlib.sh. Diff vs a pre-rebuild snapshot: exactly the 3 reworded comment lines, nothing else (stemlib otherwise byte-identical).
  • R4.5 scripts/build_morph_lxx.py — summary no longer embeds the local corpus path (it was leaking into morph-lxx/summary.txt); now a neutral label. Regenerated all 64 outputs: every .mlxx + sidecar md5-identical, only summary.txt changed.
  • R4.6 Deleted stale scratch snapshots retaining pre-scrub wording (report.prev, regen.diff, nom.lxx.{pre-scrub,pre-r2regen,post-r2regen}); re-ran compare.py so report.txt/TSVs come from the current clean format.

Review (Round 4)

  • Word-boundary grep for the corpus name across all repo files (excluding .venv): zero occurrences in any morpheus-owned file. The only remaining case-insensitive substring hits are legitimate Latin dictionary entries (peccator, siccatio, occatio…).
  • pytest python/tests → 31 passed.
  • compare.py smoke run with the new CLI reproduced Round-2 metrics exactly: misses 729 forms / 1,155 tokens (0.19%), mismatches 4.43% — proving the interface change is behavior-neutral.
  • Lessons captured in lessons.md (word-boundary all-file-type greps; never hardcode the corpus location).

Round 5: update the user docs (READMEs) for the new texts and features

Context

The repo has no file named "tutorial" — the user-facing documentation is README.md (overview, build, cruncher usage) and python/README.md (toolkit tutorial). Neither mentions anything from Rounds 1–3: the LXX stemlib entries, uppercase-beta normalization, or the full-LXX annotation pipeline. Update both.

Tasks

  • R5.1 Re-measured with the current build: AF coverage 90.9→91.2%, top-1 84.5→84.8% before / 89.9→90.1% after ranking, POS 74.6→74.3%; MorphGNT-sourced subset 96.4→96.5%, top-1 89.7→89.8% / 95.9%; analyze_corpus.py total 98.2→98.4%. SBLGNT row untouched (no local checkout).
  • R5.2 Root README.md: new "Septuagint (LXX)" section — what the stemlib gained (nom.lxx names + curated common words, new verb entries), coverage result, regeneration commands; pointer to the toolkit docs for the annotation pipeline.
  • R5.3 python/README.md: intro bullet on case handling (Greek input is normalized to lowercase beta before the cruncher; Latin untouched); new "The Septuagint (LXX)" section — bundled LXX models + rebuild commands (--format mlxx), scripts/build_morph_lxx.py full-corpus annotation, agreement numbers with the circularity caveat.
  • R5.4 Refreshed the ranking table's AF rows (re-measured) with a footnote that the SBLGNT row predates the LXX additions; reworded the "Known limitations" coverage bullet (the old "~1.6%" figure was not reproducible from any local corpus).
  • R5.5 Verified: documented commands match actual --help for all four scripts; both READMEs contain zero occurrences of the scrubbed corpus name; pytest → 31 passed.

Review (Round 5)

Docs updated:

  • README.md — new "Septuagint (LXX)" section between the Python-toolkit blurb and Tests: nom.lxx contents (~4,500 names + curated common words), verb-source additions, 99.81% coverage result, regeneration commands; toolkit blurb now also points at LXX annotation support.
  • python/README.md — new "Case handling" intro bullet (uppercase-beta normalization); new "The Septuagint (LXX)" section: bundled *_lxx.json models + rebuild commands, full-corpus annotation via build_morph_lxx.py (format, fallback, lemma convention), agreement numbers with the circularity caveat; ranking table's AF rows re-measured and footnoted; coverage limitation bullet reworded.

Verification performed:

  • Re-measurement ran the exact commands already documented in python/README.md against ../apostolic-fathers (63,222 tokens) with the Round-2-built stemlib — numbers above; movement is small but real (shared biblical names now resolve).
  • Every command shown in the docs was checked against the script's actual --help output (build_lxx_stemlib.py, build_morph_lxx.py, build_pos_model.py, build_lemma_frequencies.py) — flags and choices match.
  • Cross-links use valid anchors (#septuagint-lxx, #the-septuagint-lxx).
  • Word-boundary grep of both READMEs: zero occurrences of the scrubbed corpus name (Round-4 constraint held).
  • pytest python/tests → 31 passed.

Round 6: make "morph lxx" better than the reference — convention layer + base-lemma mode ✅

Rationale

The analyzer is at its stopping point for LXX (99.81% coverage; residual = interjections

  • rare forms). Round 3's evaluation decomposed the ~10% lemma disagreement: dictionary-form conventions (articles/pronouns, ~40k tokens) and compound-vs-base verb lemmas dominate; genuine disambiguation errors are next. The significant gains live in the annotation layer. This round adds two opt-in convention modes to build_morph_lxx.py and regenerates + re-evaluates.

Design decisions (engineering calls; veto-able)

  • Opt-in flags; natural output stays default. --dictionary-forms and --base-lemmas; with both off the Round-3 behavior is byte-identical. The "better morph lxx" run uses both.
  • Dictionary forms (MorphGNT-style). Article lemma ο( → ὁ. Pronoun mapping only where it is lossless and standard — decided per class from the corpus's own coding sheet: if the corpus uses catch-all lemmas (e.g. δς for personal pronouns) we do NOT copy that; those tokens stay as Morpheus's more informative lemmas and are documented as corpus quirks.
  • Base lemmas. Strip preverbs from compound verb lemmas; a prefix is stripped only when the remainder resolves as a verb in Morpheus (guards against over-stripping).

Tasks

  • R6.1 Quantify: run eval_morph.py on the current output; count tokens per disagreement class (articles, pronoun pairs, compound-base pairs); read the corpus's *Morph-Coding sheet for its article/pronoun conventions. → Done in prior session: RA 88,429 tok (table covers 88,424), RP 26,934 (covers 26,628), RD 35,834 (covers 34,704); compounds ~17.5k head-strip + ~11.8k tail-match fixable (compound_counts{,2,3}.txt, tables.txt).
  • R6.2 Implement --dictionary-forms in scripts/build_morph_lxx.py (+ mapping table). → Three closed-set tables (articles→ὁ/RA, personal pronouns→ἐγώ|σύ/RP, demonstratives→ family nominative/RD); unambiguous forms forced before Viterbi; 4 homographic article forms left contextual; type column emits the reference code when the reading matches.
  • R6.3 Implement --base-lemmas (preverb list + verb-resolution guard). → Head preverb-stripping + tail-matching for contracted compounds, guarded on attested preverb remnants (PREVERB_REMNANTS, 40 LXX-attested ∪ full preverbs); memoized.
  • R6.4 Regenerate morph-lxx with both flags; re-run eval; before/after agreement table. → See Review below (all three metrics beat Round 3).
  • R6.5 Update python/README.md LXX section (new flags, new numbers); lessons if any. → README updated (flags + mechanism + numbers table + residual breakdown); lessons.md extended with the beta-dialect / ranking / tagger lessons from this round.

Review (Round 6)

Result: the flagged run beats Round 3 on all three metrics — no regressions.

variant lemma POS parse Δ vs Round-3
Round-3 natural (old ranking) 90.01% 85.11% 76.24% —
natural, new ranking only (morph-lxx-base2) 90.25% 85.09% 76.26% +0.24 / −0.02 / +0.02
--dictionary-forms only (morph-lxx-dict) 90.70% 90.54% 79.75% +0.69 / +5.43 / +3.51
both flags (deliverable, morph-lxx/) 94.23% 90.54% 79.75% +4.22 / +5.43 / +3.51

(623,685 tokens; 622,530 Morpheus-analyzed; 1,155 fallbacks = 0.19%.)

What the attribution runs showed (and fixed)

The first flagged regeneration (prior session) regressed parse by −0.41pp. Line-by-line attribution (diff_variants.py, with a monkey-patched reconstruction of the old ranking in morph-lxx-oldnat/) isolated three distinct causes — all now fixed:

  1. Ranking over-promotion (root cause, ranking.py). The joint-frequency lookup scored 0 when Morpheus's POS isn't a key in the lemma's row (αὐτός tagged A here but RD in the reference). The prior session's fix fell back to full plain lemma frequency, which over-credited unattested pairings: σός(N) 0.432 vs σύ(RP) 0.838 left only a ~1.2-nat emission gap that bigram context flipped (σοῦ/ἐμοῦ → possessive adjectives, τὰ/τοῦ → τίς). Fix: an unattested (lemma, POS) pair now scores a floor of one occurrence — it can never be the gold reading of any token (gold is built from the same reference), so the floor only removes wrong answers. Natural mode recovered fully (76.26% parse ≥ Round 3).
  2. Forcing removed featureless candidates. Morpheus offers a surface-form pseudo-candidate for αὐτοῦ (lemma=αὐτοῦ, POS D, empty parse code) that vacuously passes parse comparison; forcing to hits[:1] deleted it and collapsed the real αὐτός candidates arbitrarily. Fix: forcing keeps all target-reading candidates so Viterbi still uses context.
  3. Same-POS feature ties are arbitrary in the tagger (one candidate per coarse POS, first on score ties). For tied forms the reference's own accent notation correlates strongly with the reading — AU)TOU= is 97% GSM while AU)TOU\S//S are ~100% APM; AU)TW=N came out GPN though 92% of gold is GPM. Fix: a derived TIE_BREAK table (≥10 tokens, ≥90% majority per spelling; weaker cases like τὰ stay contextual) stable-sorts hits so the tie resolves toward the reference majority. This single change moved parse 75.79% → 79.75%.

Also verified: base-lemma mode is a pure lemma-column operation (+3.53pp lemma, zero POS/parse effect); spot checks pass (ἀπέστειλεν→στέλλω+ΑΠΟ; contracted ἐκέρχομαι→ἔρχομαι at Gen 1:2368 matching gold exactly); summary.txt reports the active conventions; pytest 33 passed.

Honest caveats and residuals

  • Models are corpus-trained. Both disambiguation models come from this same reference, so agreement measures domain consistency as much as absolute accuracy — a floor for untagged LXX text, not a ceiling claim.
  • The eval's "top lemma disagreements" list is misleading: it sorts by total form frequency and prints only the first disagreeing line. True per-form mismatch counts are far smaller (τοῦ: 1/10568; αὐτοῦ: 4/9181; οἱ: 1/4762; ὅτι: 8/4044 — the last was already ~right in Round 3, an earlier misread).
  • Residual lemma disagreement (~5.8%) is dominated by documented reference quirks where Morpheus is arguably more correct (surface-form lemmas τοῦ→τοῦ; ἰδοὺ lemmatized as ὁράω ×28), the genuine ἕως/ἠώς homograph (277 tokens — largest true error), κυριόω intrusions (32), and ~286 compound verbs the reference keeps unstripped while the heuristic strips (ἐγείρω, ἐπίσταμαι, …; etymologically they are compounds). Not patched: copying per-lemma corpus choices would be overfitting to annotation inconsistency.
  • Structural limit: the tagger keys states by coarse POS, so feature-level ambiguity within a POS (masc/neut homographs of article forms) is resolved only where the reference's accent notation carries signal; genuinely ambiguous cases stay at chance.

Round 7: Morpheus × Apostolic Fathers — AF pipeline, convention layer, elided forms ✅

User direction: do NOT change anything NT-related; improve Morpheus against the Apostolic Fathers corpus instead. All five approved items implemented, verified and committed (d760a3e, 259df21, 23b33d3, b3707de). Working files in tmp/af/ (git-ignored): evaluators (eval_af.py, eval_out.py), snapshots (ev2_*.txt, ev3_*.txt), driver outputs, AF table sources.

Evidence gathered (63,222 Greek tokens: 52,094 MorphGNT-derived gold + 11,128 machine-model gold)

  • Verb mismatches on trustworthy gold are ~97% convention differences, not errors (reduplication γίγνομαι→γίνομαι ×249; augment ἐθέλω→θέλω ×95; aorist-vs-base εἶδον/εἶπον→ὁράω/λέγω ~160; accents ~116; act/mid ~90) — the Round-6 pattern.
  • The "interrogative gap" was an accent artifact (MorphGNT writes τίς, HF lines τις): 42.6% → 0.2% under accent-insensitive comparison. No analyzer change needed.
  • Elided forms (544 tokens, all HF-sourced) were a real analyzer gap: Morpheus returned no candidates for any of them.
  • Joint ranking hurts lemma on AF (unlike LXX): the reference tags adjectives N where Morpheus says A; plain frequency is best for lemma, joint only for POS.

Tasks

  • 1. Commit the two toolkit fixes (frequency-builder guard for MorphGNT-format corpora — previously all token lines skipped, 106 vs 5,874 lemmas; and Analysis.matches_lemma() normalizing both sides). Verified: pytest 33/33, LXX rebuild byte-identical. (d760a3e)
  • 2. AF convention table + annotation pipeline — scripts/build_morph_af.py mirroring the LXX driver (accent-insensitive mode, AF frequency ranking, optional Viterbi/joint). Convention layer derived at run time from MorphGNT-derived gold with three guards: single target per source lemma, ≥10 tokens, and M==0 — no trustworthy token with that choice may agree with the gold. The M==0 guard rejects form-dependent splits (κύριος/κυρία gender 10-vs-613; ἐγώ/κἀγώ enclitic 22-vs-1198; ἐπεί/ἐπειδή 17-vs-33) that a plain single-target guard would have applied globally and broken. Rejected pairs are printed in the summary for auditability. (23b33d3)
  • 3. Stemlib additions for elided forms — stemsrc/nom.elided (13 forms: δι, κατ, παρ, μετ, ἀπ, ὑπ, ἐπ, ἀφ, ὑφ, ἐφ, καθ, μεθ, ἀλλ) mapping to existing full-form lemmas; rebuilt nomind. Entries resolve only in accent-insensitive mode (-n), so default-mode behavior is unchanged and vbind byte-identical. (259df21)
  • 4. Bundle AF tables + repo tooling — lemma_frequencies_af.json / pos_bigram_af.json in python/morpheus_toolkit/data/ (neutral source label, LXX precedent); scripts/agreement_report.py gained --frequencies/--model flags. (23b33d3)
  • 5. Upgrade the AF repo — tools/morpheus/analyze.py rewritten as a self-contained mirror (tables derived at run time from its own gold; conventions on by default with --no-conventions escape); all 15 data/morph-morpheus/*.txt regenerated + new .fb sidecars and summary.txt; README rewritten. Committed locally in the AF repo (2b51d98, branch morpheus-analysis) — not pushed.

Review (Round 7)

Headline numbers (driver = accent-insensitive mode + AF frequency ranking + top-1 candidate + convention layer):

subset tokens lemma acc-insens lemma POS coverage
all Greek 63,222 91.1% 92.1% 75.3% 99.35% (413 fallbacks)
source=MorphGNT (trustworthy) 52,094 96.6% 97.4% 75.7% 99.95%
source=grc_proiel_lg (machine gold) 11,128 65.4% 67.2% 73.1% 96.5%
  • Baseline before the round: 90.4/74.3 (ALL), 96.3/75.5 (MG subset) with NT tables, no -n, ~1,027 misses. The convention layer lifts the trustworthy subset to 96.6% exact / 97.4% accent-insensitive; -n mode cuts misses from ~1,027 to 413 (elided forms + unaccented spellings).
  • Config trade measured: Viterbi (--tagger) costs ~1.4pp MG-subset lemma for +1.0pp POS — the opposite of LXX, where it was net positive. Default is top-1 (lemma-first); --tagger/--joint available for POS-optimized output.
  • Verification: saved ev2 evaluator snapshots re-run byte-identical after all toolkit changes; pytest 33/33; AF repo's analyze.py output verified byte-identical to morpheus's build_morph_af.py across all 15 files; LXX driver spot-check clean. NT/LXX untouched (no shared stemlib lemma changes).

Open questions for the user: push morpheus origin/master (4 commits ahead of 75fface; the AF README pins 23b33d3, so pushing makes that pin real)? Push the AF repo's morpheus-analysis branch?

Round 8: new corpora — PapyGreek, Latin benchmark, PROIEL Greek NT cross-validation ✅

User approved items 1–3 ("Do round 8"); item 4 (classical AGLDT) was explicitly optional and is not done. All three complete and committed (a2b7a77 for item 1; 1935b8f for items 2–3). Working files in tmp/<corpus>/ (git-ignored): PapyGreek treebanks + evaluators, PROIEL clone + evaluator, AGLDT clone + Latin evaluator.

Context: where Morpheus stands, and what "improve" can still mean

The Greek biblical side is near saturation: LXX 99.81% coverage (Round 2), NT ~95% top-1 after the name additions, AF 99.35% / 96.6–97.4% lemma (Round 7). Remaining levers found in corpus research (2026-10-11):

Key caveat — most Perseus "morph texts" are circular. Celano's OGA/OLA treebanks (45M Greek / 17M Latin words; the data behind morph.perseus.org) were auto-annotated with Morpheus (LemmatizedAncientGreekXML). They cannot serve as independent gold for improving Morpheus — at best a scale benchmark or POS-bigram source. The valuable resources are the manually curated treebanks and a few independent lemmatized corpora:

corpus size annotation license / access improves
PapyGreek Treebanks (Helsinki; Zenodo 10.5281/zenodo.5074307) 44K tokens, documentary papyri ca. 300 BCE–700 CE (Zenon archive letters, petitions) semi-manual + reviewed, dependency-morphosyntax; TEI XML CC BY-SA 4.0, free new Koine domain — first test of Morpheus on papyrological Greek; safe stemlib additions from human-reviewed gold
PROIEL greek-nt (Tischendorf 1869) 140,763 tokens fully manual morphosyntax + info structure; PROIEL XML and CoNLL-X CC-BY-NC-SA 3.0, GitHub (syntacticus/syntacticus-treebank-data) independent second NT gold — validates how much of our SBLGNT-based NT numbers are robust vs convention-fitted (some sections unannotated: HEB 13, catholics)
PROIEL latin-nt (Jerome's Vulgate) + caes-gal/cic-att/per-aeth/pal-agr ~113K tokens total manual; CoNLL-X available CC-BY-NC-SA 3.0, GitHub first benchmark of the Latin analyzer (this checkout ships a full Latin stemlib — ~286K source lines — that has never been measured)
AGLDT / Perseids v2.1 (PerseusDL/treebank_data) Greek ~190K words (Homer, Hesiod, Aeschylus, Sophocles, Thucydides 1, Aesop); Latin: Caesar, Cicero, Ovid, Petronius, Vergil… manual by 200+ annotators; free on GitHub open classical register — expands/validates both languages beyond the biblical domain
Index Thomisticus (Busa) 11M lemmatized words, medieval Latin classic lemma+POS per word; plain text at corpus.thomisticum.org (+ treebank subset) free large-scale Latin frequency table source
OGA/OLA/GLAUx (Celano 2024) 45M / 17M words auto, Morpheus-derived → circular open not gold; scale benchmark only
TLG huge subscription; abridged untagged paid skip unless user has access
Hebrew (OSHB/SBLHBT) — — free N/A — this checkout has no Hebrew component

No large freely available patristic tagged corpus exists (the AF repo remains the main one); PapyGreek is the closest new-domain gold in the user's register. (Note: AGLDT v2.1 Greek turned out much larger than this table — 557,918 words; see Round 9.)

Review (Round 8, item 1: PapyGreek) ✅

Dataset: Zenodo 10.5281/zenodo.5074307 → tmp/papygreek/ezhenrik-papygreek-treebanks-b87127c/ (394 documentary + 1 literary ALDT v1.5 XML; 42,313 Greek tokens; CC BY-SA 4.0). ALDT dual layers: input = form_orig (editorial markers stripped), gold = lemma_reg/postag_reg. POSTAG is 9 chars — pos person number tense mood voice GENDER CASE degree (gender before case, opposite of MorphGNT; lowercase).

Evaluator: tmp/papygreek/eval_pg.py (+ missdump.py) — coverage, top-1 before/after ranking (raw order via runner.analyze_beta(ignore_accents=True), not the pre-ranked API), coarse POS, miss profile. The marker inventory was expanded after a full corpus audit: beyond ∼∽∤⋰⋱⸢⸣❨❩<>[]{}() + U+0324 and |num:N|/|g:…| spans, the corpus also uses 〚〛 (×310), ? (×81), ⊢⊣ (×50), ⸌⸍ (×37), ⎴⎵/⦑⦒ (×22), …, -, ⸜⸝, ⧼. The apostrophes ʼ ’ are real elision marks (kept); combining diacritics ͂ ͅ are Greek letters (kept).

Baseline → after stemlib additions (36,455 word tokens; numerals tracked separately — Morpheus has no numeral support and alphabetic numerals collide with real words; documented limitation, not added to the stemlib):

metric baseline after round-8 additions
coverage 89.4% (old marker set) / 89.9% (fixed) 92.9%
misses 3,871 (marked 1,878; numerals 863/1,454) 2,600 (marked 1,417; numerals 853/1,454)
top-1 lemma, NT tables 78.3 / 80.0 exact/acc-insens 81.2 / 83.3
top-1 lemma, PG tables 80.5 / 81.8 83.7 / 85.1
POS coarse ~80–82% 81.1 (NT) / 81.7 (PG)

Remaining misses are dominated by irreducible damage: 1,417 marked forms whose editorial reconstruction is partial, plus the 853 numeral tokens.

Stemlib additions (all additive; verified against the rebuilt indexes):

  • stemsrc/nom.papyri — generated by new scripts/build_papyri_stemlib.py (mirrors Round 2's builder: missed nominal tokens with capitalized gold lemma, count ≥ 2 threshold since papyri carry more scribal noise; dictionary-form recognition check; idempotent managed-section merge). 179 names / 407 forms — Egyptian deity/month/personal names (Θωθ, Ἰσις, Ἀμμώνους, Παῦνι, Ἐπείφ, Θερμουθᾶς, Τῦβι…), plus a curated section: papyrus dative-plural pronoun spellings ὑμεῖν→σύ / ἡμεῖν→ἐγώ and the Egyptian loanword κίκιος→κίκι. EXCLUDE: Ἀχιλλᾶς, Ἰωνᾶ (declined forms of known names used as lemmas — Morpheus already parses both correctly).
  • stemsrc/vbs.papyri — hand-curated explicit :vb: entries for dialectal verb forms the generative paradigms don't produce: reduplicated aorists of λαμβάνω (ἰλήφασι ×15, ἔλαβα/ἐλαβα, εἴληφαν — standard Attic!, αἴλαβα), -εισκ-/-οσκ- presents and -ιν infinitives of γιγνώσκω, -ιν/-εριν infinitive variants of χαίρω. 15 forms / ~75 tokens. Makefile's vbind rule now lists the file (and gained explicit prerequisites — fixes the stale-target trap).
  • stemsrc/nom.elided — added ἐγ→ἐκ (×16 in papyri) alongside Round 7's 13.

Bundled table: python/morpheus_toolkit/data/lemma_frequencies_pg.json (3,913 lemmas; punctuation/junk filtered; source label "PapyGreek Treebanks v1.01 (CC-BY-SA 4.0)" — bundling is license-fine, SBLGNT precedent). The PG table beats the NT tables by +2.5pp exact top-1 on papyri (83.7 vs 81.2) — domain tables matter for disambiguation, not just coverage.

Verification:

  • Every new form resolves with its gold lemma (analyze_beta, both modes); accented forms also hit in default mode; unaccented variants only under -n (CheckGenWords accent-exact gate — expected).
  • Index key diff vs Round 7: nomind +362 keys / vbind +13, zero collisions with pre-existing keys → purely additive.
  • AF guard: main-config snapshot (ev2_af.txt) re-runs byte-identical; the --joint config shows a ±0.5pp POS shift that reproduces on a pristine Round-7-source rebuild too — endtable regeneration noise (see lessons), not caused by these additions. ev2_af_full.txt snapshot refreshed to match.
  • LXX guard: full driver re-run over all 64 books / 623,685 tokens with the Round-7 convention flags → one token differs (elided μετ' now correctly P/META/ instead of a junky N reading — an improvement from endtable drift).
  • pytest 33/33.

Review (Round 8, item 2: Latin benchmark + AGLDT frequency table) ✅

First-ever measurement of the Latin analyzer. Evaluator: tmp/latin/eval_latin.py (MorpheusRunner(language="latin"); PROIEL UPOS mapped to lenient Morpheus classes; Morpheus's #N homonym markers stripped on both sides). Gold: PROIEL CoNLL-X (CC BY-NC-SA 4.0 → measurement/docs only, nothing committed).

corpus tokens coverage top-1 raw top-1 ranked (AGLDT) POS raw→ranked
latin-nt (Vulgate) 110,774 98.4% 81.1% 90.5% (+9.4pp) 78.3→81.9
caes-gal 27,397 96.9% 74.3% 85.1% (+10.8pp) 74.4→78.0
cic-att 45,580 97.1% 78.0% 86.5% (+8.5pp) 72.9→75.1
per-aeth 17,554 94.7% 74.8% 83.8% (+9.0pp) 73.8→78.2
pal-agr 11,979 97.5% 75.5% 85.2% (+9.7pp) 71.7→74.3

Table source — license resolution. Index Thomisticus Treebank is CC BY-NC-SA 3.0 (the Digital Classicist wiki footer's "CC BY" is wrong; the project site states NC-SA), and corpusthomisticum.org offers no clear bulk-download license → rejected. AGLDT v2.1 (PerseusDL/treebank_data) is CC BY-SA 3.0 US — commit-safe, and its Latin files carry manual lemma annotations (<word form=… lemma=… postag=…/>, Perseus-style trailing homonym digits). Built from all 11 Latin treebanks (Caesar, Cicero, Propertius, Sallust, Vergil Aen. 6, Ovid, Phaedrus, Augustus Res Gestae, Suetonius, Tacitus, Jerome's Apocalypse): 6,052 lemmas / 55,409 tokens → bundled as python/morpheus_toolkit/data/lemma_frequencies_latin.json (attribution in the file; editorial artifacts like -que/heu! filtered).

What holds up. The four classical corpora share no text with the table, yet top-1 rises +8.5 to +10.8pp — ranking generalizes across Latin registers exactly as the Greek tables did (item 3's +12pp transfer). The Vulgate row is only partially circular: the table contains Jerome's complete Apocalypse and PROIEL latin-nt shares 2.85% of its tokens with it in 10-word sequences — negligible, and even so the non-circular AGLDT table (90.5) beats a fully circular PROIEL-derived one measured earlier this round (88.3).

The # homonym-marker fix. Morpheus's Latin stemlib disambiguates irregulars with #N (sum#1, edo#1, quis#2) and the parser kept them in lemma variants, so frequency lookups for the most common words in Latin (est→sum) silently missed. normalize_lemma() now strips trailing #/digit runs (Greek lemmas contain neither — zero occurrences in all bundled tables; A/B re-run of the AF evaluation with and without the change is byte-identical). Re-running the circular PROIEL table under the new normalization: 88.3 → 91.5 — the fix alone is worth ~+3pp on the Vulgate.

Decisions.

  • Table carries plain frequencies only, no joint lemma×POS counts: Morpheus's Latin analyzer cannot separate adjectives from nouns (both surface as N unless a special stem type fires), while AGLDT tags them separately — joint counts would systematically demote correct readings. (A future joint table should be derived from Morpheus's own output on a large corpus, self-consistent by construction.)
  • Not auto-loaded: Morpheus(language="latin") still ranks with the Koine default unless frequencies are passed — same pattern as the LXX/AF tables.
  • No stemlib changes from this item; coverage gaps (94.7–98.4%) are a separate, larger effort (the Latin stemlib predates these corpora's rare forms).

Review (Round 8, item 3: PROIEL Greek NT cross-validation) ✅

Dataset: syntacticus/syntacticus-treebank-data (cloned to tmp/proiel/, git-ignored) → proiel/greek-nt.conll: Tischendorf 1869, fully manual annotation, 132,366 word tokens (the CoNLL-X export omits the unannotated HEB 13 + Catholic epistles; XML has 140,773). License CC BY-NC-SA 4.0 → measurement/docs only; nothing PROIEL-derived is committed.

Evaluator: tmp/proiel/eval_proiel_nt.py (CoNLL-X reader; gold = col 3 lemma, col 4 UPOS with a lenient class map; SBLGNT-style #1 homonym suffixes and οὕτω(ς) optional-parens normalized before comparison).

metric value
coverage 99.6% (492 misses)
top-1 lemma, raw Morpheus order 78.0 / 79.3 exact/acc-insens
top-1 lemma, ranked with NT tables 90.7 / 91.7
POS coarse 81.6% (n=130,209)

What holds up: the SBLGNT-trained frequency table transfers +12pp to a text it never saw — ranking is robust, not convention-fitted. The gap vs our SBLGNT numbers (~95%) decomposes as:

  • Lemma-convention differences (~3,100 tokens ≈ 2.4pp): PROIEL heads the pronoun families at the plural (ὑμεῖς/ἡμεῖς) where SBLGNT/Morpheus use the singular (σύ/ἐγώ); PROIEL lemmatizes εἶπεν to λέγω where Morpheus keeps the reduplicated aorist εἶπον as its own lemma. Convention-adjusted top-1 acc-insens ≈ 94%.
  • True residual errors (~7.9pp before adjustment): dominated by known homographs — ὡς→ὅς ×455, εἰ→εἰμί ×329, ἢ→εἰμί ×326, τε→σύ ×213, ὃ/ἃ→ὁ ×~280 — the same Round-6 residual profile (not patched there for overfitting reasons; confirmed here to be stable across independent gold).
  • Misses (492): 148 are names, mostly edition spelling/case conventions — PROIEL lowercases σατανᾶς as a common noun while the stemlib entry is uppercase-only (Σατανᾶς resolves fine); PROIEL spells Elijah Ἡλίας (iota) where SBLGNT has Ἠλίας (eta, which resolves). The rest are loanwords/dialectal one-offs (ῥαββεί ×15, ὡσαννὰ, …).

Decision: no stemlib changes from this item — the findings are convention differences between two human-annotated editions of the same text, and adding PROIEL's spellings would change NT behavior (out of scope per user direction) and commit NC-SA-derived content. Numbers documented here only.

Open questions — resolution: pushed morpheus origin/master and the AF repo's morpheus-analysis branch; user has no interest in further Latin work (the Round-8 benchmark + bundled table remain as committed; nothing more planned); PROIEL-derived content stays out of the repo regardless. Record migrated from todo.md; Round 9 drafted there.