Conformance-driven compiler + executor: 47/47 oracle equivalence with the Java reference engine - #23
Open
dhes wants to merge 11 commits into
Open
Conformance-driven compiler + executor: 47/47 oracle equivalence with the Java reference engine#23dhes wants to merge 11 commits into
dhes wants to merge 11 commits into
Conversation
…stic)
Result on current main: 0/16 WHO smart-immunizations FML maps compile
("StructureMap must have at least one group").
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LGgxE9TK56BNPJJX2PnYhj
…line-numbered errors All 16 WHO smart-immunizations maps now clear the header and reach rule parsing (previously 0 — every map died before its first group). Adds structure/import to the StructureMap type (FHIR R4 shape), '/* */' comment lexing, 'uses ... alias X as mode' and 'imports ...' clauses, optional ':' types on group inputs, and token line/context on all parser errors. Corpus status: 0/16 compile (failures now inside rule bodies — next rung). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LGgxE9TK56BNPJJX2PnYhj
Rewrites rule parsing to the spec shape the WHO corpus exercises:
comma-separated sources/targets; source list modes, 'as' bindings, and
where/check FHIRPath capture (tokens re-joined; strings re-quoted);
target transform invocations (create/append/translate/c/uuid),
'=' copy/evaluate forms, and literal params in FHIR value[x] shape;
nested 'then { }' rule blocks; dependent group calls; trailing rule
names. Adds rule/dependent/listMode/condition/check to the types and
switches transform parameters from List<String> to TransformParameter.
Corpus: 16/16 compile (was 0/16). Execution is NOT there yet: the
executor returns success=true with an empty {} for C4 — silent
no-op output; that wall is next.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LGgxE9TK56BNPJJX2PnYhj
…mpty output) Replaces the no-op executor with FmlEngine: execution over generic JSON trees (mutable target nodes; sources read from input JSON or from target-built logical-model instances), scope chains, import-aware dependent-group resolution, choice-type (value[x]) reads and writes, and the corpus's transform set — copy/create/append/uuid/translate (code|coding|display modes, real ConceptMaps)/c/evaluate. Unknown transforms and unresolved groups/variables are hard errors, never silent no-ops. Type-aware where the reference engine is: registered StructureDefinitions drive choice-element naming (declared LM element types beat lexical guessing) and singular-vs-repeating semantics (declared max 1 -> last write wins; computed re-writes of singular elements replace, creations append). Score vs github.com/dhes/fmlrunner-conformance oracles: 31/33 match, 0 errors. The 2 residual diffs are a reference-engine terminology quirk (drops the display it cannot verify for a LOINC coding on tx.fhir.org; keeps the same-shaped IMMZ coding's display) — our output renders the map faithfully. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LGgxE9TK56BNPJJX2PnYhj
…es match The 2 residual diffs were not an oracle quirk to waive — they were a c() semantics gap: the reference engine resolves Coding.display from terminology, ignoring the transform's third argument, and omits the display when the code system is unavailable (e.g. unlicensed LOINC). Implemented: registerCodeSystem() feeds a display store; c() looks up (system, code) and emits no display on a miss. LOINC's absence now falls out naturally instead of being explained away. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LGgxE9TK56BNPJJX2PnYhj
Grammar gaps found by an org-wide parse sweep (ddcc, smart-ot, smart-example-immz) and closed: - source type cast: entry.resource : Patient - comma-less multiline group signatures - element names that are quoted strings (src."-260") or FML keywords (Name.prefix), and keywords in value/parameter positions - parenthesized FHIRPath as target transform and as transform argument - anonymous expression targets: (expr) as var FmlParseSweep jvmTest runs the sweep over FML_SWEEP_DIR and tallies failures by error signature. Oracle regression unchanged: 33/33. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LGgxE9TK56BNPJJX2PnYhj
Two mechanisms: logical-model roots need resourceType emission; bare target contexts (-> tgt then ...) unsupported by the executor. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LGgxE9TK56BNPJJX2PnYhj
Executor semantics learned from ddcc/smart-ot oracle diffs: - entry roots emit resourceType from the SD's type (resource + logical kinds) - bare target contexts (-> tgt then ...) bind without mutating - source type casts filter by resourceType - declared max governs replace-vs-append even for computed writes - declared child types resolve from their own SD (nested LM URL, named datatype core SD, backbone falls back to parent's leaf map) - primitive writes into complex-declared elements drop (reference behavior) - created nodes remember their type for choice-element naming and core-SD cardinality lookups - FHIRPath conditions: in(), and, exists(), exists().not(), .value on prims - leaf-flattened SD maps: shallower element wins collisions - comparison contract: uuids number by fullUrl (entry identity) first Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LGgxE9TK56BNPJJX2PnYhj
Immz maps now load from the conformance repo's maps/ dir, making github.com/dhes/fmlrunner-conformance the single dependency for the 33-fixture run. The ncsa spike skips without its (unpublished) corpus. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LGgxE9TK56BNPJJX2PnYhj
Pure-common code; bumps kotlinx-serialization/datetime to wasm/native-capable versions. Publishes org.litlfred.fmlrunner:fmlrunner:0.1.0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LGgxE9TK56BNPJJX2PnYhj
Singleton values of repeating elements (name, given, entry, ...) were emitted bare, which normalized-diff scoring tolerated but strict FHIR parsers reject. Arrays now come from declared cardinality when an SD is registered, with the REPEATING_ELEMENTS set as fallback. 47/47 intact. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LGgxE9TK56BNPJJX2PnYhj
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Following up on #22 and the "test cases + real integration" direction: this PR rebuilds the FML compiler and StructureMap executor and holds them to an external standard — full-output equivalence with the reference Java engine (validator_cli
-transform) on WHO's published content.Scores
What changed
FmlCompiler.kt): full rule grammar as exercised by the WHO corpus — uses/imports headers, typed and comma-less group signatures, source type casts (entry.resource : Patient), listMode/where/check, quoted and keyword element names, parenthesized FHIRPath targets and arguments, anonymous expression targets, nestedthen {}blocks, dependent group calls (import-aware), line-numbered errors everywhere.executor/FmlEngine.kt, replacingStructureMapExecutor.kt): tree-building executor with semantics learned from oracle diffs rather than assumed — SD-driven choice-element naming and cardinality (replace-vs-append), type-cast filtering, terminology-lookupc()display, translate() modes, cross-SD type propagation for nested logical models, and hard errors instead of silent no-ops (the old executor returnedsuccess=truewith{}output on unimplemented constructs; nothing fails silently now).jvmTest):ImmzConformanceSpike(33 fixtures; skips with instructions when the corpus isn't present),NcsaConformanceSpike(skips unless its license-separated corpus is present),FmlParseSweep(org-wide parse tally, env-pointed).Test corpus
The 33-fixture suite lives in a standalone CC0 repo — https://github.com/dhes/fmlrunner-conformance — containing the pinned WHO IG package, the 16 .fml sources, reference-engine oracle outputs, and the regeneration harness. It's engine-agnostic: any FML implementation can be scored against it. Clone it (or set
FMLRUNNER_CONFORMANCE_DIR) andgradle jvmTestruns the full 33.Happy to restructure — vendor the fixtures into this repo, split the PR, adjust layout per #22 — whatever fits the project's direction. CANDIDATE-ENGINES.md in the corpus repo documents every discrepancy found on the way to 47/47 and what each taught us about the reference engine's semantics; WALKTHROUGH.md there is a non-technical end-to-end example.
Honest scope
Corpus-complete, not spec-complete:
conceptmapblocks, multi-source rules,extends, and several transforms (cc,qty,reference, …) are unimplemented and now fail loudly rather than silently. The FHIRPath subset covers what the corpus exercises. The natural next step — adopting kotlin-fhir as the model layer and kotlin-fhirpath for expressions — is exactly what this conformance suite makes safe to attempt.🤖 Generated with Claude Code
https://claude.ai/code/session_01LGgxE9TK56BNPJJX2PnYhj