WASM/JS external ingestion API - #179
Draft
abkfenris wants to merge 2 commits into
Draft
Conversation
This was referenced Jun 24, 2026
Merged
Javascript size report
✅ Size decreased -56.6%. |
abkfenris
force-pushed
the
phase-2a-feature-gate-embedded-data
branch
from
June 24, 2026 23:42
9b84003 to
2fb2382
Compare
abkfenris
force-pushed
the
phase-2b-wasm-js-ingestion-api
branch
from
June 24, 2026 23:42
ad21361 to
f4285cd
Compare
This was referenced Jun 24, 2026
Draft
Remove embedded data from the default WASM binary (462 KB, down from 1.0 MB). The JS crate now builds with default-features = false on core, so the compressed 4 MB vocabulary blobs are excluded. New JS methods: loadCfStandardsFromYaml, loadCfStandardsFromJson, loadStandards (JsValue), loadKnowledgeFromYaml, loadKnowledgeFromJson, loadKnowledgeObjects (JsValue). Old loadCfStandards/loadKnowledge are kept but gated behind the embedded-data feature. gen-data.mjs combines the 41 per-standard YAML files into public/knowledge.json and copies _cf_standards.yaml to public/, so Vite can serve them during dev/test without embedding them. Vitest, Playwright e2e, and pack smoke tests all updated to use the fetch-based ingestion path. Full nox suite (rust_test, rust_test_no_embedded, js_test) passes: 19+11 Rust tests, 12 Vitest, 5 Playwright, 1 pack smoke.
Replace load_cf_standards() / load_knowledge() (which are gated behind
the embedded-data feature and absent in the default build) with the new
external-ingestion API:
- load_cf_standards_from_yaml: source YAML embedded via include_str!
so the test binary is self-contained without a dev server
- load_knowledge_from_json: minimal inline JSON with the one entry
exercised by the metadata / filter tests
abkfenris
force-pushed
the
phase-2a-feature-gate-embedded-data
branch
from
June 25, 2026 20:36
2fb2382 to
65273d0
Compare
abkfenris
force-pushed
the
phase-2b-wasm-js-ingestion-api
branch
from
June 25, 2026 20:36
f4285cd to
5caceff
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Remove embedded data from the default WASM binary (462 KB, down from
1.0 MB). The JS crate now builds with default-features = false on core,
so the compressed 4 MB vocabulary blobs are excluded.
New JS methods: loadCfStandardsFromYaml, loadCfStandardsFromJson,
loadStandards (JsValue), loadKnowledgeFromYaml, loadKnowledgeFromJson,
loadKnowledgeObjects (JsValue). Old loadCfStandards/loadKnowledge are
kept but gated behind the embedded-data feature.
gen-data.mjs combines the 41 per-standard YAML files into
public/knowledge.json and copies _cf_standards.yaml to public/, so
Vite can serve them during dev/test without embedding them.
Vitest, Playwright e2e, and pack smoke tests all updated to use the
fetch-based ingestion path. Full nox suite (rust_test, rust_test_no_embedded,
js_test) passes: 19+11 Rust tests, 12 Vitest, 5 Playwright, 1 pack smoke.
This is part 3 of 6 in a stack made with GitButler: