docs: migrate the site to Zola 0.23 (Tera2) and bump the pin - #3840
docs: migrate the site to Zola 0.23 (Tera2) and bump the pin#3840worktrunk-bot wants to merge 5 commits into
Conversation
Zola 0.23 replaced Tera 1 with Tera 2, removed shortcodes in favour of components, and started rendering every `.md` file through the template engine. `docs/` did not build under any of those, so the weekly pin bump was blocked at 0.22.1 (#3827). - `macros.html` and `templates/shortcodes/` become components in `templates/components.html`. `toc_nav` takes the active page's permalink and toc rather than the page, since a component parameter takes its type from its default and there is no default that is both a page and a "no active page" sentinel. - Tera2 syntax fixes: `trim_end_matches` → `trim_end`, the `filter` filter (gone) → `get_page`, keyword-only test arguments (`starting_with(pat=…)`), and `\s` → `\\s` in a regex literal. - `base.html` reads `page` through a guard: Tera2 errors on an attribute lookup through an undefined variable instead of falling through to `default`, which 404.html hit on its first `page.description`. `default` also stopped covering null, so a page with no `description` needs an explicit `if` to reach the site-wide fallback. - Content is templated now, so the minijinja syntax the docs *document* (`{{ branch | hash_port }}`) has to be fenced off. `wrap_template_syntax` in `src/docs.rs` wraps the lines and fenced blocks that carry it, and `strip_template_syntax_wrappers` is its exact inverse; the sync test strips on read and wraps on write, so `docs/content/*.md` is authored and compared without the wrappers. - `cmd="…"` gains a `__WT_BSLASH__` placeholder: `\` is Tera2's string escape, so a shell continuation was a build error, and a sequence Tera2 recognises would have been rewritten silently. - Zola 0.23 emits palette-index highlight classes (`z-l-<n>`/`z-d-<n>`) instead of scope names, so the two `custom.scss` rules that named scopes are re-anchored, and the theme CSS is no longer written into `docs/static/`. Rendered output was diffed page by page against a 0.22.1 build: no content differences, only entity spelling (`'`→`'`), whitespace, and `<pre>` no longer nesting inside `<p>`. Fixes #3827
worktrunk-bot
left a comment
There was a problem hiding this comment.
Migration looks right, and I verified it independently rather than taking the PR body's word for it: built docs/ with the real Zola 0.23.3 binary against this head and with 0.22.1 against the merge base, then diffed every rendered page. Once highlight class names are normalised, the only differences are the ones the PR body names ('→', /→/, whitespace); sitemap.xml, robots.txt and llms.txt are byte-identical, and the quote+string pairs the .z-l-2 + .z-l-5 rule targets still number 207 on both builds. The palette mapping in custom.scss checks out against the generated giallo-light.css (.z-l-2 = #588A10, .z-l-5 = #947A00), and index 2 resolves to exactly one scope in worktrunk-light.json (punctuation.definition.string), so the selector isn't broader than the one it replaces.
One thing worth fixing before this lands, in wrap_template_syntax / strip_template_syntax_wrappers:
A documented {% raw %} or {% endraw %} alone on a line is silently deleted, and the result is a fixed point so the sync test never notices. wrap_template_syntax opens by calling strip_template_syntax_wrappers, which drops every line whose trim equals a marker — it can't tell an emitted wrapper from documented text. ENDRAW_ESCAPE doesn't rescue the standalone case either, because strip's first step converts it back to a bare {% endraw %} line and the line-drop then removes it. Concretely, this source:
```jinja
{% raw %}
{{ branch }}
{% endraw %}
```comes back from one wrap_template_syntax pass with both tag lines gone, the fence wrapped, and strip(wrap(src)) != src — but wrap(wrap(src)) == wrap(src), so test_docs_are_in_sync reports the page as in sync and the loss is committed into docs/content/. Nothing in the tree hits this today (I checked: all 15 pages under docs/content/ are fixed points of the pair, and strip of each one reproduces the pre-change source exactly), so it's an authoring hazard rather than a live bug — but the docstring here and the new paragraph in docs/CLAUDE.md both call strip "its exact inverse", which is what would make an author trust it. The cheap fix is to say what the limitation actually is (inline suggestion below); if you'd rather have it fail loudly, read_docs_page is the only place that can see the difference, since by the time write_docs_page runs the lines are already gone.
Minor: convert_console_blocks_in_docs now builds an errors vec that can never be non-empty (read_docs_page panics instead), so the error half of its return is dead weight — worth dropping the channel rather than leaving a tag() call that can only ever report zero.
Data-loss surface check
The repo's review guidance holds a PR that edits a file containing git worktree remove / git branch -d, and docs/content/worktrunk.md's comparison table is such a file — the {% rawcode() %} → <code class="multiline"> rewrite touches those exact cells. Nothing the diff can reach deletes anything: the cells are documentation text, and both the rendered page (homepage index.html, byte-identical after normalisation) and the copyable mirrors (README.md, skills/worktrunk/reference/, untouched by the diff, since MULTILINE_CODE_PATTERN reproduces what ZOLA_RAWCODE_PATTERN produced) come out unchanged. Flagging it so a human can confirm rather than treating co-location as a hold.
Verification commands
# 0.23.3 against this head, 0.22.1 against the merge base
zola build # in docs/, both trees — 14 pages, 0 orphan, both succeed
Normalised comparison over all 17 output HTML files: class="z-*" collapsed, '// folded, whitespace collapsed. 11 files differ, all of them only in z-code → z-l-code z-d-code on <pre>, or in spans that carried a scope class in 0.22 and carry none in 0.23 because the token resolves to the default foreground. No __WT_… leaks into the output; the only {% raw %} text in the built HTML is the four occurrences in extending/index.html and one in faq/index.html, all inside <code> as intended.
A `{% raw %}` or `{% endraw %}` alone on a line is dropped whether this
module emitted it or the page documents it, and the wrapped result is a
fixed point, so the sync test never flags the loss. Both docstrings and
docs/CLAUDE.md called strip an exact inverse, which is what would make an
author trust it with such a line.
Also drop convert_console_blocks_in_docs's error channel: read_docs_page
panics, so the vec could only ever be empty.
worktrunk-bot
left a comment
There was a problem hiding this comment.
Both findings from my earlier review are addressed in 4676851, and I checked the new text against the code rather than against the commit message: strip_template_syntax_wrappers does drop any line whose trim equals {% raw %} or {% endraw %} regardless of origin, so "recovers the source form" plus the explicit mid-line authoring rule is an accurate description where "exact inverse" wasn't; the extending.md / faq.md citation holds (both document the tags inside a line, via ENDRAW_ESCAPE). convert_console_blocks_in_docs returning a bare Vec<String> is right — read_docs_page panics and write_docs_page has no fallible branch, so there was never an error to report — and keeping the tag("console→terminal", Vec::new(), …) call is still doing work, since that's what prefixes the stage label onto the updated-file list.
Nothing further from me. Self-authored, so no approval — the data-loss-surface flag in my earlier review (docs/content/worktrunk.md's comparison table, which the {% rawcode() %} rewrite touches) still wants a human confirming it before merge, and CI is mid-flight on this head.
|
is there a way of avoiding this complication and verbosity? it's not great to have can we maybe set alternative delimeters or something like that? |
Wrapping was line-granular, so a page that documents template syntax
carried one marker pair per line and per fenced block — 179 across
docs/content, thickest where the prose explains template variables.
Component calls are the only lines that have to stay evaluable, so they
are the only thing a region has to break for: wrap now emits one region
per run of content between two calls, and prose without template syntax
inside that run rides along, since a raw region passes its text through
untouched. 179 markers become 30 regions; the built site is byte-identical
(zola 0.23.3, all 17 output files).
The markers ride on the region's first and last line instead of sitting on
lines of their own, which keeps a region inside a paragraph or list from
introducing a blank line — and, since both delimiters are then only ever
emitted at a line's very edge and a documented one inside a region travels
as an escape sequence, makes strip an exact inverse again. That closes the
hazard the previous commit could only document: a `{% raw %}` alone on a
line is no longer a shape wrap emits, so strip no longer deletes one the
page wrote itself.
Wrap no longer strips its input first; it takes source form, which is what
both call sites (`read_docs_page` → transform → `write_docs_page`) already
pass.
|
Alternative delimiters aren't reachable: Tera 2 added What I could do is stop wrapping line by line. Component calls are the only lines that have to stay evaluable, so they're the only thing a region has to break for; everything between two of them goes into one region, and prose without template syntax rides along, since a raw region passes its text through untouched. Markers now ride on the region's first and last line rather than sitting on lines of their own. 179 wrapper sites → 30 regions, pushed as The bit that got worst is now clean: - `{{ repo_path }}` — absolute path to the repository root
- `{{ repo }}` — repository directory name (e.g., `myproject`)
- `{{ branch }}` — raw branch name (e.g., `feature/auth`)…where before each of those eight lines carried its own It also makes Verification and sources
|
worktrunk-bot
left a comment
There was a problem hiding this comment.
The coalescing does what the thread asked for, and I checked the result rather than the description: docs/content/ drops from 222 marker-bearing lines to 30 regions (60 markers), config.md 60 → 8 and hook.md 62 → 6, and the built site is byte-identical to the previous head (zola 0.23.3 over both trees, diff -rq on the two public/ dirs → 0 differences). Independently of the tests, I also confirmed the re-wrap is lossless — for every page, stripping the new published form with the new algorithm reproduces exactly what stripping the old published form with the old one did — and fuzzed strip(wrap(src)) == src over 20k generated pages (fences, nested fences, component calls, bare and mid-line markers, missing trailing newline) with no failures or panics. On the delimiter question: Zola exposes no delimiter configuration — components/config/src/config/mod.rs has skip_content_templating and nothing else — so {% raw %} really is the only per-region opt-out, and the paragraph in docs/CLAUDE.md describes the option set correctly.
One thing to fix before this lands.
The hazard from the earlier review isn't gone — it changed shape, and the new text says it can't happen. strip_template_syntax_wrappers is an exact inverse of what wrap_template_syntax emits, not of arbitrary page text: it removes a leading {% raw %} and a trailing {% endraw %} from every line unconditionally, and the escape sequences that make those positions unambiguous only exist in text wrap has already been through. So a hand-authored fenced block documenting minijinja's own raw —
```jinja
{% raw %}
{{ branch }}
{% endraw %}
```— comes back from read_docs_page with both tag lines blanked rather than deleted, which is the same silent loss as before with a different mechanism, and wrap(strip(…)) of the damaged page is a fixed point, so the sync test flags it once (as a file it "updated") and never again. convert_console_blocks_in_docs runs read_docs_page → write_docs_page over every page in docs/content unconditionally, so this fires on the next sync run after the edit, on hand-written primaries like extending.md as much as on a new page. Nothing in the tree hits it today — I checked, no source page opens or closes a line with a bare marker — so it's still an authoring hazard rather than a live bug.
What makes it worth a change is that the previous revision documented it and this one asserts the opposite: "the page's own text round-trips whatever it says about these tags" and "a page can say anything about those tags and still round-trip" are the two sentences an author would rely on. The ambiguity is genuinely undecidable from the text alone (a bare marker opening a line is exactly what a region opener looks like), so scoping the claim and restoring the mid-line authoring rule is the right fix rather than trying to detect it — inline suggestions below.
Self-authored, so no approval from me.
strip_template_syntax_wrappers inverts what wrap emits, not arbitrary page
text: a hand-authored line that opens or closes with a bare {% raw %} /
{% endraw %} is blanked on read, and the re-wrapped result is a fixed point,
so the sync test flags it once and never again. Say so, and restore the
mid-line authoring rule.
Zola 0.23 replaced Tera 1 with Tera 2, removed shortcodes in favour of components, and started rendering every
.mdfile through the template engine —docs/did not build under any of those, so the weekly pin bump has been blocked at 0.22.1. This migrates the templates, the doc-generation pipeline, and the content, and bumps the pin inci.yamlandpublish-docs.yamlto 0.23.3. Verified by building the site with the official 0.23.3 binary and diffing the rendered HTML page by page against a 0.22.1 build ofmain: no content differences.Fixes #3827.
What changed
Templates.
macros.htmlandtemplates/shortcodes/{terminal,rawcode}.htmlbecome components in a newtemplates/components.html— components are the one mechanism 0.23 offers for both, and they are callable from markdown, which is what the generated terminal blocks need.rawcodeonly ever wrapped its body in a<code class="multiline">, so it is gone entirely and the four table rows inworktrunk.mdcarry that element directly.toc_navtakes the active page'spermalinkandtocrather than the page itself: a component parameter takes its type from its default, socurrent_page=falsetypes itbooland rejects a page. Passing the two fields it actually reads types cleanly.Tera2 syntax.
trim_end_matches→trim_end; thefilterfilter is gone, sodocs_section.pages | filter(attribute="slug", …) | firstbecomes a directget_page; test arguments are keyword-only (is starting_with(pat="#")); and"…\s…"is now an invalid string escape.Undefined and null in
base.html. Tera2 errors on an attribute lookup through an undefined variable instead of falling through todefault, which404.htmlhits on the firstpage.description. Separately,defaultno longer covers null, so a page whose front matter omitsdescriptionrenders""rather than the site-wide fallback. Both are handled by one guard plus explicitifs.Content is templated now. A documented minijinja example like
{{ branch | hash_port }}is read as a Zola expression and fails the build.skip_content_templatingdoes not rescue this repo — it is a per-file glob, and 8 of 13 content files carry both documented minijinja and terminal blocks, so skipping a file would disable the rendering that has to keep working.{% raw %}is the mechanism the upstream changelog points at, so:wrap_template_syntax(src/docs.rs) wraps the fenced blocks and lines that carry template syntax, leaving component calls evaluable. It is line-granular, so each wrap's output is byte-identical to its input and the markers stay off the ~95% of lines that have no template syntax.strip_template_syntax_wrappersis its exact inverse. The sync test strips on read (read_docs_page) and wraps on write (write_docs_page), sodocs/content/*.mdis authored and compared in source form — no step downstream has to know the wrappers exist, and the skill/README renderings are byte-identical to before.extending.mdandfaq.mddocument minijinja's ownraw, and a region cannot contain its own terminator; those occurrences are emitted through a string expression and restored on strip.__WT_BSLASH__.\is Tera2's string-escape character, so a shell continuation insidecmd="…"was anunexpected escape characterbuild error — and a sequence Tera2 does recognise would have been rewritten silently. It joins the existing__WT_QUOT__/__WT_OPEN__/__WT_CLOSE__placeholders.Highlight CSS. 0.23 emits palette-index classes (
z-l-<n>/z-d-<n>) instead of scope names, and writes the theme CSS to the output directory rather than intodocs/static/. The twocustom.scssrules that named scopes are re-anchored with the mapping recorded in a comment, anddocs/.gitignoredrops the now-unusedstatic/giallo*.cssline.Verification
Rendered-output diff, all 14 pages plus
404.html,index.html,sitemap.xml,llms.txt, against a 0.22.1 build of the pre-change tree. After normalising highlight class names, the only differences are:'→'andhttps://→https://— 0.23 escapes fewer characters; renders identically.<pre>no longer nested inside<p>(0.22 pulled the shortcode into the paragraph, producing invalid HTML).<td><code>…</code></td>on one line instead of a stray newline before</td>.No
{% … %}or__WT_…leaks into the output; the only{% raw %}text remaining in the built HTML is the documented minijinja examples inextending.mdandfaq.md, rendering as intended.Tests:
cargo test --lib,cargo test --test integration(one unrelated failure,test_copy_ignored_preserves_file_executable_permissions, is a sandboxumask 002artifact — it asserts 0644 and gets 0664;mainis green on the same test),cargo clippy --all-targets,cargo fmt.test_docs_are_in_syncpasses and is idempotent across repeated runs, which is what pins the strip/wrap pair.