Skip to content

docs: migrate the site to Zola 0.23 (Tera2) and bump the pin - #3840

Open
worktrunk-bot wants to merge 5 commits into
mainfrom
fix/issue-3827-zola-023
Open

docs: migrate the site to Zola 0.23 (Tera2) and bump the pin#3840
worktrunk-bot wants to merge 5 commits into
mainfrom
fix/issue-3827-zola-023

Conversation

@worktrunk-bot

Copy link
Copy Markdown
Collaborator

Zola 0.23 replaced Tera 1 with Tera 2, removed shortcodes in favour of components, and started rendering every .md file through the template engine — docs/ did not build under any of those, so the weekly pin bump has been blocked at 0.22.1. This migrates the templates, the doc-generation pipeline, and the content, and bumps the pin in ci.yaml and publish-docs.yaml to 0.23.3. Verified by building the site with the official 0.23.3 binary and diffing the rendered HTML page by page against a 0.22.1 build of main: no content differences.

Fixes #3827.

What changed

Templates. macros.html and templates/shortcodes/{terminal,rawcode}.html become components in a new templates/components.html — components are the one mechanism 0.23 offers for both, and they are callable from markdown, which is what the generated terminal blocks need. rawcode only ever wrapped its body in a <code class="multiline">, so it is gone entirely and the four table rows in worktrunk.md carry that element directly.

toc_nav takes the active page's permalink and toc rather than the page itself: a component parameter takes its type from its default, so current_page=false types it bool and rejects a page. Passing the two fields it actually reads types cleanly.

Tera2 syntax. trim_end_matchestrim_end; the filter filter is gone, so docs_section.pages | filter(attribute="slug", …) | first becomes a direct get_page; test arguments are keyword-only (is starting_with(pat="#")); and "…\s…" is now an invalid string escape.

Undefined and null in base.html. Tera2 errors on an attribute lookup through an undefined variable instead of falling through to default, which 404.html hits on the first page.description. Separately, default no longer covers null, so a page whose front matter omits description renders "" rather than the site-wide fallback. Both are handled by one guard plus explicit ifs.

Content is templated now. A documented minijinja example like {{ branch | hash_port }} is read as a Zola expression and fails the build. skip_content_templating does not rescue this repo — it is a per-file glob, and 8 of 13 content files carry both documented minijinja and terminal blocks, so skipping a file would disable the rendering that has to keep working. {% raw %} is the mechanism the upstream changelog points at, so:

  • wrap_template_syntax (src/docs.rs) wraps the fenced blocks and lines that carry template syntax, leaving component calls evaluable. It is line-granular, so each wrap's output is byte-identical to its input and the markers stay off the ~95% of lines that have no template syntax.
  • strip_template_syntax_wrappers is its exact inverse. The sync test strips on read (read_docs_page) and wraps on write (write_docs_page), so docs/content/*.md is authored and compared in source form — no step downstream has to know the wrappers exist, and the skill/README renderings are byte-identical to before.
  • extending.md and faq.md document minijinja's own raw, and a region cannot contain its own terminator; those occurrences are emitted through a string expression and restored on strip.

__WT_BSLASH__. \ is Tera2's string-escape character, so a shell continuation inside cmd="…" was an unexpected escape character build error — and a sequence Tera2 does recognise would have been rewritten silently. It joins the existing __WT_QUOT__ / __WT_OPEN__ / __WT_CLOSE__ placeholders.

Highlight CSS. 0.23 emits palette-index classes (z-l-<n> / z-d-<n>) instead of scope names, and writes the theme CSS to the output directory rather than into docs/static/. The two custom.scss rules that named scopes are re-anchored with the mapping recorded in a comment, and docs/.gitignore drops the now-unused static/giallo*.css line.

Verification

Rendered-output diff, all 14 pages plus 404.html, index.html, sitemap.xml, llms.txt, against a 0.22.1 build of the pre-change tree. After normalising highlight class names, the only differences are:

  • &#x27;&#39; and https:&#x2F;&#x2F;https:// — 0.23 escapes fewer characters; renders identically.
  • <pre> no longer nested inside <p> (0.22 pulled the shortcode into the paragraph, producing invalid HTML).
  • <td><code>…</code></td> on one line instead of a stray newline before </td>.
  • Whitespace between block elements.

No {% … %} or __WT_… leaks into the output; the only {% raw %} text remaining in the built HTML is the documented minijinja examples in extending.md and faq.md, rendering as intended.

Tests: cargo test --lib, cargo test --test integration (one unrelated failure, test_copy_ignored_preserves_file_executable_permissions, is a sandbox umask 002 artifact — it asserts 0644 and gets 0664; main is green on the same test), cargo clippy --all-targets, cargo fmt. test_docs_are_in_sync passes and is idempotent across repeated runs, which is what pins the strip/wrap pair.

Zola 0.23 replaced Tera 1 with Tera 2, removed shortcodes in favour of
components, and started rendering every `.md` file through the template
engine. `docs/` did not build under any of those, so the weekly pin bump
was blocked at 0.22.1 (#3827).

- `macros.html` and `templates/shortcodes/` become components in
  `templates/components.html`. `toc_nav` takes the active page's
  permalink and toc rather than the page, since a component parameter
  takes its type from its default and there is no default that is both a
  page and a "no active page" sentinel.
- Tera2 syntax fixes: `trim_end_matches` → `trim_end`, the `filter`
  filter (gone) → `get_page`, keyword-only test arguments
  (`starting_with(pat=…)`), and `\s` → `\\s` in a regex literal.
- `base.html` reads `page` through a guard: Tera2 errors on an attribute
  lookup through an undefined variable instead of falling through to
  `default`, which 404.html hit on its first `page.description`. `default`
  also stopped covering null, so a page with no `description` needs an
  explicit `if` to reach the site-wide fallback.
- Content is templated now, so the minijinja syntax the docs *document*
  (`{{ branch | hash_port }}`) has to be fenced off. `wrap_template_syntax`
  in `src/docs.rs` wraps the lines and fenced blocks that carry it, and
  `strip_template_syntax_wrappers` is its exact inverse; the sync test
  strips on read and wraps on write, so `docs/content/*.md` is authored
  and compared without the wrappers.
- `cmd="…"` gains a `__WT_BSLASH__` placeholder: `\` is Tera2's string
  escape, so a shell continuation was a build error, and a sequence Tera2
  recognises would have been rewritten silently.
- Zola 0.23 emits palette-index highlight classes (`z-l-<n>`/`z-d-<n>`)
  instead of scope names, so the two `custom.scss` rules that named
  scopes are re-anchored, and the theme CSS is no longer written into
  `docs/static/`.

Rendered output was diffed page by page against a 0.22.1 build: no
content differences, only entity spelling (`&#x27;`→`&#39;`), whitespace,
and `<pre>` no longer nesting inside `<p>`.

Fixes #3827

@worktrunk-bot worktrunk-bot left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Migration looks right, and I verified it independently rather than taking the PR body's word for it: built docs/ with the real Zola 0.23.3 binary against this head and with 0.22.1 against the merge base, then diffed every rendered page. Once highlight class names are normalised, the only differences are the ones the PR body names (&#x27;&#39;, &#x2F;/, whitespace); sitemap.xml, robots.txt and llms.txt are byte-identical, and the quote+string pairs the .z-l-2 + .z-l-5 rule targets still number 207 on both builds. The palette mapping in custom.scss checks out against the generated giallo-light.css (.z-l-2 = #588A10, .z-l-5 = #947A00), and index 2 resolves to exactly one scope in worktrunk-light.json (punctuation.definition.string), so the selector isn't broader than the one it replaces.

One thing worth fixing before this lands, in wrap_template_syntax / strip_template_syntax_wrappers:

A documented {% raw %} or {% endraw %} alone on a line is silently deleted, and the result is a fixed point so the sync test never notices. wrap_template_syntax opens by calling strip_template_syntax_wrappers, which drops every line whose trim equals a marker — it can't tell an emitted wrapper from documented text. ENDRAW_ESCAPE doesn't rescue the standalone case either, because strip's first step converts it back to a bare {% endraw %} line and the line-drop then removes it. Concretely, this source:

```jinja
{% raw %}
{{ branch }}
{% endraw %}
```

comes back from one wrap_template_syntax pass with both tag lines gone, the fence wrapped, and strip(wrap(src)) != src — but wrap(wrap(src)) == wrap(src), so test_docs_are_in_sync reports the page as in sync and the loss is committed into docs/content/. Nothing in the tree hits this today (I checked: all 15 pages under docs/content/ are fixed points of the pair, and strip of each one reproduces the pre-change source exactly), so it's an authoring hazard rather than a live bug — but the docstring here and the new paragraph in docs/CLAUDE.md both call strip "its exact inverse", which is what would make an author trust it. The cheap fix is to say what the limitation actually is (inline suggestion below); if you'd rather have it fail loudly, read_docs_page is the only place that can see the difference, since by the time write_docs_page runs the lines are already gone.

Minor: convert_console_blocks_in_docs now builds an errors vec that can never be non-empty (read_docs_page panics instead), so the error half of its return is dead weight — worth dropping the channel rather than leaving a tag() call that can only ever report zero.

Data-loss surface check

The repo's review guidance holds a PR that edits a file containing git worktree remove / git branch -d, and docs/content/worktrunk.md's comparison table is such a file — the {% rawcode() %}<code class="multiline"> rewrite touches those exact cells. Nothing the diff can reach deletes anything: the cells are documentation text, and both the rendered page (homepage index.html, byte-identical after normalisation) and the copyable mirrors (README.md, skills/worktrunk/reference/, untouched by the diff, since MULTILINE_CODE_PATTERN reproduces what ZOLA_RAWCODE_PATTERN produced) come out unchanged. Flagging it so a human can confirm rather than treating co-location as a hold.

Verification commands
# 0.23.3 against this head, 0.22.1 against the merge base
zola build   # in docs/, both trees — 14 pages, 0 orphan, both succeed

Normalised comparison over all 17 output HTML files: class="z-*" collapsed, &#x27;/&#x2F; folded, whitespace collapsed. 11 files differ, all of them only in z-codez-l-code z-d-code on <pre>, or in spans that carried a scope class in 0.22 and carry none in 0.23 because the token resolves to the default foreground. No __WT_… leaks into the output; the only {% raw %} text in the built HTML is the four occurrences in extending/index.html and one in faq/index.html, all inside <code> as intended.

Comment thread src/docs.rs Outdated
A `{% raw %}` or `{% endraw %}` alone on a line is dropped whether this
module emitted it or the page documents it, and the wrapped result is a
fixed point, so the sync test never flags the loss. Both docstrings and
docs/CLAUDE.md called strip an exact inverse, which is what would make an
author trust it with such a line.

Also drop convert_console_blocks_in_docs's error channel: read_docs_page
panics, so the vec could only ever be empty.

@worktrunk-bot worktrunk-bot left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both findings from my earlier review are addressed in 4676851, and I checked the new text against the code rather than against the commit message: strip_template_syntax_wrappers does drop any line whose trim equals {% raw %} or {% endraw %} regardless of origin, so "recovers the source form" plus the explicit mid-line authoring rule is an accurate description where "exact inverse" wasn't; the extending.md / faq.md citation holds (both document the tags inside a line, via ENDRAW_ESCAPE). convert_console_blocks_in_docs returning a bare Vec<String> is right — read_docs_page panics and write_docs_page has no fallible branch, so there was never an error to report — and keeping the tag("console→terminal", Vec::new(), …) call is still doing work, since that's what prefixes the stage label onto the updated-file list.

Nothing further from me. Self-authored, so no approval — the data-loss-surface flag in my earlier review (docs/content/worktrunk.md's comparison table, which the {% rawcode() %} rewrite touches) still wants a human confirming it before merge, and CI is mid-flight on this head.

@max-sixty

Copy link
Copy Markdown
Owner

is there a way of avoiding this complication and verbosity? it's not great to have raw & endraw littered across the code

can we maybe set alternative delimeters or something like that?

Wrapping was line-granular, so a page that documents template syntax
carried one marker pair per line and per fenced block — 179 across
docs/content, thickest where the prose explains template variables.
Component calls are the only lines that have to stay evaluable, so they
are the only thing a region has to break for: wrap now emits one region
per run of content between two calls, and prose without template syntax
inside that run rides along, since a raw region passes its text through
untouched. 179 markers become 30 regions; the built site is byte-identical
(zola 0.23.3, all 17 output files).

The markers ride on the region's first and last line instead of sitting on
lines of their own, which keeps a region inside a paragraph or list from
introducing a blank line — and, since both delimiters are then only ever
emitted at a line's very edge and a documented one inside a region travels
as an escape sequence, makes strip an exact inverse again. That closes the
hazard the previous commit could only document: a `{% raw %}` alone on a
line is no longer a shape wrap emits, so strip no longer deletes one the
page wrote itself.

Wrap no longer strips its input first; it takes source form, which is what
both call sites (`read_docs_page` → transform → `write_docs_page`) already
pass.
@worktrunk-bot

Copy link
Copy Markdown
Collaborator Author

Alternative delimiters aren't reachable: Tera 2 added Tera::set_delimiters, but Zola never calls it and exposes no config for it (set_delimiters has zero hits in getzola/zola), so there's no knob from a site. Zola's only two opt-outs are {% raw %} and the whole-file skip_content_templating glob added in 0.23 — and skipping a file takes its terminal calls with it, so it can't help here: every page that documents template syntax is also a page with terminal calls (config.md 41 wrappers / 52 calls, hook.md 53 / 5, tips-patterns.md 20 / 17). The only way to switch content templating off wholesale is to stop invoking components from markdown, which means giving up Giallo highlighting of commands or the CSS-generated $ prompt — worth doing on purpose, not as a side effect of this migration, so I left it alone.

What I could do is stop wrapping line by line. Component calls are the only lines that have to stay evaluable, so they're the only thing a region has to break for; everything between two of them goes into one region, and prose without template syntax rides along, since a raw region passes its text through untouched. Markers now ride on the region's first and last line rather than sitting on lines of their own. 179 wrapper sites → 30 regions, pushed as 3638954, and the built site is byte-identical — zola build at 0.23.3 before and after, diff -rq clean across all 17 output files.

The bit that got worst is now clean:

- `{{ repo_path }}` — absolute path to the repository root
- `{{ repo }}` — repository directory name (e.g., `myproject`)
- `{{ branch }}` — raw branch name (e.g., `feature/auth`)

…where before each of those eight lines carried its own {% raw %}…{% endraw %}. Per page: config.md 41 → 4, hook.md 53 → 3, extending.md 26 → 5, tips-patterns.md 20 → 8, llm-commits.md 18 → 1.

It also makes strip_template_syntax_wrappers an exact inverse again, which closes the hazard I could only document last time: markers are now only ever emitted at a line's very edge, and a documented delimiter inside a region travels as an escape sequence, so a {% raw %} alone on a line is no longer a shape wrap emits and strip no longer deletes one a page wrote itself.

Verification and sources
  • Custom delimiters: Tera MIGRATION.md, "Custom delimiters"Tera::set_delimiters exists in v2. gh api "search/code?q=set_delimiters+repo:getzola/zola"total_count: 0, so Zola neither calls it nor exposes it.
  • skip_content_templating: Zola 0.23.0 changelog ("There is also a new config option, skip_content_templating, to skip templating for some of the files"). It's a glob matched against the file path in components/site/src/md_render.rs (render_page / render_section) — whole-file, no per-block or front-matter form.
  • Site diff: zola build (real 0.23.3 binary) on 4676851 and on this head, diff -rq over the two output trees → no differences at all, so the coalescing changes nothing rendered.
  • Sync test round-trips: cargo test --test integration readme_sync (15 passed) after regenerating; docs/content is a fixed point of read→write, and README.md / skills/worktrunk/reference/ are untouched by the regeneration, confirming the wrappers stay a published-form detail.
  • Region shape: markers break only at terminal calls, so a page like config.md ends up with one long region between its quick-start calls and its generated help region. That's the honest description of the content — literal except for the component calls.

@worktrunk-bot worktrunk-bot left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The coalescing does what the thread asked for, and I checked the result rather than the description: docs/content/ drops from 222 marker-bearing lines to 30 regions (60 markers), config.md 60 → 8 and hook.md 62 → 6, and the built site is byte-identical to the previous head (zola 0.23.3 over both trees, diff -rq on the two public/ dirs → 0 differences). Independently of the tests, I also confirmed the re-wrap is lossless — for every page, stripping the new published form with the new algorithm reproduces exactly what stripping the old published form with the old one did — and fuzzed strip(wrap(src)) == src over 20k generated pages (fences, nested fences, component calls, bare and mid-line markers, missing trailing newline) with no failures or panics. On the delimiter question: Zola exposes no delimiter configuration — components/config/src/config/mod.rs has skip_content_templating and nothing else — so {% raw %} really is the only per-region opt-out, and the paragraph in docs/CLAUDE.md describes the option set correctly.

One thing to fix before this lands.

The hazard from the earlier review isn't gone — it changed shape, and the new text says it can't happen. strip_template_syntax_wrappers is an exact inverse of what wrap_template_syntax emits, not of arbitrary page text: it removes a leading {% raw %} and a trailing {% endraw %} from every line unconditionally, and the escape sequences that make those positions unambiguous only exist in text wrap has already been through. So a hand-authored fenced block documenting minijinja's own raw

```jinja
{% raw %}
{{ branch }}
{% endraw %}
```

— comes back from read_docs_page with both tag lines blanked rather than deleted, which is the same silent loss as before with a different mechanism, and wrap(strip(…)) of the damaged page is a fixed point, so the sync test flags it once (as a file it "updated") and never again. convert_console_blocks_in_docs runs read_docs_pagewrite_docs_page over every page in docs/content unconditionally, so this fires on the next sync run after the edit, on hand-written primaries like extending.md as much as on a new page. Nothing in the tree hits it today — I checked, no source page opens or closes a line with a bare marker — so it's still an authoring hazard rather than a live bug.

What makes it worth a change is that the previous revision documented it and this one asserts the opposite: "the page's own text round-trips whatever it says about these tags" and "a page can say anything about those tags and still round-trip" are the two sentences an author would rely on. The ambiguity is genuinely undecidable from the text alone (a bare marker opening a line is exactly what a region opener looks like), so scoping the claim and restoring the mid-line authoring rule is the right fix rather than trying to detect it — inline suggestions below.

Self-authored, so no approval from me.

Comment thread src/docs.rs Outdated
Comment thread docs/CLAUDE.md Outdated
strip_template_syntax_wrappers inverts what wrap emits, not arbitrary page
text: a hand-authored line that opens or closes with a bare {% raw %} /
{% endraw %} is blanked on read, and the re-wrapped result is a fixed point,
so the sync test flags it once and never again. Say so, and restore the
mid-line authoring rule.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Zola 0.23 (Tera2) breaks the docs build — zola pin blocked at 0.22.1

2 participants