Skip to content

Improve the default HTML parser and document normalizer extension#70

Draft
gschlager wants to merge 3 commits into
mainfrom
html-parser-headings-and-code-language
Draft

Improve the default HTML parser and document normalizer extension#70
gschlager wants to merge 3 commits into
mainfrom
html-parser-headings-and-code-language

Conversation

@gschlager

@gschlager gschlager commented Jul 25, 2026

Copy link
Copy Markdown
Member

While reviewing an HTML migration adapter a colleague built on Markbridge, I found several gaps in the default HTML parser, the renderer, and the docs. This PR closes them. This is the first of two PRs from that review — the deeper design changes (ancestry matching for AST subclasses and the AST::Code block flag) are stacked on top in #72.

  • The HTML handler registry had no handler for <h1><h6>, so headings from HTML input lost their structure and only their text survived. There is now a HeadingHandler that stores the level in the existing AST::Heading node; the renderer side already worked through the auto-registered HeadingTag.
  • RawHandler used the whole class attribute as the code language, so styling classes like hljs ended up on the fence line, and it never looked at the <code> child inside <pre>, where the CommonMark convention puts the language-* class. The language now comes from the first of: a language-* class on the element or its direct <code> child, the lang attribute, or a lone class used as-is. The result must be a single clean token, so it is always safe on the fence line.
  • Empty elements left junk in the output: <code></code> rendered as a bare `` pair, an empty details block as an empty [details] shell, and an empty spoiler as `[spoiler][/spoiler]`. All three now render to nothing.
  • The html_mode contract check now ships with the gem: require "markbridge/rspec" provides the shared example "an html_mode safe tag", so adapter authors can put their custom tags under the same check we run against the built-in tags. The check itself moved into Renderers::Discourse::HtmlBlockSafety and our contract spec runs through the shared example.
  • The HTML parser docs no longer show a handler signature with a processor: parameter that does not exist, a RawHandler.new call without its required argument, or lambda handlers, which the parser does not support. The docs site build output is now in .gitignore.

Behavior changes to be aware of: an explicit lang attribute now wins over a lone class without language- prefix, because a class can be pure styling while lang is always set on purpose. And empty code/details/spoiler elements disappear from the output instead of leaving empty shells.

@gschlager
gschlager force-pushed the html-parser-headings-and-code-language branch 2 times, most recently from 41a9d5e to db56159 Compare July 25, 2026 09:15
The default HTML handler registry had no handler for h1-h6, so
headings from HTML input lost their structure and only their text
survived. The new HeadingHandler stores the level in the existing
AST::Heading node; the renderer side already worked through the
auto-registered HeadingTag.

The html_mode contract check also ships with the gem now:
require "markbridge/rspec" provides the shared example
"an html_mode safe tag", backed by the new
Renderers::Discourse::HtmlBlockSafety module. Our own contract spec
runs through the shared example, so the shipped check is exercised
by the same suite that guards the built-in tags.
RawHandler used the whole class attribute as the code language, so
styling classes like "hljs" ended up on the fence line, and it never
looked at the <code> child inside <pre>, where the CommonMark
convention puts the language-* class. The language now comes from the
first of: a language-* class on the element or its direct <code>
child, the lang attribute, or a lone class used as-is. A lone class
ranks below lang because a class can be pure styling. The result must
be a single clean token, otherwise the fence line would break.

Empty elements also left junk in the output: an empty AST::Code
rendered as a bare `` pair, an empty Details as an empty [details]
shell, and an empty Spoiler as [spoiler][/spoiler]. All three now
render to nothing. An empty AST::Quote already collapsed to nothing
through its blank-line bracketing, so it needed no change.
@gschlager
gschlager force-pushed the html-parser-headings-and-code-language branch 3 times, most recently from 9f0a7c6 to 40de262 Compare July 25, 2026 20:48
The HTML parser docs showed a handler signature with a processor:
parameter that the parser never passes, a RawHandler example without
the required element_class argument, and lambda handlers, which the
parser does not support — it always calls #process on the registered
handler. The examples now match the real API.

The Astro docs tooling leaves docs/.astro, docs/dist,
docs/node_modules, and .pnpm-store in the working tree. They were
untracked but not ignored, so staging a directory could sweep them
into a commit. docs/src stays visible because it is real source on
the docs branch.
@gschlager
gschlager force-pushed the html-parser-headings-and-code-language branch from 55da43b to 7e8e0c8 Compare July 27, 2026 08:49
@gschlager
gschlager marked this pull request as draft July 27, 2026 09:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants