Skip to content

Reorganize docs around deliverables and build modes - #3733

Merged
Mpdreamz merged 38 commits into
mainfrom
docs/re-reorg
Jul 28, 2026
Merged

Reorganize docs around deliverables and build modes#3733
Mpdreamz merged 38 commits into
mainfrom
docs/re-reorg

Conversation

@Mpdreamz

@Mpdreamz Mpdreamz commented Jul 27, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Why

The docs were organized around authoring tasks (syntax, configure, contribute) which made sense when docs-builder was primarily a docs tool. Now that it has multiple deliverables (assembler, codex, isolated builds, exporters, MCP, API, release notes, OpenAPI/CLI schema support), newcomers had no clear path to understand what docs-builder produces or how to onboard to a specific build mode.

What

Complete restructuring of the documentation from an authoring-centric layout to a deliverable-centric one:

  • Getting Started — decision tree, installation, serve & preview (with HUD screenshots), writing content, pages & links, publish (with GitHub Pages + branding)
  • Syntax — top-level (preserves /syntax/ URLs linked from elastic.co)
  • Documentation Builds — distributed model, cross-links, then per-mode sections (isolated → assembler → codex), each with scoped configuration, navigation, URLs, and infrastructure docs
  • Structured Data — MCP, REST API, OpenAPI, CLI Schema, exporters (Elasticsearch, LLM, OKF, plain text), release notes
  • Elastic Integrations — GitHub Actions, Elastic CLI
  • CLI Reference
  • Development Notes — trimmed from 23 pages to 4 with deprecation warnings

Dedicated test docset: docs-tests/

Test fixture content (redirects, mover pages, nested structures, settings YAML, release notes fixtures) is now separated into docs-tests/ with its own docset.yml. This cleanly splits real published documentation (docs/) from content that only exists to exercise test scenarios.

  • docs-builder serve defaults to docs/ (the real docs)
  • docs-builder serve --path docs-tests picks up the test docset
  • All .NET tests (NavigationTests, MoverTests, BreadCrumbTests, SettingsInclusion, etc.) now build against docs-tests/ instead of docs/testing/
  • New test fixtures can be added to docs-tests/ without cluttering the published docs

All 2505 tests pass (1913 Markdown + 592 Configuration).

Also removes obsolete migration docs and unsupported-feature stubs, and adds comprehensive redirects for every moved path.

Mpdreamz and others added 30 commits July 27, 2026 12:25
@Mpdreamz @claude @cursoragent
Reorganize docs around deliverables and structured data
b61b553 
Restructure the documentation from an authoring-centric layout to a
deliverable-centric one, making it clearer what docs-builder produces
and providing a smoother onramp for newcomers.

New top-level sections:
- getting-started/ — decision tree, installation, serve, writing, pages & links
- documentation/ — assembler, codex, isolated builds + syntax, configure, how-to
- data/ (Structured Data) — MCP, API, OpenAPI, CLI Schema, exporters, release notes
- integrations/ — docs-actions, elastic CLI
- architecture/ — renamed from building-blocks, reframed as service architecture

Key changes:
- Remove asciidoc migration docs (no longer relevant)
- Separate test fixtures into docs-tests/ with own docset.yml
- Remove unsupported-feature stub pages (sidebars, passthrough, etc.)
- Add comprehensive redirects for all moved paths
- New content: assembler/codex/isolated build docs, exporters (Elasticsearch,
  LLM, OKF, plain text), docs-actions, elastic CLI, REST API placeholder,
  release notes explorer placeholder, serve & preview with HUD screenshots

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Merge architecture and development into single development section
b903fcf 
Combine the 11-page architecture/ section and the 12-page development/
section into a single 8-page development/ section organized by subsystem.

New pages:
- building-blocks.md — conceptual overview with mermaid diagrams
  (docset lifecycle, distributed model, assembly flow, link resolution)
- link-infrastructure.md — merges link-service, link-index, link-catalog,
  outbound/inbound cross-links into one page with mermaid diagrams
- navigation.md — merges 8 navigation sub-pages into 1 focused page
  (keeps SVG diagrams, drops C# implementation detail)

Other changes:
- Replace ASCII art in changelog-bundle-registry.md with mermaid diagram
- Flatten ingest/ and link-validation/ subdirectories
- Move all images to development/images/
- Add redirects for all old architecture/ and development/navigation/ paths
- Update all cross-references throughout the docs

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Fix build errors from docs reorg
64fe228 
- Fix broken relative path to developer-notes.md from deep in configure/
- Fix absolute include paths for release-notes snippets (now under data/)
- Replace unsupported grid directive with list in navigation.md

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Rename Integrations to Elastic Integrations, simplify docs-actions page
a060d42 
- Integrations nav title → Elastic Integrations
- docs-actions nav title → GitHub
- Simplify docs-actions page to link to the repo rather than
  attempting to document it; note it covers publishing, validation,
  style enforcement, and process enforcement

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Rename Development to Development Notes, add deprecation warnings
d74f995 
Mark all development pages as old development notes likely to be
deleted, with a warning admonition on each page and the landing page.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Move syntax guide to top-level folder after Getting Started
13f525d 
Preserves the /syntax/ URL path that elastic.co/docs links to directly.
Add documentation/syntax/ → syntax/ redirects for the intermediate location.
Update all cross-references throughout the docs.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Add publish page to getting started
27d304d 
Cover GitHub Pages publishing with the docs-builder action for isolated
builds. Link to assembler/codex docs for more complex publishing setups.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Add branding section to publish page
fb2f5c4 
Document the branding: docset.yml config for white-labelling. Note that
docs-builder is org-agnostic — Elastic chrome only applies when the
GitHub org is 'elastic'.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Merge shell autocompletion into installation, remove build from source
3afc8f9 
- Fold shell-autocompletion.md content into installation.md
- Replace build-from-source section with a link to the GitHub repo
- Add redirects for old shell-autocompletion paths

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Rework documentation section by build mode
f23df28 
Restructure so each build mode is its own section with scoped
configuration, how-to guides, and infrastructure reference:

- Isolated (baseline): docset.yml config, frontmatter, general how-tos
- Assembler: site config, content sources, infrastructure, assembler how-tos
- Codex: environment.yml config, infrastructure

Renames: content-set-level → docset.yml, page-level → Frontmatter,
site-level → assembler.yml. Content sources moved directly under
assembler. Added infrastructure pages for both assembler and codex.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Move products.yml and versions.yml to top-level Catalog section
9c3d3a2 
Products and versions are global metadata, not assembler-specific.
Give them their own top-level section between Documentation and
Structured Data.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Add version substitution docs to versions.yml page
0bad9b2 
Document version mutations (M.x, M.M, M+1, etc.) with examples
and link to the substitutions syntax reference.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Move synonyms to Catalog Configuration as search.yml
da73f7e 
Rename from synonyms.yml to search.yml (matching the actual config
file name). Simplify to a brief description linking to the
Elasticsearch exporter.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Restructure isolated section: URLs, navigation, extensions, CTA as to…
822a5a6 
…p-level pages

- file-structure → isolated/urls.md (URL structure)
- navigation concept → isolated/navigation.md (simple overview)
- navigation reference stays at content-set/navigation.md (full reference)
- extensions and cta moved from content-set/ to isolated/ (top-level)
- content-set/index.md becomes docset.yml reference

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Move redirects to top-level under Documentation
9456f6a 
Redirects apply across build modes, not just isolated. Highlight that
they are currently only implemented for assembler and codex builds.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Merge navigation reference and attributes into docset.yml reference
a98721e 
- navigation.md reference content → content-set/index.md (docset.yml ref)
- Navigation patterns → isolated/navigation.md (concept page)
- attributes.md → inlined as 'subs' section in docset.yml ref, links to
  substitutions syntax. Term 'attribute' replaced with 'substitution'.
- Remove separate navigation.md and attributes.md pages

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Flatten isolated configure to single docset.yml reference page
2c3ca63 
Remove empty page.md shell and configure index. Promote content-set
index to be the configure entry point with nav title 'Configuration'.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Reorder isolated sub-nav: Configuration, Navigation, URLs, CTA, Exten…
ff84905 
…sions

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Move legacy-url-mappings to Catalog Configuration
2ae6fdf 
Add admonition noting this is Elastic-specific migration config
that should not be manually edited.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Restructure assembler sub-nav: Configuration, Navigation, URLs, Infra…
5a580c9 
…structure

- Flatten configure/site/ into configure/ (assembler.yml reference)
- Extract navigation.yml as its own page under assembler
- Add URLs page explaining path_prefix composition
- Reorder: Configuration → Navigation → URLs → How-to → Infrastructure

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Flatten assembler configure: content.md becomes index, content-source…
4265f88 
…s moves up

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Add navigation_title to assembler navigation page
b215495 
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Create Assembler Environments page, consolidate content sources and b…
682b84d 
…ranching

- New environments.md merging content-sources + branching-strategy:
  publish environments, content source phases, branching strategies,
  CI configuration for tagged branching
- Delete content-sources.md, branching-strategy.md, release-new-version.md
- Move add-repo.md up from how-to/ to assembler/
- Remove empty how-to/ subfolder
- Add redirects for all moved/deleted paths

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Fix content sources section: per-repo config, not shared_configuration
f03a9c5 
Content source phases are declared per repository in assembler.yml,
defaulting to main/main for continuous deployment.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Add Navigation and URLs pages to Codex section
9cf616d 
Navigation: codex renders isolated nav as-is, landing page groups by
codex.group. URLs: /g/group, /r/repository, then isolated URL structure.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Add Cross-links page under Documentation with mermaid diagrams
fe30fe5 
New page covering cross-link syntax, dependency declaration, validation
flow (with sequence diagram), link index lifecycle, build resilience,
and inbound validation. Placed after Catalog Configuration, with
Redirects moved to follow it.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Split Cross-links + Distributed builds, delete absorbed dev notes
30f9ff4 
- Split cross-links.md into:
  - cross-links.md: syntax, declaring deps, validation, inbound checks
  - distributed-builds.md: link index, link service, link catalog,
    build resilience, assembler/codex composition
- Delete development/building-blocks.md, link-infrastructure.md,
  link-validation.md — all useful content absorbed into the new pages
- Update all cross-references

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Fix all build errors: redirects, broken links, stale cross-references
63803f6 
Rewrite _redirects.yml with correct targets for all moved pages.
Fix ~30 broken relative/absolute links across content files after
the documentation-by-build-mode restructure. Zero errors/warnings.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Rename Documentation → Documentation Builds, Distributed builds → Dis…
0844e9c 
…tributed

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Move Distributed and Cross-links above Isolated in nav
eccf62b 
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Fix tests for docs-tests separation
634c09e 
Point NavigationTestsBase at docs-tests/ docset instead of docs/.
Update all test assertions for the new path structure:
- Remove testing/ prefix from redirect, breadcrumb, and mover paths
- Fix relative path depths in mover assertions
- Read settings YAML from docs-tests/ on disk
- Add nested-toc test fixture to docs-tests/
- Update PhysicalDocsetTests for new docs/ structure
- Rename _redirects.yml to redirects.yml (matches docset.yml convention)

All 1913 Markdown tests and 592 Configuration tests pass.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz
Mpdreamz requested review from a team as code owners July 27, 2026 14:24
@Mpdreamz Mpdreamz added the documentation Improvements or additions to documentation label Jul 27, 2026
@Mpdreamz
Mpdreamz requested a review from cotti July 27, 2026 14:24
theletterf
theletterf approved these changes Jul 27, 2026
View reviewed changes

@theletterf theletterf left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

Mpdreamz and others added 6 commits July 27, 2026 16:52
@Mpdreamz @claude @cursoragent
Fix CI failures: Navigation.Tests + redirect validation for testing/ …
46f12d9 
…paths

- Update Navigation.Tests/PhysicalDocsetTests for new docs/ structure
  (cross-links test now uses docs-tests/ docset)
- Add redirects for all 28 testing/* paths to satisfy CI diff-validate
  (test fixtures moved to docs-tests/, redirects point to index.md)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Fix all CI failures: authoring tests, Navigation.Tests, redirect vali…
7f4b6e3 
…dation

- Update authoring Setup.fs to read redirect fixtures from docs-tests/
- Restore complex redirect test entries in _redirects.yml (needed by
  cross-link redirect authoring tests)
- Regenerate links.json snapshot with new page paths and redirect entries
- Add development/toc.yml (assembler requires named toc section)
- Update Navigation.Tests/PhysicalDocsetTests for new structure
- Add testing/* redirects for CI diff-validate

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Fix all CI failures: authoring tests, redirect validation, assembler nav
ca558f0 
Root cause: authoring tests read docs/_redirects.yml from disk and need
complex redirect entries (with anchors, many:, etc.) for cross-link
redirect testing. But the assembler validates that redirect targets
exist as real files — the complex entries point to testing/redirects/*
which only exist in the mock, not on disk.

Fix: Setup.fs now merges docs-tests/redirects.yml entries into the
mock _redirects.yml at test time, with path prefix rewriting
(redirects/ → testing/redirects/). The real _redirects.yml stays
simple (testing/* → index.md) for assembler validation.

Also fixed:
- copyTargetsFromRealDocsIntoMock checks docs-tests/ for testing/* paths
- Final redirects.yml write uses merged version (was overwriting with original)
- !redirects/ path prefix handled in replacements
- PhysicalDocsetTests: development is now a toc: ref, not a folder
- Regenerated links.json snapshot with correct redirect structure

All 3007 tests pass (1913 + 599 + 181 + 314).

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Fix remaining CI failures
274f616 
- Add missing testing/redirects/* entries to _redirects.yml for
  CI diff-validate (third-page, 4th-page, 6th-page, 7th-page,
  9th-page, first-page-old, second-page-old)
- Add development/link-validation toc section required by assembler
  navigation config
- Fix Windows line ending issue in redirect YAML merge (handle \r\n)
- Strip simple redirect entries before merging complex test entries
  to avoid YAML duplicate key errors

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz @claude @cursoragent
Simplify _redirects.yml: remove intermediate-state redirects
3fee79d 
Remove ~100 lines of redirects for paths that only existed within
this PR's commit history (architecture/*, documentation/configure/*,
documentation/syntax/*, documentation/how-to/*, etc.). Keep only
redirects from main's actual paths to their final destinations.

178 lines down from 284.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Mpdreamz
Merge branch 'main' into docs/re-reorg
c0b5c0f
@Mpdreamz
Mpdreamz enabled auto-merge (squash) July 28, 2026 09:54
@Mpdreamz
Mpdreamz merged commit e5c3338 into main Jul 28, 2026
24 checks passed
@Mpdreamz
Mpdreamz deleted the docs/re-reorg branch July 28, 2026 10:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@theletterf theletterf theletterf approved these changes
@cotti cotti Awaiting requested review from cotti cotti is a code owner automatically assigned from elastic/docs-engineering

Assignees

No one assigned

Labels

documentation Improvements or additions to documentation

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Mpdreamz @theletterf