docs: add spec-style description of the consolidated metadata format

[d-v-b]

Summary

This is a claude-authored contribution that adds specification-style documentation for how zarr-python implements consolidated metadata. I'm pretty busy these days with a newborn baby so I can't give this careful review. I am opening this as a draft and leaving it to other folks to push it forward.

cc @normanrz

🤖 AI text below 🤖

Add a new user-guide page that describes exactly what zarr-python reads and writes for consolidated metadata in Zarr formats 2 and 3, so that other implementations can interoperate. Quotes and attributes the schema text from zarr-specs#309 (Tom Augspurger, CC-BY-4.0), documents the key ordering, the empty child-group marker, the v2 .zmetadata layout and its deviation from zarr-python 2.x, and the reader/writer procedures.

Also correct the 3.1.1 sort-order note on the existing page, which said "lexicographic" while the implementation uses NFKC-casefolded ordering.

Assisted-by: ClaudeCode:claude-fable-5

Author attestation

• I am a human, these are my changes, and I have reviewed and understood every change and can explain why each is correct.

TODO

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/user-guide/*.md
• Changes documented as a new file in changes/
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

[d-v-b]

@TomAugspurger you should probably also have a look, since this is an LLM summarizing your work

[TomAugspurger]

I'm pretty busy these days with a newborn baby so I can't give this careful review.

Congrats! I'll take a look when I get a chance.

Do you have any more info on the motivation for this document. I gather it's primarily for other implementations looking to interoperate with what we write, and am trying to balance this approach vs. telling them to read the spec + source code :)

[normanrz]

I asked for a spec on Zulip and Davis was kind enough to create one. The motivation is that we picked up the consolidated metadata in the newly-formed Zarr Format Working Group and I wanted to get a better understanding of the current zarr-python implementation. Having a spec here would be great, but even better would be if we could eventually bring this document into the Zarr 3 spec.

[d-v-b]

Do you have any more info on the motivation for this document. I gather it's primarily for other implementations looking to interoperate with what we write, and am trying to balance this approach vs. telling them to read the spec + source code :)

IMO we don't really have a place for a consolidated metadata spec in the zarr-specs repo until we unblock the blockage that prevented your original PR from getting merged in some form or another. For folks curious about the normative structure of consolidated metadata, but also zarr-python's particular implementation choices, I think some kind of document in our docs is a good play. Long term our goal should be to replace an actual spec with a link to a spec defined elsewhere.

[normanrz]

IMO we don't really have a place for a consolidated metadata spec in the zarr-specs repo until we unblock the blockage that prevented your original PR from getting merged in some form or another.

Right. With @joshmoore, I am working on a proposal for the ZFWG governance and I could imagine using consolidated metadata as testcase for the governance process.