fix: use ungated llama tokenizer mirrors

[eexwhyzee]

Note

Medium Risk
Central tokenizer loading now depends on third-party mirror repos for two production model IDs; a bad mirror change could affect templates/encoding until overrides are reviewed, though trust_remote_code stays off and overrides are narrowly scoped.

Overview
Gated Meta Llama-3.2 Instruct tokenizers can be loaded without HuggingFace license access by routing load_tokenizer (and offset-tokenizer reloads) through audited unsloth mirror repos while callers still pass canonical meta-llama/Llama-3.2-*-Instruct IDs.

Adds TOKENIZER_SOURCE_OVERRIDES plus helpers that pick the load repo, apply existing trust/revision policy on the mirror path, and rewrite tokenizer.name_or_path back to the requested Meta ID so MODEL_RENDERER_MAP auto-resolution still picks Llama3Renderer.

Shared test matrices now use the canonical Meta model name with "auto" instead of calling the mirror directly; new unit tests cover mirror selection, name preservation, and offset-tokenizer behavior.

^{Reviewed by Cursor Bugbot for commit 0dc19a0. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Fix tokenizer loading for gated Meta Llama-3.2 models by routing to ungated unsloth mirrors

• Adds TOKENIZER_SOURCE_OVERRIDES in renderers/base.py mapping canonical meta-llama/Llama-3.2-1B-Instruct and 3B-Instruct IDs to their ungated unsloth mirrors, so tokenizer loading no longer fails for users without Hugging Face access to the gated repos.
• Introduces _tokenizer_source_for and _tokenizer_load_kwargs helpers to apply overrides and compute trust/revision kwargs consistently across load_tokenizer and _get_offset_tokenizer.
• Adds _preserve_requested_tokenizer_name to ensure the returned tokenizer's name_or_path always reflects the originally requested canonical model ID, not the mirror path.
• Updates tests to use canonical meta-llama/ IDs and adds coverage for mirror routing, name preservation, and offset-tokenizer reload behavior.

^{Macroscope summarized 0dc19a0.}

[macroscopeapp]

Approvability

Verdict: Needs human review

This PR changes runtime tokenizer loading behavior by redirecting Meta Llama models to load from unsloth mirrors while preserving canonical names for renderer auto-resolution. The new source override logic and name preservation mechanism warrant human review to verify correctness.

^{You can customize Macroscope's approvability policy. Learn more.}