feat(autograd): reduce the TapeM surface to one lemma, and prove it - #31
Open
NicolasRouquette wants to merge 1 commit into
Open
feat(autograd): reduce the TapeM surface to one lemma, and prove it#31NicolasRouquette wants to merge 1 commit into
NicolasRouquette wants to merge 1 commit into
Conversation
The pure tape engine is well covered by proofs; the TapeM wrapper users actually write eager programs in was not, so a `do`-block had no route back to a statement about the tape it built. Every TapeM op wrapper that threads the tape is the same reshuffle: read the tape, run the pure op, write the tape back, return the node id. This names that shape once as `TapeM.opM`, so each wrapper is definitionally `opM` at its own pure counterpart, and adds `NN.Proofs.Autograd.Tape.Builder`: * `opM_run_ok` / `opM_run_error` carry a pure op's outcome to the monadic `run`, and `opM_run_inv` carries it back; * `run_bind_inv` splits a successful run of `m >>= f` into its two stages — what peels a `do`-block one statement at a time — with `exec_inv` turning a successful `exec` into such a run; * `run_<op>_ok` for all 31 tape-threading wrappers, each the proof term `opM_run_ok` with `g` supplied and nothing else. If one stops typechecking, its wrapper has stopped being `opM` at its pure op, which is the fact worth learning. `run_leaf` is stated separately because `leaf` is total. Everything is generic in the carrier; `opM` is generic in the returned value. The layer contributes no axioms of its own: `opM` is axiom-free and the three `opM_run_*` lemmas depend on `propext` alone. Where a `run_<op>_ok` reports more — `Classical.choice` and `Quot.sound` for `conv` and `multiHeadAttention`, `Quot.sound` for `sum` — the profile is exactly that of the pure op it names, which those ops already carry. NN/Tests/Runtime/Rationals/TapeBuilderTest.lean does both things to one program: runs it and checks the value, and proves from nothing but "the block executed successfully" the pure-Tape fact behind each of its four statements. lake build: 4139 jobs, no errors or warnings. nn_tests_suite: all curated tests pass, including tape_builder_test (Rat). scripts/checks/repo_lint.py: OK, no issues found.
NicolasRouquette
added a commit
to NicolasRouquette/TorchLean
that referenced
this pull request
Sep 5, 2026
Brings in the upstream PR branch (lean-dojo#31): TapeM.opM, the Proofs.Autograd.Builder family of run lemmas for all 31 tape-threading wrappers, and the rationals TapeBuilderTest that peels a do-block. Pure proof/test layer — no runtime, native or FFI surface is touched, so the CUDA work already on combined is unaffected.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gap
The pure tape engine (
Runtime.Autograd.Tape) is well covered by proofs.TapeM— theStateT (Tape α) Resultwrapper that users actually write eager programs in — has no proofs atall:
git grep -l TapeMonmainfinds the runtime definition, the training helpers, and threetest files, and nothing under
NN/Proofs/. So ado-block has no route back to a statement aboutthe tape it built, and a proof about a program has to be written against a hand-threaded
Tape.<op>chain that is not the program.The observation this rests on
Every
TapeMwrapper that threads the tape is the same four lines:There are 31 of these, differing only in the pure op and its binders. This PR names the shape once:
so each wrapper is definitionally
opMat its own pure counterpart —TapeM.mul aId bIdisopM fun t => Tape.mul (t := t) aId bId. The existing wrappers are left exactly as they are; theidentity is real either way.
What the PR adds
NN/Runtime/Autograd/Engine/TapeM.leangainsTapeM.opM(20 lines, definition + docstring).NN/Proofs/Autograd/Tape/Builder.lean(new) proves, all generic in the carrierα:opM_run_ok/opM_run_error— a pure op's outcome becomes the monadicrun's outcome. Thepair swaps: a pure op returns tape-then-value,
runreturns value-then-state.opM_run_inv— and back again.run_bind_inv— a successfulrunofm >>= fsplits into its two successful stages. This iswhat peels a
do-block one statement at a time.exec_inv— a successfulexecis a successfulrunthat returned something.run_leaf— stated separately, becauseleafis total: its pure counterpart returns a bare pairrather than a
Result.run_<op>_okfor all 31 tape-threading wrappers —add,sub,mul,div,scale,abs,sqrt,clamp,max,min,relu,linear,matmul,conv,convTranspose,maxPool,smoothMaxPool,avgPool,layerNorm,batchNorm,multiHeadAttention,mseLoss,sigmoid,tanh,softmaxLast,softplus,exp,log,inv,safeLog,sum.Each member of the family is the proof term
opM_run_okwithgsupplied and nothing else — nounfolding lemma in between:
That is the property worth having: if one of these ever stops typechecking, its wrapper has stopped
being
opMat its pure op. The family cannot drift into stating something weaker than the wrapperdoes, because it is not stating anything separately.
TapeM.backwardScalaris deliberately outside the family — it reads the tape without writing oneback, so it is a different shape.
Axiom profile
The layer contributes nothing of its own. Verified, not assumed:
Where a per-op lemma reports more, it is inherited from the op it names, not introduced here:
The test does both things to one program
NN/Tests/Runtime/Rationals/TapeBuilderTest.leantakes a user-style block overℚ,runs it and checks the value (
4 * ([2,3] * [5,7])is[40, 84]), and proves — from nothing but"the block executed successfully" — the pure-
Tapefact behind each of its four statements:The proof is the peel and nothing else:
exec_inv, thenrun_bind_invonce per statement, thenrun_leaforopM_run_invat each op.Tape.emptyhas no native implementation available to the elaborator, so the numeric half cannotbe forced by a compile-time
#guard; it runs undernn_tests_suite, which is where the note in thefile points.
Checks run
PR checklist
lake buildsucceeds — 4139 jobs, no errors and no warnings.TapeBuilderTest, wired intoTests.Runtime.Rationals.Suite.sorryinNN/.Optional follow-up, not in this PR
The 31 wrappers could have their bodies replaced by
opM <their g>, which would makethe identity manifest instead of incidental. It is
defeqeither way,so the lemmas here hold unchanged.
Kept out so this PR stays purely additive; happy to do it in a second PR if preferred.
Notes for Nicolas (not part of the PR body)
git push -u origin tapem-run-lemmas, then open the PR againstlean-dojo/TorchLean:main.AI_USAGE.mdupstream discloses AI assistance at the repository level; nothing in this PR needsa per-PR disclosure beyond whatever you normally add.
cuda-arch-targetbefore this;tapem-run-lemmaswas cut fromupstream/main, so it carries none of the CUDA work.examples/.../TapeMBridge.leanis the file this generalizes. If this merges, that fileshrinks to the demo plus the eager-provenance endpoint — its
opM, the threeopM_run_*lemmas,run_bind_inv,exec_invand its 13 per-op lemmas all come from upstream instead, and theyarrive generic in the carrier rather than fixed at
ℝ.