Skip to content

feat: automate type generation with append-only version log#207

Open
sameh-farouk wants to merge 2 commits intomasterfrom
master-automate-typegen
Open

feat: automate type generation with append-only version log#207
sameh-farouk wants to merge 2 commits intomasterfrom
master-automate-typegen

Conversation

@sameh-farouk
Copy link
Copy Markdown
Member

@sameh-farouk sameh-farouk commented Mar 31, 2026

Summary

  • Automate src/types/ generation via make typegen using Subsquid typegen tooling
  • Add append-only version log (typegen/tfchainVersions.jsonl) tracking all processed spec versions
  • Add scripts/seed-versions.sh to bootstrap the version log from chain metadata
  • Add scripts/merge-versions.js to merge new versions into the existing log
  • Rename type references in all mapping files to match generated output (e.g. v122v123, v124v127)
  • Remove stale/duplicate type definitions that were artifacts of manual maintenance
  • Update docs/typeChanges.md with complete version change documentation

Related Issues

Test plan

  • npx tsc --noEmit passes — no type errors
  • make typegen produces identical output on re-run (deterministic)
  • Full re-index against archive node produces same GraphQL results

Replace hand-maintained src/types/ with auto-generated types from
Subsquid typegen, seeded from all TFChain networks (devnet, qanet,
testnet, mainnet). JSONL sorted by specVersion so typegen assigns
the lowest version label for each unique event hash.

Infrastructure:
- Add scripts/merge-versions.js: append-only JSONL merger, sorted output
- Add scripts/seed-versions.sh: multi-network metadata discovery
  (order: devnet -> qanet -> testnet -> mainnet)
- Seed typegen/tfchainVersions.jsonl with 93 specVersions from all networks
- Update typegen/typegen.json: output directly to src/types/, add 3 events
  to config that were used by processor but missing from typegen config
  (FarmCertificationSet, NodeCertificationSet, FarmingPolicyUpdated)
- Add Makefile targets: typegen, typegen-seed, typegen-add
- Rewrite docs/typeChanges.md with automated workflow, deployment order,
  resync guidance, and notes on typesBundle.json being frozen

Type generation:
- Regenerate src/types/ from chain metadata (correct hashes)
- Remove src/typesDevelopment/ (no longer needed)
- Remove src/types/context.ts (unused)

Mapping renames (all verified: identical hashes and type structures):
- V49 -> V9: FarmDeleted, FarmPayoutV2, NodeDeleted, NodeUptimeReported,
  TwinDeleted, TwinStored, TwinUpdated, PricingPolicyStored
- V49 -> V19: NodeContractCanceled, NameContractCanceled, PriceStored
- V49 -> V12: NodePublicConfigStored
- V101 -> V63: FarmStored, FarmUpdated, NodeStored, NodeUpdated
- V122 -> V123: All ServiceContract events
- V124 -> V125: TwinStored, TwinUpdated
- V134 -> V140: NodeExtraFeeSet
- Removed dead branches with phantom hashes from wrong field ordering
  (FarmStored/Updated V63, NodeStored/Updated V63, TwinStored/Updated V49)
- Removed unused v63 import from farms.ts
sameh-farouk added a commit that referenced this pull request Apr 3, 2026
Existing indexer snapshots on all networks were built with an old
typesBundle that had a typo: "dedicatedFarm:" (trailing colon) in the
Farm struct. The stored JSON has "dedicatedFarm:" as the key, but the
processor expects "dedicatedFarm" (no colon) after the typesBundle was
corrected. This causes an assertion failure when decoding pre-V14
FarmStored/FarmUpdated events (specs 63-70).

Production 2.12.3 had this workaround in the isV101 branch, which was
removed by the typegen automation (PR #207) when relabeling the hash
as isV63.

The patch sets dedicatedFarm = false on the raw args before decoding.
This is safe because farmStored hardcodes dedicatedFarm = false anyway.
Remove this workaround after all indexers are resynced with the
corrected typesBundle.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@sameh-farouk sameh-farouk force-pushed the master-automate-typegen branch from a4fb1b7 to f97b850 Compare April 4, 2026 01:12
Existing indexer snapshots on all networks were built with an old
typesBundle that had a typo: "dedicatedFarm:" (trailing colon) in the
Farm struct. The stored JSON has "dedicatedFarm:" as the key, but the
processor expects "dedicatedFarm" (no colon) after the typesBundle was
corrected. This causes an assertion failure when decoding pre-V14
FarmStored/FarmUpdated events (specs 63-70).

Patch the raw event args by reading the actual value from the colon key
(falling back to the normal key for corrected indexers, then false for
versions that lack the field entirely), instead of hardcoding false
which discarded the real on-chain value.

Also read dedicatedFarm from the parsed event in farmStored instead of
hardcoding false, so farms created as dedicated are recorded correctly.

Production 2.12.3 had the workaround in the isV101 branch, which was
removed by the typegen automation when relabeling the hash as isV63.
Remove this workaround after all indexers are resynced with the
corrected typesBundle.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@sameh-farouk sameh-farouk force-pushed the master-automate-typegen branch from f97b850 to 94fdaf4 Compare April 6, 2026 14:43
@sameh-farouk
Copy link
Copy Markdown
Member Author

sameh-farouk commented Apr 7, 2026

Tetsed on test vm
No regression found.
inconsitnet data:

  • Twin 18589 accountID: test_vm=5HVuXQ... (correct, matches latest on-chain TwinUpdated), devnet=5FEt3k... (stale, from TwinStored) due to this fix 614c79a but devnet processor didn’t resync after deploy the fix so it has stale data (operational not code issue)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Type generation is manual and error-prone — needs automation typegen directory and events inconsistency

1 participant