Make Lewis-structure perception independent of PYTHONHASHSEED - #989
Closed
calvinp0 wants to merge 2 commits into
Closed
Make Lewis-structure perception independent of PYTHONHASHSEED#989calvinp0 wants to merge 2 commits into
calvinp0 wants to merge 2 commits into
Conversation
Graph.get_all_edges() de-duplicated the edges through a set of Edge objects and returned list(edge_set), so the order came out of the set's iteration. Atom and Bond hash on their symbols and bond order rather than on identity, and those hashes are derived from string hashes, which Python randomises per process. Every carbon in a molecule therefore lands in one hash bucket and the resulting edge order differs from one process to the next. Molecule perception consumes that order: generate_lewis_structure() walks the bond list in an A* search whose equal-cost states are explored in the order the bonds are listed, so a molecule with several equal-cost Lewis structures could be perceived differently in different processes. Diphenylprolinol methyl ether was perceived with a methoxy C=O double bond and a carbene ring carbon in about 2% of hash seeds, which then failed RDKit's valence check. Return the edges in the graph's vertex order instead, de-duplicating on the edge identities, so the order is the same in every process.
calvinp0
force-pushed
the
fix_hash_seed_dependent_perception
branch
from
August 16, 2026 22:19
f5adf52 to
872d8e2
Compare
The edge order returned by get_all_edges() used to follow the iteration order of a set of Edge objects, which is governed by the per-process randomized string hash, so this asserts the property directly: two subprocesses started at different PYTHONHASHSEED values must report the same order. The child processes are given PYTHONPATH explicitly. A subprocess inherits the parent's working directory but not pytest's sys.path, so without it the child imports whichever ARC `import arc` resolves to -- which, with an editable install present, is not necessarily the tree under test. The test then either fails spuriously when run from outside the repository root, or passes while having validated a different checkout.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #989 +/- ##
==========================================
- Coverage 64.15% 64.11% -0.05%
==========================================
Files 119 119
Lines 39564 39564
Branches 10265 10265
==========================================
- Hits 25383 25366 -17
- Misses 11206 11221 +15
- Partials 2975 2977 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This was referenced Aug 17, 2026
Member
Author
|
Superseded by #992, which combines this PR with #991. The two cannot land separately. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ARC perceives a different Lewis structure depending on the process's
PYTHONHASHSEED. Same molecule, same code, same commit — the answer depends on which process it runs in.That is
COC1=C(CN[C@H]2C3CCN(CC3)[C@H]2C(C2=CC=CC=C2)C2=CC=CC=C2)C=C(C=C1)C(C)Cbuilt from the same xyz both times. Roughly 2% of seeds give the wrong structure.The chain
arc/molecule/graph.pyx::get_all_edgesde-duplicated through a set and returnedlist(edge_set):So the returned order was the set's iteration order.
AtomandBondtake content-derived hashes —hash(('Atom', self.symbol)),hash(('Bond', order, sym1, sym2))— while__eq__is identity. Every carbon therefore collides into one bucket, and the resulting order is governed by Python's per-process randomized string hash.arc/species/perceive.py::generate_lewis_structureconsumes that order directly:and runs an A* that expands equal-cost states in
bond_pairsorder. The molecule above has two degenerate minimum-cost Lewis structures, so the tie is broken by the hash seed.Seed 35 places the double bond on the methoxy C–O rather than inside the anisole ring, leaving ring carbon #5 with three single bonds and a lone pair — a carbene.
to_rdkit_mol's carbene handling then sets two radical electrons on it, giving explicit valence 5, and RDKit refuses to sanitize.Why this matters beyond the crash
Perception decides bond orders. Bond orders feed
bond_corrections, which feed Arkane's BAC, which feed energies and rates. A calculation is therefore not guaranteed to be reproducible across processes — including between a run and its own restart.No downstream numerical impact has been measured. That is stated as an open question, not a claim.
The fix
Return edges in the graph's vertex order, de-duplicating on edge identity rather than on hash-and-equality:
Same edges, same complexity, order identical in every process.
This is deliberately at the producer.
get_all_edgesalso feedspy_rdlSSSR ring perception (graph.pyx:962,:995), so fixing the order where it is manufactured fixes every consumer at once, rather than sorting at each call site.How it was found
A CI run of an unrelated PR failed on one xdist worker with
IndexError: list index out of rangefromconformers_test.py::test_embed_rdkit. It looked like flakiness, then like cross-test contamination — both wrong. Replaying that worker's exact 90-test prefix node-by-node passes. Fourteen full-suite runs and ~8,300 per-test canary checks came back clean because they varied test order; the variable was the seed.Each xdist worker is a separate process with its own hash seed, which is why parallel CI surfaced it and serial runs never did.
Checks
PYTHONHASHSEEDvalues against the affected molecule: 588/600 pass before, 600/600 after, all reporting one identical bond census. Known-bad seeds were 35, 79, 87, 91, 166.bond_pairsorder for seeds 1 and 35: different before, identical after.arc/molecule/graph_test.py: 41 → 43 passed. The new test asserts the vertex-order contract and spawns two subprocesses at differentPYTHONHASHSEEDvalues, asserting identical edge order — it fails against the unfixed code.arc/species/conformers_test.pyserial: 48 passed, unchanged.-n 6 --dist worksteal: 2745 → 2747 passed, same 5 pre-existingtorch_anifailures either side.Deliberately not addressed
is_mol_valid. A neutral carbene carbon with three single bonds and a lone pair should not surviveget_octet_deviation. This change makes the affected molecule always take the good branch, but the validity gate remains permissive, and another molecule's degenerate optimum could land on a carbene deterministically. That wants its own change.Atom.__hash__/Bond.__hash__are content-derived while__eq__is identity. Legal, but it makes every atom or bond set degenerate to a single bucket per element — an O(n²) hazard as well as an ordering one. Changing it would trade a seed-dependent order for an address-dependent one, so it fixes nothing on its own.get_all_edgesis where that order escaped into a chemical decision.