fix: MPI runs for smoke tests with surfaceGenerator#4068
Conversation
|
@jhuang2601 was |
|
Right @rrsettgast, it was not effective in resolving the issue for these two examples. |
There was a problem hiding this comment.
Pull request overview
This PR aims to make MPI smoke-test results involving SurfaceGenerator deterministic so CI baselines don’t fluctuate between runs. It does this by (1) forcing consistent MPI communication ordering in the relevant XML inputs and (2) ensuring deterministic iteration/ordering for the element-location map used during surface generation.
Changes:
- Enable deterministic MPI message handling for the two affected smoke tests by setting
mpiCommOrder="1"in their XML inputs. - Replace pointer-address-based ordering of
elemLocationswith a deterministic comparator based on element region/subregion indices. - Update the ALM ATS decks to run both serial and MPI partitions, and bump integrated test baselines accordingly.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
src/coreComponents/physicsSolvers/surfaceGeneration/SurfaceGenerator.hpp |
Introduces a deterministic comparator and an ElemLocMapType alias; updates function signatures to use it. |
src/coreComponents/physicsSolvers/surfaceGeneration/SurfaceGenerator.cpp |
Switches elemLocations to the deterministic map type and updates insertion/iteration accordingly. |
inputFiles/poromechanicsFractures/Contact/AugmentedLagrangianMultipliers/AugmentedLagrangianMultipliers.ats |
Adds an MPI partition configuration for the two smoke tests. |
inputFiles/poromechanicsFractures/Contact/AugmentedLagrangianMultipliers/ALM_inclinedFault_ISG_smoke.xml |
Enables mpiCommOrder="1" for deterministic MPI comm ordering in the test. |
inputFiles/poromechanicsFractures/Contact/AugmentedLagrangianMultipliers/ALM_inclinedFault_ISG_DruckerPrager_smoke.xml |
Enables mpiCommOrder="1" for deterministic MPI comm ordering in the test. |
BASELINE_NOTES.md |
Records the baseline update for PR #4068. |
.integrated_tests.yaml |
Points integrated tests to the new PR #4068 baseline tarball. |
jafranc
left a comment
There was a problem hiding this comment.
Clever fix ! I can't think of any tie-blocking insertion configuration. 👍
…erator.hpp Co-authored-by: Jacques Franc <49998870+jafranc@users.noreply.github.com>
Another attempt to fix CI failures for MPI runs of two newly added smoke tests using
surfaceGenerator, see PR#4062The original code is functionally correct but generates results ordering which is not strictly deterministic across runs.
mpiCommOrder="1"in the XML inputselemLocationswith a comparator using deterministic region/subregion indices