Use kernelSpecs to explicitly instantiate fluid models and resolve CUDA builds#4060
Open
dkachuma wants to merge 21 commits into
Open
Use kernelSpecs to explicitly instantiate fluid models and resolve CUDA builds#4060dkachuma wants to merge 21 commits into
kernelSpecs to explicitly instantiate fluid models and resolve CUDA builds#4060dkachuma wants to merge 21 commits into
Conversation
kernelSpecs to explicitly instantiate fluid models and resolve CUDA builds
Contributor
There was a problem hiding this comment.
Pull request overview
This PR restructures several compositional fluid-flow kernels to reduce CUDA compilation memory pressure by moving heavy template implementations into _impl.hpp files and shifting dispatch/instantiation into generated and dedicated translation units, restoring GPU availability for previously disabled multifluid models.
Changes:
- Split large templated kernel implementations into
_impl.hppheaders and add explicit instantiation generation viakernelSpecs.json. - Introduce dedicated factory
.cpptranslation units for Dirichlet flux kernel dispatch/launch to keep solver TUs smaller. - Re-enable previously GPU-disabled multifluid models by removing
GEOS_DEVICE_COMPILEguards and related runtime disables.
Reviewed changes
Copilot reviewed 26 out of 26 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/coreComponents/physicsSolvers/fluidFlow/wells/CompositionalMultiphaseWell.cpp | Switch FluidUpdateKernel usage to explicitly-typed kernel specialization. |
| src/coreComponents/physicsSolvers/fluidFlow/kernelSpecs.json | Define explicit instantiation matrices for Dirichlet flux, fluid update, hydrostatic kernels. |
| src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/zFormulation/DirichletFluxComputeZFormulationKernelFactory.cpp | New TU for Z-formulation Dirichlet flux dispatch/launch. |
| src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/zFormulation/DirichletFluxComputeZFormulationKernel.hpp | Convert to declaration-only + new template parameter ordering (wrapper, policy, sizes). |
| src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/zFormulation/DirichletFluxComputeZFormulationKernel_impl.hpp | Add implementation header for Z-formulation Dirichlet flux kernel. |
| src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/ThermalDirichletFluxComputeKernel.hpp | Convert to declaration-only and align template params with isothermal base. |
| src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/ThermalDirichletFluxComputeKernel_impl.hpp | Add implementation header for thermal Dirichlet flux kernel. |
| src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/HydrostaticPressureKernel.hpp | Make HydrostaticPressureKernel wrapper-templated and move implementations out-of-line. |
| src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/HydrostaticPressureKernel_impl.hpp | Add implementation header for hydrostatic pressure kernel (centralized constants). |
| src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/FluidUpdateKernel.hpp | Refactor FluidUpdateKernel into a class template declared in header. |
| src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/FluidUpdateKernel_impl.hpp | Add implementation header for FluidUpdateKernel launches. |
| src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/DirichletFluxComputeKernelFactory.cpp | New TU for isothermal/thermal Dirichlet flux factory dispatch/launch. |
| src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/DirichletFluxComputeKernel.hpp | Convert to declaration-only and align template params (wrapper, policy, sizes). |
| src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/DirichletFluxComputeKernel_impl.hpp | Add implementation header for isothermal Dirichlet flux kernel. |
| src/coreComponents/physicsSolvers/fluidFlow/CompositionalMultiphaseFVMKernel_thermal.cpp.template | Remove old generator template (superseded by new kernelSpecs-driven templates). |
| src/coreComponents/physicsSolvers/fluidFlow/CompositionalMultiphaseFVMKernel_isothermal.cpp.template | Remove old generator template (superseded by new kernelSpecs-driven templates). |
| src/coreComponents/physicsSolvers/fluidFlow/CompositionalMultiphaseFVM.cpp | Update calls to new factory types / createAndLaunch style. |
| src/coreComponents/physicsSolvers/fluidFlow/CompositionalMultiphaseFVM_ThermalDirichletFluxComputeKernels.cpp.template | New generated explicit-instantiation template for thermal Dirichlet flux kernels. |
| src/coreComponents/physicsSolvers/fluidFlow/CompositionalMultiphaseFVM_IsothermalDirichletFluxComputeKernels.cpp.template | New generated explicit-instantiation template for isothermal Dirichlet/Z-formulation kernels. |
| src/coreComponents/physicsSolvers/fluidFlow/CompositionalMultiphaseBase.cpp | Switch FluidUpdateKernel and HydrostaticPressureKernel usage to explicit specializations. |
| src/coreComponents/physicsSolvers/fluidFlow/CompositionalMultiphaseBase_HydrostaticPressureKernel.cpp.template | New generated explicit-instantiation template for hydrostatic pressure kernel. |
| src/coreComponents/physicsSolvers/fluidFlow/CompositionalMultiphaseBase_FluidUpdateKernel.cpp.template | New generated explicit-instantiation template for fluid update kernel. |
| src/coreComponents/physicsSolvers/fluidFlow/CMakeLists.txt | Wire new sources/templates and remove legacy Dirichlet kernel generator calls. |
| src/coreComponents/constitutive/fluid/multifluid/MultiFluidSelector.hpp | Remove GPU compile guards to restore full model selection on GPU. |
| src/coreComponents/constitutive/fluid/multifluid/CO2Brine/CO2BrineFluid.hpp | Remove GPU-disabling pre-subgroup initialization override declaration. |
| src/coreComponents/constitutive/fluid/multifluid/CO2Brine/CO2BrineFluid.cpp | Remove GPU runtime disable/throw for Ezrokhi thermal CO2-brine model. |
Comment on lines
+16
to
+21
| /** | ||
| * @file DirichletFluxComputeKernel_impl.hpp | ||
| */ | ||
|
|
||
| #include "DirichletFluxComputeKernel.hpp" | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In
CompositionalMultiphaseBase.cppandCompositionalMultiphaseWell.cppcalls toFluidUpdateKernelinvolved a construct of the form:In
DirichletFluxComputeKernelFactorywe haveThe first is a loop over the different types of multiphase fluid models while the second also includes, in addition to this, a loop over the number of components. When these loops are expanded in place, the size of the compilation units for
CompositionalMultiphaseBase.cpp,CompositionalMultiphaseWell.cppandCompositionalMultiphaseFVM.cppbecome too large. This causes out-of-memory issues for the CUDA compilation on most machines including the CI machines. The "solution" for this was to disable some fluid models for GPU (see MultiFluidSelector.hpp:53-63)). This means that some models were not available for GPU even though they were developed for GPU execution.This PR tries to resolve the out-of-memory issues by breaking up the massive translation units and offloading template instantiations into auto generated, independent source files.
.hppfiles and into dedicated_impl.hppfilesDirichletFluxComputeKernel_impl.hpp,FluidUpdateKernel_impl.hpp,HydrostaticPressureKernel_impl.hpp.kernelSpecs.jsongenerator to explicitly instantiate the combinations that are required for these kernels..cppfiles (e.g.,DirichletFluxComputeKernelFactory.cpp) to handle the heavycreateAndLaunchdispatch logic, keeping it out of the core solver classes.#if !defined(GEOS_DEVICE_COMPILE)guards inMultiFluidSelector.hppandCO2BrineFluid.cpp, restoring full GPU support for all previously disabled multiphase fluid models likeCO2BrineEzrokhiThermalFluidandCompositionalTwoPhaseLohrenzBrayClarkViscosity.