Skip to content

Use kernelSpecs to explicitly instantiate fluid models and resolve CUDA builds#4060

Open
dkachuma wants to merge 21 commits into
developfrom
dkachuma/fluid-kernel-specs
Open

Use kernelSpecs to explicitly instantiate fluid models and resolve CUDA builds#4060
dkachuma wants to merge 21 commits into
developfrom
dkachuma/fluid-kernel-specs

Conversation

@dkachuma
Copy link
Copy Markdown
Contributor

@dkachuma dkachuma commented May 15, 2026

In CompositionalMultiphaseBase.cpp and CompositionalMultiphaseWell.cpp calls to FluidUpdateKernel involved a construct of the form:

constitutiveUpdatePassThru( fluid, [&] ( auto & castedFluid )
{
  using FluidType = TYPEOFREF( castedFluid );
  // ...
} );

In DirichletFluxComputeKernelFactory we have

constitutive::constitutiveComponentUpdatePassThru( fluidBase, numComps, [&]( auto & fluid, auto NC )
{
  using FluidType = TYPEOFREF( fluid );
  integer constexpr NUM_COMP = NC();
  // ...	
}

The first is a loop over the different types of multiphase fluid models while the second also includes, in addition to this, a loop over the number of components. When these loops are expanded in place, the size of the compilation units for CompositionalMultiphaseBase.cpp, CompositionalMultiphaseWell.cpp and CompositionalMultiphaseFVM.cpp become too large. This causes out-of-memory issues for the CUDA compilation on most machines including the CI machines. The "solution" for this was to disable some fluid models for GPU (see MultiFluidSelector.hpp:53-63)). This means that some models were not available for GPU even though they were developed for GPU execution.

This PR tries to resolve the out-of-memory issues by breaking up the massive translation units and offloading template instantiations into auto generated, independent source files.

  • Split the heavy kernel implementations out of .hpp files and into dedicated _impl.hpp files DirichletFluxComputeKernel_impl.hpp, FluidUpdateKernel_impl.hpp, HydrostaticPressureKernel_impl.hpp.
  • Use the kernelSpecs.json generator to explicitly instantiate the combinations that are required for these kernels.
  • Introduced dedicated .cpp files (e.g., DirichletFluxComputeKernelFactory.cpp) to handle the heavy createAndLaunch dispatch logic, keeping it out of the core solver classes.
  • Removed #if !defined(GEOS_DEVICE_COMPILE) guards in MultiFluidSelector.hpp and CO2BrineFluid.cpp, restoring full GPU support for all previously disabled multiphase fluid models like CO2BrineEzrokhiThermalFluid and CompositionalTwoPhaseLohrenzBrayClarkViscosity.

@dkachuma dkachuma self-assigned this May 15, 2026
@dkachuma dkachuma added type: cleanup / refactor Non-functional change (NFC) ci: run CUDA builds Allows to triggers (costly) CUDA jobs ci: run integrated tests Allows to run the integrated tests in GEOS CI flag: no rebaseline Does not require rebaseline ci: run code coverage enables running of the code coverage CI jobs labels May 15, 2026
@dkachuma dkachuma marked this pull request as ready for review May 15, 2026 21:09
@dkachuma dkachuma changed the title Fluid kernel specs Use kernelSpecs to explicitly instantiate fluid models and resolve CUDA builds May 15, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR restructures several compositional fluid-flow kernels to reduce CUDA compilation memory pressure by moving heavy template implementations into _impl.hpp files and shifting dispatch/instantiation into generated and dedicated translation units, restoring GPU availability for previously disabled multifluid models.

Changes:

  • Split large templated kernel implementations into _impl.hpp headers and add explicit instantiation generation via kernelSpecs.json.
  • Introduce dedicated factory .cpp translation units for Dirichlet flux kernel dispatch/launch to keep solver TUs smaller.
  • Re-enable previously GPU-disabled multifluid models by removing GEOS_DEVICE_COMPILE guards and related runtime disables.

Reviewed changes

Copilot reviewed 26 out of 26 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/coreComponents/physicsSolvers/fluidFlow/wells/CompositionalMultiphaseWell.cpp Switch FluidUpdateKernel usage to explicitly-typed kernel specialization.
src/coreComponents/physicsSolvers/fluidFlow/kernelSpecs.json Define explicit instantiation matrices for Dirichlet flux, fluid update, hydrostatic kernels.
src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/zFormulation/DirichletFluxComputeZFormulationKernelFactory.cpp New TU for Z-formulation Dirichlet flux dispatch/launch.
src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/zFormulation/DirichletFluxComputeZFormulationKernel.hpp Convert to declaration-only + new template parameter ordering (wrapper, policy, sizes).
src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/zFormulation/DirichletFluxComputeZFormulationKernel_impl.hpp Add implementation header for Z-formulation Dirichlet flux kernel.
src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/ThermalDirichletFluxComputeKernel.hpp Convert to declaration-only and align template params with isothermal base.
src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/ThermalDirichletFluxComputeKernel_impl.hpp Add implementation header for thermal Dirichlet flux kernel.
src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/HydrostaticPressureKernel.hpp Make HydrostaticPressureKernel wrapper-templated and move implementations out-of-line.
src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/HydrostaticPressureKernel_impl.hpp Add implementation header for hydrostatic pressure kernel (centralized constants).
src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/FluidUpdateKernel.hpp Refactor FluidUpdateKernel into a class template declared in header.
src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/FluidUpdateKernel_impl.hpp Add implementation header for FluidUpdateKernel launches.
src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/DirichletFluxComputeKernelFactory.cpp New TU for isothermal/thermal Dirichlet flux factory dispatch/launch.
src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/DirichletFluxComputeKernel.hpp Convert to declaration-only and align template params (wrapper, policy, sizes).
src/coreComponents/physicsSolvers/fluidFlow/kernels/compositional/DirichletFluxComputeKernel_impl.hpp Add implementation header for isothermal Dirichlet flux kernel.
src/coreComponents/physicsSolvers/fluidFlow/CompositionalMultiphaseFVMKernel_thermal.cpp.template Remove old generator template (superseded by new kernelSpecs-driven templates).
src/coreComponents/physicsSolvers/fluidFlow/CompositionalMultiphaseFVMKernel_isothermal.cpp.template Remove old generator template (superseded by new kernelSpecs-driven templates).
src/coreComponents/physicsSolvers/fluidFlow/CompositionalMultiphaseFVM.cpp Update calls to new factory types / createAndLaunch style.
src/coreComponents/physicsSolvers/fluidFlow/CompositionalMultiphaseFVM_ThermalDirichletFluxComputeKernels.cpp.template New generated explicit-instantiation template for thermal Dirichlet flux kernels.
src/coreComponents/physicsSolvers/fluidFlow/CompositionalMultiphaseFVM_IsothermalDirichletFluxComputeKernels.cpp.template New generated explicit-instantiation template for isothermal Dirichlet/Z-formulation kernels.
src/coreComponents/physicsSolvers/fluidFlow/CompositionalMultiphaseBase.cpp Switch FluidUpdateKernel and HydrostaticPressureKernel usage to explicit specializations.
src/coreComponents/physicsSolvers/fluidFlow/CompositionalMultiphaseBase_HydrostaticPressureKernel.cpp.template New generated explicit-instantiation template for hydrostatic pressure kernel.
src/coreComponents/physicsSolvers/fluidFlow/CompositionalMultiphaseBase_FluidUpdateKernel.cpp.template New generated explicit-instantiation template for fluid update kernel.
src/coreComponents/physicsSolvers/fluidFlow/CMakeLists.txt Wire new sources/templates and remove legacy Dirichlet kernel generator calls.
src/coreComponents/constitutive/fluid/multifluid/MultiFluidSelector.hpp Remove GPU compile guards to restore full model selection on GPU.
src/coreComponents/constitutive/fluid/multifluid/CO2Brine/CO2BrineFluid.hpp Remove GPU-disabling pre-subgroup initialization override declaration.
src/coreComponents/constitutive/fluid/multifluid/CO2Brine/CO2BrineFluid.cpp Remove GPU runtime disable/throw for Ezrokhi thermal CO2-brine model.

Comment on lines +16 to +21
/**
* @file DirichletFluxComputeKernel_impl.hpp
*/

#include "DirichletFluxComputeKernel.hpp"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci: run code coverage enables running of the code coverage CI jobs ci: run CUDA builds Allows to triggers (costly) CUDA jobs ci: run integrated tests Allows to run the integrated tests in GEOS CI flag: no rebaseline Does not require rebaseline type: cleanup / refactor Non-functional change (NFC)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants