- Introduction
- Prerequisites
- Installation
- Dependency Version Compatibility
- Codebase Overview
- Source Directory Structure
- Key Components
- Configuration System
- Advanced Solver Configuration
- Building and Testing
- Development Workflow
- UMAT Development Resources
- Contributing Guidelines
ExaConstit is a high-performance, velocity-based, updated Lagrangian finite element code for nonlinear solid mechanics problems with a focus on micromechanics modeling. Built on the MFEM library, it specializes in crystal plasticity simulations and bulk constitutive property determination for polycrystalline materials.
Key Features:
- Velocity-based updated Lagrangian formulation
- Crystal plasticity and micromechanics modeling
- GPU acceleration with CUDA/HIP support
- MPI parallelization for HPC systems
- Integration with ExaCMech material library
- UMAT interface support
- Advanced post-processing capabilities
- C++17: Modern C++ standards and best practices
- Finite Element Method (FEM): Theory and implementation
- Solid Mechanics: Nonlinear mechanics, crystal plasticity
- Numerical Methods: Newton-Raphson, Krylov iterative solvers
- Parallel Computing: MPI, OpenMP, GPU programming concepts
- C++17 compatible compiler (GCC 7+, Clang 5+, Intel 19+)
- MPI implementation (OpenMPI, MPICH, Intel MPI)
- CMake 3.24 or higher
- Git for version control
For detailed installation instructions, refer to the build scripts in scripts/install/:
- Linux/Unix:
scripts/install/unix_install_example.sh - GPU (CUDA):
scripts/install/unix_gpu_cuda_install_example.sh - GPU (HIP/AMD):
scripts/install/unix_gpu_hip_install_example.sh
Core Dependencies:
- MFEM (v4.8+): Finite element library with parallel/GPU support
- ExaCMech (v0.4.3+): Crystal plasticity constitutive model library
- RAJA (≥2024.07.x): Performance portability framework
- UMPIRE (≥2024.07.x): (GPU-only) Performance portability framework
- CHAI (≥2024.07.x): (GPU-only) Performance portability framework
- BLT: LLNL build system
- SNLS: Nonlinear solver library
Optional Dependencies:
- ADIOS2: (MFEM-based) High-performance I/O for visualization
- Caliper: Performance profiling
git submodule init && git submodule update
# Create build directory
mkdir build && cd build
# Configure
cmake .. \
-DENABLE_MPI=ON \
-DENABLE_FORTRAN=OFF \
-DMFEM_DIR=${MFEM_INSTALL_DIR} \
-DECMECH_DIR=${EXACMECH_INSTALL_DIR} \
-DRAJA_DIR=${RAJA_INSTALL_DIR} \
-DSNLS_DIR=${SNLS_INSTALL_DIR}
# Build
make -j 4ExaConstit requires a specific MFEM development branch with ExaConstit-specific features:
- Repository: https://github.com/rcarson3/mfem.git
- Branch:
exaconstit-dev - Version Dependencies:
- v0.9.0: Compatible with MFEM hashes
a6bb7b7c2717e991b52ad72460f212f7aec1173e - v0.8.0: Compatible with MFEM hashes
31b42daa3cdddeff04ce3f59befa769b262facd7or29a8e15382682babe0f5c993211caa3008e1ec96 - v0.7.0: Compatible with MFEM hash
78a95570971c5278d6838461da6b66950baea641 - v0.6.0: Compatible with MFEM hash
1b31e07cbdc564442a18cfca2c8d5a4b037613f0 - v0.5.0: Required MFEM hash
5ebca1fc463484117c0070a530855f8cbc4d619e
- v0.9.0: Compatible with MFEM hashes
# Required dependencies for MFEM
cmake .. \
-DMFEM_USE_MPI=ON \
-DMFEM_USE_METIS_5=ON \
-DMFEM_USE_HYPRE=ON \ # v2.26.0-v2.30.0
-DMFEM_USE_RAJA=ON \ # v2022.x+
-DMFEM_USE_ADIOS2=ON \ # Optional: high-performance I/O
-DMFEM_USE_ZLIB=ON # Optional: compressed mesh supportNote: Future releases will integrate these changes into MFEM master branch, eliminating the need for the development fork.
- Repository: https://github.com/LLNL/ExaCMech.git
- Branch:
develop(required) - Version: v0.4.3+ required
- SNLS Dependency: https://github.com/LLNL/SNLS.git
For GPU builds of ExaCMech >= v0.4.3:
- RAJA: Performance portability framework
- Umpire: Memory management
- CHAI: Array abstraction
- Tag:
v2024.07.0for all RAJA Portability Suite repositories - Important: All RAJA suite components (RAJA, Umpire, CHAI) must use matching versions
- Minimum RAJA: v2024.07.0
- Note: Version mismatch between RAJA components can cause build failures or runtime errors. For GPU builds, we recommend v2025.09.x as the base version for the RAJA Portability Suite. Although, we do require a slightly newer version of Umpire for a small bug fix related to an API: git hash 091305d8ef40aa8f2d75d684fbabeabff2e0c1fc . This fix is necessary for us to address a segfault noted during the program shutdown due to conflicts with their internal logging features and our own.
- HYPRE: v2.26.0 - v2.30.0 (algebraic multigrid / various preconditioners)
- METIS: Version 5 (mesh partitioning)
- ADIOS2: Optional (high-performance parallel I/O)
- ZLIB: Optional (compressed mesh and data support)
ExaConstit follows a modular architecture designed for extensibility and performance:
ExaConstit/
├── src/ # Main source code
├── test/ # Test cases and examples
├── scripts/ # Build scripts and utilities
├── workflows/ # Optimization and UQ workflows
└── cmake/ # Build system configuration
- Modularity: Clear separation of concerns between FEM operators, material models, and solvers
- Performance: GPU acceleration and memory-efficient algorithms
- Extensibility: Plugin architecture for material models and boundary conditions
- Standards: Modern C++17 practices and comprehensive documentation
The src/ directory contains the core ExaConstit implementation organized into modular components:
mechanics_driver.cpp: Main application entry point and simulation orchestrationsystem_driver.hpp/cpp: Core driver class managing the Newton-Raphson solution processuserumat.h: Interface definitions for UMAT material model integration
Purpose: Boundary condition management and enforcement
BCData.hpp/cpp: Data structures for boundary condition storageBCManager.hpp/cpp: Boundary condition application and management
Key Features:
- Dirichlet velocity and velocity gradient boundary conditions
- Time-dependent BC scaling and ramping
- Component-wise BC application for selective spatial directions
- Support for multiple BC regions with different behaviors
Purpose: Finite element operators and integration routines
mechanics_operator.hpp/cpp: Nonlinear mechanics operator implementationmechanics_operator_ext.hpp/cpp: Extended operator functionalitymechanics_integrators.hpp/cpp: Element-level integration kernels
Key Features:
- Element assembly (EA) and partial assembly (PA) modes
- B-bar integration for near-incompressible materials
- GPU-optimized kernel implementations
- Matrix-free operator evaluation
Purpose: Material constitutive model interface and implementations
mechanics_model.hpp/cpp: Abstract base classExaModelinterfacemechanics_ecmech.hpp/cpp: ExaCMech crystal plasticity integrationmechanics_umat.hpp/cpp: UMAT interface implementationmechanics_multi_model.hpp/cpp: Multi-region material management
Supported Models:
- ExaCMech crystal plasticity (FCC, BCC, HCP)
- User-defined UMAT subroutines
- Multi-material region support
Purpose: Configuration file parsing and option management
option_parser_v2.hpp/cpp: Modern TOML-based parseroption_material.cpp: Material configuration parsingoption_mesh.cpp: Mesh and geometry optionsoption_boundary_conditions.cpp: BC configuration parsingoption_solvers.cpp: Linear and nonlinear solver settingsoption_time.cpp: Time-stepping parametersoption_post_processing.cpp: Post-processing configurationoption_enum.cpp: Enumeration type conversionsoption_util.hpp: Utility functions for option parsing
Features:
- Backward compatibility with legacy formats
- Hierarchical configuration structure
- Comprehensive validation and error reporting
Purpose: Output management and field calculations
postprocessing_driver.hpp/cpp: Main post-processing orchestrationpostprocessing_file_manager.hpp: File I/O and directory managementprojection_class.hpp/cpp: Field projection operationsmechanics_lightup.hpp/cpp: Lattice strain calculations
Capabilities:
- Volume-averaged stress/strain/deformation gradient
- Lattice strain calculations for diffraction experiments
- Visualization output (VisIt, ParaView, ADIOS2)
- Structured output file organization
Purpose: Simulation state management and field storage
simulation_state.hpp/cpp: Central state container class
Manages:
- Finite element spaces and mesh data
- Solution vectors (displacement, velocity)
- Material properties and state variables
- Time-stepping information
Purpose: Linear and nonlinear solver implementations
mechanics_solver.hpp/cpp: Newton-Raphson solver and variants
Features:
- Standard Newton-Raphson
- Newton with line search
- Adaptive step size control
- Device-aware implementations
Purpose: Example UMAT implementations for testing
umat.f: Example Fortran UMAT implementationumat.cxx: Example C++ UMAT implementationuserumat.cxx: UMAT loader exampleuserumat.h: UMAT interface definitions
Purpose: Helper functions and utility classes
mechanics_log.hpp: Logging and performance monitoringunified_logger.hpp/cpp: Unified logging system for all componentsmechanics_kernels.hpp/cpp: Computational kernels for mechanics operationsassembly_ops.hpp: Assembly operation utilitiesrotations.hpp: Rotation and orientation utilitiesstrain_measures.hpp: Strain computation utilitiesdynamic_umat_loader.hpp/cpp: Runtime UMAT library loading
Provides:
- Performance profiling integration
- Mathematical operations for mechanics
- Debugging and diagnostic tools
- Dynamic loading of user material models
Purpose: MFEM extensions and experimental features
partial_qspace.hpp/cpp: Partial quadrature space implementationspartial_qfunc.hpp/cpp: Partial quadrature function utilities
Features:
- Experimental finite element enhancements
- Performance optimizations for specific use cases
- Research and development components
- Modular Design: Clear separation between components
- Header/Implementation Pairs: Consistent
.hpp/.cpporganization - Device Portability: GPU-aware implementations throughout
- Template Usage: Modern C++17 templates for performance
- Namespace Structure:
exaconstit::for internal components
The SystemDriver class orchestrates the entire simulation workflow, managing the Newton-Raphson solution process and coordinating between components.
Responsibilities:
- Newton-Raphson nonlinear solution management
- Linear solver and preconditioner setup
- Boundary condition enforcement and updates
- Material model coordination
- Solution advancement
Key Methods:
void SystemDriver::Solve(); // Main Newton-Raphson solution
void SystemDriver::SolveInit(); // Initial corrector step for BC changes
void SystemDriver::UpdateEssBdr(); // Update essential boundary conditions
void SystemDriver::UpdateVelocity(); // Apply velocity boundary conditions
void SystemDriver::UpdateModel(); // Update material models after convergenceThe finite element operator extending MFEM's NonlinearForm that provides:
- Residual evaluation for Newton-Raphson iterations
- Jacobian computation and assembly
- Essential DOF management
- Support for different assembly strategies (PA/EA/FULL)
Central container managing all simulation data and providing unified access to:
- Mesh and finite element spaces
- Solution fields (velocity, displacement)
- Material properties and state variables
- Quadrature functions for field data
- Time-stepping information
Base class ExaModel defines the constitutive model interface:
// Main execution method for material model computations
virtual void ModelSetup(nqpts, nelems, space_dim, nnodes,
jacobian, loc_grad, vel) = 0;
// Update state variables after converged solution
virtual void UpdateModelVars() = 0;
// Get material properties for this region
const std::vector<double>& GetMaterialProperties() const;Manages multiple material regions within a single simulation:
- Coordinates material model execution across regions
- Routes region-specific data from SimulationState
- Handles heterogeneous material configurations
Manages all output and post-processing operations:
- Volume averaging calculations (stress, strain, etc.)
- Field projections for visualization
- File output management
- Support for VisIt, ParaView, and ADIOS2
Singleton pattern manager for boundary conditions:
- Tracks time-dependent boundary condition changes
- Manages multiple BC types (velocity, velocity gradient)
- Provides BC data to SystemDriver and operators
ExaConstit uses TOML-based configuration files for all simulation parameters:
basename = "simulation_name"
version = "0.9.0"
[Mesh]
filename = "mesh.mesh"
refine_serial = 0
[Time.Fixed]
dt = 1.0e-3
t_final = 1.0
[Solvers]
assembly = "ea"
[Solvers.Krylov]
rel_tol = 1.0e-12
abs_tol = 1.0e-30
linear_solver = "CG"
[Materials]
# Material definitions...
[BCs]
# Boundary condition specifications...- External material files:
materials = ["material1.toml", "material2.toml"] - Grain data files:
grain_file = "grain.txt",orientation_file = "orientations.txt"
ExaConstit supports multiple finite element assembly strategies optimized for different hardware:
[Solvers]
assembly = "PA"- Memory efficient: No global matrix formation
- GPU optimized: Ideal for GPU acceleration only for very high p-refinement
- Matrix-free: Jacobian actions computed on-the-fly
- Preconditioning: Currently limited to Jacobi preconditioning
[Solvers]
assembly = "EA"- Element-level: Only element matrices formed
- Memory balanced: minimal memory requirements for quadratic or fewer elements
- GPU compatible: Supports GPU execution
- Flexibility: Suitable for complex material models
- Preconditioning: Currently limited to Jacobi preconditioning
[Solvers]
assembly = "FULL"- Traditional: Complete global matrix assembly
- Preconditioning: Full preconditioner options available
- Memory intensive: Requires moderate memory for large problems due to sparse matrix formats
- CPU optimized: Best initial set-up for investigating new material models
[Solvers]
integ_model = "DEFAULT"- Full integration: Complete quadrature point evaluation
- Standard: Traditional finite element approach
- Most materials: Suitable for general material models
[Solvers]
integ_model = "BBAR"- Mixed formulation: Deviatoric fully integrated and elemental averaged volume contribution
- Near-incompressible: Prevents volumetric locking
- Advanced: Based on Hughes-Brezzi formulation (Equation 23)
- Limitation: Not compatible with partial assembly
[Solvers.Krylov]
linear_solver = "GMRES" # or "cg", "minres"
rel_tol = 1.0e-6
abs_tol = 1.0e-10
max_iter = 1000GMRES: General minimal residual
- Nonsymmetric systems: Handles general Jacobian matrices
- Memory: Requires restart for memory management
- Robust: Suitable for challenging material models
Conjugate Gradient (CG):
- Symmetric positive definite: Requires symmetric Jacobian
- Memory efficient: Minimal memory requirements
- Fast convergence: Optimal for appropriate problems
MINRES: Minimal residual for symmetric indefinite
- Symmetric indefinite: Handles saddle point problems
- Specialized: Useful for constrained problems
[Solvers.Krylov]
preconditioner = "JACOBI" # Assembly-dependent optionsAssembly-Dependent Availability:
For PA/EA Assembly (limited to):
- JACOBI: Diagonal scaling, GPU-compatible (automatic selection)
For FULL Assembly (all options available):
- JACOBI: Diagonal scaling
- AMG: Algebraic multigrid
- ILU: Incomplete LU factorization
- L1GS: ℓ¹-scaled Gauss-Seidel
- CHEBYSHEV: Polynomial smoother
Preconditioner Details:
JACOBI (Diagonal Scaling):
- Characteristics: Simple and fast, works everywhere but slow convergence
- Assembly: Works with PA, EA, and FULL
- GPU: Fully GPU-compatible
- Use case: Default for PA/EA assembly, baseline option
AMG (Algebraic Multigrid):
- Characteristics: Fewer iterations but expensive setup, can fail on some problems
- Implementation: HYPRE BoomerAMG
- Configuration: Pre-tuned for 3D elasticity
- Use case: Large-scale problems with single materials
ILU (Incomplete LU Factorization):
- Characteristics: Good middle-ground option
- Implementation: HYPRE Euclid
- Use case: Particularly useful for multi-material systems
- Try this: If JACOBI convergence is too slow
L1GS (ℓ¹-Scaled Gauss-Seidel):
- Characteristics: Advanced smoother
- Implementation: HYPRE smoother
- Use case: Multi-material systems with contrasting properties
- Try this: When materials have very different stiffness values
CHEBYSHEV (Chebyshev Polynomial):
- Characteristics: Polynomial smoother
- Implementation: HYPRE smoother
- Use case: Problems with multiple material scales
- Try this: For heterogeneous material distributions
Practical Selection Guidelines:
For Single Material Problems:
- Start with JACOBI (simple, predictable)
- Try AMG if convergence is slow
- Use ILU as a reliable alternative
For Multi-Material Systems:
- Start with ILU (good middle-ground)
- Try L1GS for contrasting material properties
- Use CHEBYSHEV for multiple material scales
- AMG may struggle with material interfaces
Performance Tips:
- PA/EA assembly automatically uses JACOBI
- If JACOBI convergence is too slow with FULL assembly, try ILU → L1GS → CHEBYSHEV
- AMG has high setup cost but fewer iterations
- Multi-material systems often benefit from experimenting with different preconditioners
[Solvers.NR]
nonlinear_solver = "NR" # or "NRLS"
rel_tol = 1.0e-5
abs_tol = 1.0e-10
max_iter = 25Standard Newton-Raphson:
- Full steps: Always takes complete Newton step
- Fast convergence: Quadratic convergence near solution
- Robustness: May fail for poor initial guesses
Newton with Line Search:
- Globalization: Backtracking line search for robustness
- Convergence: Improved convergence from poor starting points
- Cost: Additional function evaluations per iteration
# Enable GPU support
-DENABLE_CUDA=ON
-DENABLE_HIP=ON
# Enable specific features
-DENABLE_CALIPER=ON# Run example simulations
cd test/data
mpirun -np 4 ../../build/mechanics -opt example.tomlThe test/data/ directory contains various example cases:
- Crystal plasticity simulations
- Multi-material problems
- Complex boundary condition examples
- GPU acceleration tests
- Header-only utilities: Place in
utilities/directory - New material models: Extend
ExaModelbase class inmodels/ - Post-processing features: Add to
postprocessing/directory - Configuration options: Update corresponding
option_*.cppfiles
- Create header/source in
models/mechanics_newmodel.hpp/cpp - Inherit from
ExaModelbase class - Implement required virtual methods
- Add configuration parsing support
- Update
CMakeLists.txt
- Extend
BCManagerclass - Add parsing support in
option_boundary_conditions.cpp - Update documentation and examples
- GPU kernels: Use RAJA for performance portability
- Memory management: Follow MFEM memory patterns
- MPI communication: Minimize collective operations
- Assembly strategy: Choose PA vs EA based on problem size
- Caliper integration: Built-in performance profiling
- MFEM debugging: Use MFEM's debugging capabilities
- GPU debugging: CUDA/HIP debugging tools
- MPI debugging: TotalView, DDT support
While UMAT interfaces are traditionally described using Fortran signatures, ExaConstit supports implementation in Fortran, C++, or C:
SUBROUTINE UMAT(STRESS,STATEV,DDSDDE,SSE,SPD,SCD,
1 RPL,DDSDDT,DRPLDE,DRPLDT,
2 STRAN,DSTRAN,TIME,DTIME,TEMP,DTEMP,PREDEF,DPRED,CMNAME,
3 NDI,NSHR,NTENS,NSTATV,PROPS,NPROPS,COORDS,DROT,PNEWDT,
4 CELENT,DFGRD0,DFGRD1,NOEL,NPT,LAYER,KSPT,KSTEP,KINC)extern "C" void umat_(double* stress, double* statev, double* ddsdde,
double* sse, double* spd, double* scd,
// ... additional parameters
int* ndi, int* nshr, int* ntens, int* nstatv,
double* props, int* nprops,
// ... remaining parameters
);- ExaConstit handles: State variable allocation and persistence
- UMAT responsible: Local variable management within subroutine
- No dynamic allocation: Avoid malloc/new within UMAT calls
- No global variables: UMATs must be thread-safe
- Local computations: All calculations using passed parameters
- State persistence: Only through provided state variable arrays
- Convergence issues: Set appropriate flags for Newton-Raphson
- Material failure: Handle through state variables or stress reduction
- Numerical stability: Check for divide-by-zero and overflow conditions
- CPU execution only: No current GPU acceleration for UMATs but might be possible in future updates
- Vectorization: Ensure compiler optimization is possible
- Minimal function calls: Reduce computational overhead within UMAT
src/umat_tests/: Example UMAT implementations and conversion guides- Template UMATs: Starting points for custom development
- NJIT UMAT Collection: https://web.njit.edu/~sac3/Software.html
- Academic examples: Various constitutive models available
- License considerations: Verify licensing before use
# Compile UMAT to shared library (Fortran)
gfortran -shared -fPIC -o my_umat.so my_umat.f90
# Compile UMAT (C++)
g++ -shared -fPIC -o my_umat.so my_umat.cpp
# Compile UMAT (C)
gcc -shared -fPIC -o my_umat.so my_umat.c[Materials.regions.model.UMAT]
library_path = "/path/to/my_umat.so"
num_props = 8
num_state_vars = 12
props = [
210000.0, # Young's modulus
0.3, # Poisson's ratio
# ... additional parameters
]- C++17 compliance: Use modern C++ features
- Documentation: Doxygen-style comments for all public interfaces
- Testing: Include test cases for new features
- Performance: Maintain GPU and MPI scalability
- Name Formating:
- Function names should be in
PascalCasefor any file but those related to IO (src/options/* and src/utilities/unified_loggers.*) which aresnake_case. - Class / enum names should be in
PascalCase - Enum values should be
UPPER_CASE - Class member variables going forward should be
snake_caseand preferably have am_prefix. However, them_prefix is not required if it makes things harder to understand. We're still converting variables over from previous in-consistent naming conventions so if you spot something that needs fixing please do so. - Local / function variables going forward should be
snake_case. Like above we are slowly in the process of converting old code over to this new format so feel free to help out if you can. - If doing formatting changes split those into their own commits so it's easier to track changes. Additionally try to change the world all at once and do things in piece meal as it makes it easier to track down where a bug might have been introduced during renaming of things.
- Function names should be in
- Code Formating: We have a
.clang-formatthat we make use to enfore a unified coding experience across the code base. An example of how to run the formatter is:find src -type f \( -name "*.cpp" -o -name "*.hpp" -o -name "*.h" \) ! -path "*/TOML_Reader/*" -exec $CLANG_FORMAT -i {} +. Note, if you see any changes in thesrc/TOML_Readerdirectory to revert those changes as that is a TPL that we directly include in the repo and not something we want to update unless directly bringing in the changes from its upstream repo.
- Fork the repository (if non-LLNL employee)
- Create feature branch from
exaconstit-dev - Implement changes with tests
- Ensure all existing tests pass
- Submit pull request with detailed description
- BSD-3-Clause license: All contributions must use this license
- Third-party code: Ensure compatible licensing for external dependencies
- Primary Developer: Robert A. Carson (carson16@llnl.gov)
- GitHub Issues: Report bugs and feature requests
- Documentation: Refer to MFEM and ExaCMech documentation for underlying libraries
- ExaCMech: Crystal plasticity library (https://github.com/LLNL/ExaCMech)
- MFEM: Finite element library (https://mfem.org)
- ExaCA: Cellular automata for microstructure generation
- Optimization workflows: Multi-objective genetic algorithm parameter optimization
- UQ workflows: Uncertainty quantification for additive manufacturing
- Post-processing tools: Python scripts for data analysis
If using ExaConstit in your research, please cite:
@misc{ exaconstit,
title = {{ExaConstit}},
author = {Carson, Robert A. and Wopschall, Steven R. and Bramwell, Jamie A.},
abstractNote = {The principal purpose of this code is to determine bulk constitutive properties and response of polycrystalline materials. This is a nonlinear quasi-static, implicit solid mechanics code built on the MFEM library based on an updated Lagrangian formulation (velocity based). Within this context, there is flexibility in the type of constitutive model employed, with the code allowing for various UMATs to be interfaced within the code framework or for the use of the ExaCMech library. Using crystal-mechanics-based constitutive models, the code can be used, for example, to compute homogenized response behavior over a polycrystal. },
howpublished = {[Computer Software] \url{https://doi.org/10.11578/dc.20191024.2}},
url = {https://github.com/LLNL/ExaConstit},
doi = {10.11578/dc.20191024.2},
year = {2019},
month = {Aug},
annote = {
https://www.osti.gov//servlets/purl/1571640
https://www.osti.gov/biblio/1571640-exaconstit
}
}This guide provides a foundation for new developers to understand and contribute to ExaConstit. For specific implementation details, refer to the extensive inline documentation throughout the codebase and the example configurations in test/data/.