CFD simulations can be a powerful tool when designing an internal-combustion engine; however, the results of CFD simulations often change as the grid is refined. This is known as grid dependence. When grid dependence is observed, the traditional approach is to optimize the results by tuning grid sizes and model constants based on experimental data. This is a viable approach only in the presence of high-fidelity experimental data.

CFD simulations become an even more powerful tool when reliable results can be obtained outside the range of experimental validation in what are known as predictive studies. To have confidence in predictive studies, they must achieve grid convergence. This means that simulation results should converge as the grid resolution is refined.

Because grid-convergent ICE simulations are computationally expensive, high-performance computing (HPC) tools are necessary. HPC tools have been used to simulate complex processes such as spray or combustion, but the full potential of HPC tools for engine simulations has not yet been realized. Motivated by the untapped potential of HPC tools for engine simulations, personnel from Convergent Science Inc.; Caterpillar Inc.; and Argonne National Laboratory combined forces to carry out a series of engine simulations including a simulation of nearly 34 million cells that is believed to be the largest-ever diesel engine simulation.

The Converge CFD software package provides the computational framework for these engine simulations. Converge includes several models to simulate turbulence, spray injection, liquid drop dynamics, and combustion of hydrocarbon fuels.

For combustion modeling, these simulations use the SAGE detailed chemistry solver or the Characteristic Time Combustion (CTC) model. The SAGE detailed chemistry model solves the detailed chemical kinetics, including knock and emissions levels in the combustion process. To lessen the computational burden of this detailed model, the SAGE simulations in this study incorporate the multi-zone solver. The multi-zone solver groups cells with similar temperatures and equivalence ratios into zones and then solves the chemistry calculations for each zone rather than each cell.

The CTC model is another combustion model offered in Converge. This model incorporates a single global fuel oxidation reaction and calculates a characteristic time to achieve equilibrium. The characteristic time depends on the relevant time scales based on chemical kinetic and turbulent mixing. The CTC model is used for cells in the combustion phase while the Shell auto-ignition model is used for cells in the ignition phase. This model is computationally inexpensive in comparison to the SAGE model, even when the SAGE model incorporates multi-zone modeling. These two models—SAGE and CTC—were selected for comparison purposes.

The large size of these simulations presented challenges for the original load balancing algorithm in Converge, which was found to be effective for no more than 128 processors. The METIS load balancing algorithm is widely known for its efficiency in partitioning complex geometries and its ability to minimize the connectivity and to enforce contiguousness between partitions. METIS version 5.0.2 was implemented in Converge and used for all of the simulations in this study.

Argonne National Laboratory provided the HPC resources for these simulations. Specifically, these simulations used the Fusion cluster and the Surveyor machine at Argonne. Fusion is an Intel cluster comprised of 320 compute nodes for a total of 2560 compute cores. Surveyor is one rack of an IBM Blue Gene/P system with 1024 nodes and one 850 MHz quad-core processor for a total of 4096 cores.

For both the CTC and SAGE models, the 1- and 2-mm (0.04- and 0.08-in) cases differ significantly from the other grid resolutions. For the CTC model, it appears that the results are quite sensitive to the grid resolution. Grid convergence is not achieved for the CTC model simulations. The results of the SAGE model simulations, however, appear grid convergent because the 0.5, 0.25, and 0.125 mm (0.0197, 0.0098, and 0.049 in) results are quite close to one another in terms of peak values and phasing.

The results for the 1- and 2-mm cell sizes are quite different from the simulations conducted at other grid resolutions. The SAGE results for the 0.5, 0.25, and 0.125 mm cases are close to one another and provide further evidence that grid convergence has been attained.

Once again, the three CTC cases vary significantly from one another when measuring temperature distributions for both the CTC and SAGE model simulations. The temperature distributions for the SAGE simulations, especially those at 0.25 and 0.125 mm, are quite similar, which provides further evidence that grid convergence has been attained.

The computational cost for large diesel engine simulations is considerable, especially when the SAGE detailed chemistry solver is used. In general, the computational cost (wall-clock time) for the SAGE model simulations was approximately 55-75% greater than the computational cost for corresponding CTC simulations. The SAGE model simulations require the solution of additional transport equations associated with each species in the simulation. Although the SAGE model simulations used considerably more computational resources, these were the only simulations that achieved grid convergence. Based on effort to balance accuracy and computational demands, the 0.25-mm grid size may be the optimal grid resolution for the SAGE model simulations. Note that 0.25 mm is on the order of the nozzle exit diameter.

The peak cell count for the SAGE simulation with 0.125 mm resolution was 33.69 million cells. To the best of the researchers’ knowledge this is the largest-ever CFD simulation for a compression-ignition engine with realistic geometries and operating conditions.