The U.S. Department of Energy is backing a sweeping software to bring wind turbine and wind farm modeling into the exascale era with the open source “ExaWind” modeling and simulation environment. To be fully realized it will need to keep pace with the GPU trend in top supercomputers but it is nonetheless an example of the kind of large-scale simulation work that deliver on the promise of exascale.
There are two main codes that comprise the core of the work, which is designed to produce high-fidelity simulations of both turbines and clusters of turbines. The goal, according to the project leads is to enable “disruptive changes to turbine and plant design and operation. They add that research will focus on better understanding the complex flow dynamics in wind farms, in other words, the way wakes have an impact on surrounding turbines and how flow changes on complex terrain.
It is no surprise that massive compute power is required. These are complex multi-physics applications that will require high scalability across nodes designed for high performance parallel processing. Although the program is designed to run on the top petascale and forthcoming exascale supercomputers, it is not yet GPU ready (most of the current and future top systems have several GPUs per node). There are some CPU-only machines that offer high scalability, of course, but much of what’s on the horizon for exascale is GPU-based.
The two main codes, Nalu-Wind and OpenFAST have to integrate the many physics-based layers of simulation and modeling of complex machines in motion individually and in groups. This means weaving together everything from hybrid turbulence models, CFD elements, nonlinear structural dynamics, and fluid flow, among simulations while emphasizing high fidelity. The team says that the current environment does support lower-fidelity modeling as well.
Nalu-Wind is a C++ based CFD code baed on the open source Nalu code and optimized for wind. OpenFAST is a Fortran 2003 code that evolved out of the existing open source FAST code that can handle whole-turbine simulations. As one might imagine, this is a dense simulation given the required modeling for everything from the turbine control system, the tower itself (how it bends, for instance), and an additional element called BeamDyn for understanding blade dynamics. More about the codes can be found on GitHub.
While a top supercomputer like Summit, for instance, might have massive scalability for ExaWind, to get competitive time allocations on the machine the proposal requires demonstration of effective use of GPUs. “However,” the ExaWind team notes, “enabling CFD codes to run effectively on GPUs is no small task. Nalu-Wind and Trillinos developer teams are actively preparing for next-generation architectures, like GPUs with Kokkos, a parallel performance abstraction layer.” They add that the Nalu-Wind team is also working closely with the hypre team in preparing it for effective use of GPUs.
Currently, the team says, ‘multiple levels of fidelity have been provided, including turbine-resolved hybridRANS/LES capabilities with fluid-structure interaction and full turbine mobility.” In regard to planned work, the ExaWind team is actively trying to minimize time to solution (i.e., time per time step) through optimizing time-step algorithms, optimizing linear-system solvers and preconditioners, and enabling the use of GPUs.
Full details about the ExaWind project and codes here.
Be the first to comment