#X01 C++/Kokkos Refactor of HOMME

Abstract

Under the CMDV Software Modernization project, we are working to refactor the HOMME spectral element atmosphere dycore. We are rewriting HOMME in C++ and making use of the Kokkos programming model with the goal of achieving high performing code on CPU, Intel Phi, and GPU architectures with a single code base. Our plan is to reproduce the algorithms in HOMME exactly to avoid introducing new verification/validation issues. Our initial effort involved extracting the main HOMME kernels for dynamics into a mini-app and comparing single-node performance of Fortran, C++, and C++-with-Kokkos implementations on different architectures. Significant effort has gone into improving the performance of the code on GPUs, and some work on OpenMP performance as well, that fed back to improvements in Kokkos itself. As a second step, we are now porting those kernels back into a fork of the ACME repository for HOMME runs. This project raises many issues for discussion: achieving performance, programming language/compiler issues, the difficulties of writing performant code on numerous architectures, the ability to staff climate projects for performance work, processes to expedite code development, and exploiting synergies with other DOE code development efforts.