A three-level MPI+NUMA+Threads method for constructing parallel programs to solve hydrodynamic problems for cluster systems with multiprocessor NUMA nodes

A parallel implementation of the program solving the three-phase filtration problem for a viscous compressible fluid to run on cluster systems with distributed memory with multiprocessor NUMA nodes is discussed. The traditional approach of constructing parallel programs to run on clusters with distributed memory uses the MPI library. Due to the specificity of the problem, we have to deal with unstructured computational grids and with the necessity of modeling dynamically changing sets of wells passing through a large number of grid blocks. This fact introduces a number of difficulties when using the traditional approach: a disbalance in loading the computing nodes, an increased transfer between MPI processes, and an increased amount of memory in use. It is proposed a three-level MPI + NUMA + Threads method of constructing a parallel program whose purpose is to eliminate the above difficulties. The method implements the idea of projecting the cluster node architecture (multi-core and nonuniform memory access nodes) onto the architecture of the parallel program. A comparison of programs implemented by using the proposed method and the traditional approach in terms of speed and memory usage and some results of numerical experiments performed on a large number of real problems is analyzed.