Polynomial reduction with full domain decomposition preconditioner for spectral element poisson solvers

Minimizing communication is central to realizing high performance for scalable execution of parallel algorithms. On GPU-based systems, iterative solvers can be prohibitively expensive without an algorithm that concentrates most of the work on the GPU devices and lightens the load on the network. This work focuses on increased local work per iteration to reduce iteration counts and thus internode communication. We present a polynomial reduction with full domain decomposition (PR+FDD) preconditioner that targets the solution of spectral-element-based Poisson problems discretized by high-order spectral elements on GPU-based exascale architectures. The algorithm constructs local composite grids by first reducing the polynomial order of the elements adjacent to the GPU-local partition, followed by progressive geometric coarsening all the way to the domain boundary. During the preconditioning step of the iterative solver, the residual is restricted to the different levels of the coarsening tree and communicated so that each processor can solve its local problem independently. Once completed, the local solutions are stitched together and the global iterative solver continues. This class of algorithms is known to achieve fast convergence at the cost of more expensive preconditioner evaluations. The added extra cost can be offset by using GPUs in order to retain an overall solver speedup. On structured domains, our method achieves a solve time of 0.39 s with 8 billion DOFs on 4096 GPUs, surpassing the 0.54 s of geometric-multigrid (GMG) for the same problem. On unstructured domains, we demonstrate the effectiveness of the preconditioner in reducing the number of outer iterations, achieving a 1.5-3 times reduction compared to low-order preconditioning on different computational domains. Strong and weak scaling results are presented along with timing data for each algorithm component.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2022

Author dc:creator

• Bello-Maldonado, Pedro D.

Contributors dc:contributor

• Fischer, Paul F
• Olson, Luke N
• Kloeckner, Andreas
• Kolev, Tzanio

• Poisson
• scientific computing
• numerical analysis
• finite elements
• spectral element method
• high performance computing
• GPU

Statement dc:rights

• Copyright 2022 Pedro Bello-Maldonado

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/117701

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F117701