Towards unified full-stack performance analysis and automated computer system design at CERN with Adaptyst

Slow performance is often a major blocker of new visionary applications in scientific computing and related fields, regardless of whether it is embedded or distributed computing. This issue is becoming more and more challenging to tackle as it is no longer enough to do only algorithmic optimisations, only hardware optimisations, or only (operating) system optimisations: all of them need to be considered together.

Architecting full-stack computer systems customised for a use case comes to the rescue, namely software-system-hardware co-design. However, doing this manually per use case is cumbersome as the search space of possible solutions is vast, the number of different programming models is substantial, and experts from various disciplines need to be involved. Moreover, performance analysis tools often used here are fragmented, with state-of-the-art programs tending to be proprietary and not compatible with each other.

This is why automated full-stack system design is promising, but the existing solutions are few and far between and do not scale. Adaptyst is an open-source project at CERN (the world-leading particle physics laboratory) aiming to solve this problem. It is meant to be a comprehensive architecture-agnostic tool which:

• unifies performance analysis across the entire software-hardware stack by calling state-of-the-art software and APIs under the hood with any remaining gaps bridged by Adaptyst (so that performance can be inspected both macro- and microscopically regardless of the workflow and platform type)
• suggests automatically the best solutions of workflow performance bottlenecks in terms of one or more of: software optimisations, hardware choices and/or customisations, and (operating) system design
• scales easily from embedded to high-performance/distributed computing and allows adding support for new software/system/hardware components seamlessly by anyone thanks to the modular design

The tool is in the early phase of development with small workforce and concentrating on profiling at the moment. Given that Adaptyst has broad application potential and we want it to be for everyone’s benefit, we are building an open-source community around the project.

This talk is an invitation to join us: we will explain the performance problems we face at CERN, tell you in detail what Adaptyst is and how you can get involved, and demonstrate the current version of the project on CPU and CUDA examples.

Project website: https://adaptyst.web.cern.ch