Laboratoire en Informatique Haute Performance pour le Calcul et la simulation

SEANERGYS creates an integrated European software solution that optimises the operation of supercomputers. In doing so, it addresses four different objectives: reducing the amount of energy used for real-world workload mixes as the primary objective, optimising resource utilisation, enhancing system throughput and reducing response time as secondary objectives. Since these objectives can conflict with each other, site-specific policies define the weights attached to each, and the SEANERGYS SW suite will tailor system operation towards the combined optimum. Possible scenarios include improving the throughput of HPC systems, generating more R&D results for a given energy budget, or produce a fixed set of R&D results with less energy, while striving to keep response times constant. The solution consists of a comprehensive monitoring infrastructure (CMI), an Artificial Intelligence data analytics system (AIDAS), and a dynamic scheduling and resource management system (DSRM). The CMI gathers data from hardware and software sensors, and correlates it with scheduler information to identify jobs that do not fully utilize allocated resources. Users receive automatic feedback on energy and resource use for each run, plus information on how to optimize these. The DAIS leverages AI models trained with a vast set of operational data of the participating HPC sites. It fingerprints resource usage patterns, predicts future job behaviour, and identifies complementary job profiles for potential co-scheduling. Finally, the DSRM utilizes these insights to develop scheduling policies that maximize resource utilization and energy efficiency, and supports jobs/applications with dynamic and adaptable resource profiles. The SEANERGYS solution will be ready for deployment up to Exascale level. To ensure production-quality, the project builds on results from European projects, the competency of well-established research groups and companies, and widely used open-source codes. These are input for an integrated software system that achieves the functionality, performance and stability needed by European HPC centres, defined by KPIs and acceptance criteria and processes established at the project start. An agile, professional software development method will leverage a modern DevOps framework and strive to provide end-to-end traceability by linking and tracking requirements, interface, functional and performance specifications, code design and development steps, and validation/verification throughout the development lifecycle. Validation/verification measures will include code reviews, automated SW quality analysis, unit and integration tests and a verification suite. The project will implement a staged testing and validation process, with functionality tests on single-nodes, scaling tests on mid-sized platforms, and finally acceptance tests on production supercomputers.