The HPC group PRAM of the IT Center continuously develops its methodological expertise in high-performance computing and artificial intelligence on HPC systems in close collaboration with the Chair of Computer Science 12.
The main tasks of this working group include both research and methodological support for users regarding questions related to program development, debugging, performance tuning, and parallelization.
In addition, the group provides the necessary tools and resources for these tasks.
Design and implementation of a unified AI gateway.
Further development of a cross-site inference gateway that consolidates resources from different systems (HPC clusters, Kubernetes environments).
Definition, implementation, and maintenance of standardized APIs/interfaces for unified access to inference backends.
Configuration, deployment, and optimization of the execution of open-source or open-weight LLMs on high-performance computing (HPC) systems as well as in Kubernetes clusters.
Adjustment of operational conditions (resource allocation, scaling, container images) to the respective hosting scenarios.
Ensuring scalable load balancing, high resilience, and redundancy at both the gateway level and the local inference cluster.
Further development and configuration of monitoring for the inference infrastructure and individual model instances (metrics, alerting, dashboarding).
Design and execution of automated regression tests for AI models (performance, functionality, various modalities).
Planning and overseeing load and penetration tests of the entire inference infrastructure to ensure security and robustness.
Support in the design and implementation of agent-based processes (e.g., Retrieval-Augmented Generation) that utilize the provided AI models.
Integration of inference resources into RAG setups as well as connection to Model-Context-Protocol (MCP) services for seamless orchestration.
Linked-in
Google