backend.ai

Sokovan addresses the challenges of running resource-intensive batch workloads in a containerized environment. It offers acceleration-aware, multi- tenant, batch-oriented job scheduling and fully integrates multiple hardware acceleration technologies into various system layers to unleash the potential performance.

Sokovan has two levels of scheduling: a cluster-level node assignment scheduler and a node-level resource/device assignment scheduler. The cluster-level scheduler allows users to customize job placement strategies and control the density and priority of workloads, while the node-level scheduler optimizes per-container performance by automatically detecting and enabling hardware accelerators to each container.

This helps to improve the performance of AI workloads compared to Slurm and other existing tools. Sokovan has also been deployed on a large scale in various industries for a range of GPU workloads, including AI training and services. Its design and capabilities can help container-based MLOps platforms better utilize the latest hardware technologies.