Skip to main content
FOSDEM PWA

Single-source cross-platform GPU LLM inference with Slang and Rust

UD2.120 (Chavanne) | Day 1 | 13:05 - 13:25 | Speakers: Crozet Sébastien

Single-source cross-platform GPU LLM inference with Slang and Rust
A picture of a devroom at FOSDEM 2024
Open in browser

Notes

Abstract

Leveraging Rust and Khronos' emerging Slang initiative, we introduce our efforts toward a cross-platform GPU LLM inference ecosystem. With a single-source approach we aim to minimize backend-specific code and foster community participation by writing inference kernels once and run them everywhere.

Speakers

Crozet Sébastien

Sébastien Crozet has been in love with the Rust programming language since its earliest days. He is the creator and maintainer of popular open-source libraries, including nalgebra and Rapier, for the Rust ecosystem that specialize in linear algebra, geometry, and physics. He is the founder of Dimforge where he focuses on contributing to the future of cross-platform scientific computing and AI.

Notice: The placeholder video image is licensed under CC BY-SA 4.0. The original image can be found hereChanges made to the image are: Cropped the image to a new ratio, part of the image was cut off.