From Infrastructure to Production: A Year of Self-Hosted LLMs
UD2.120 (Chavanne) | Day 1 | 16:30 - 16:50 | Speakers: Mateusz Charytoniuk, Gosia Zagajewska, Luiz Miguel
Abstract
Last year, I shared Paddler, an open-source LLM load balancer. A year of community feedback and building Poet (a static site generator with AI features) on top of it taught me what actually matters when self-hosting LLMs. This talk shares practical patterns the open-source community needs. What works, what doesn't, and what tooling we still need to build together.
Attachments
Speakers
Mateusz Charytoniuk is a Technical Founder at Intentee, where he leads the development of innovative conversational applications builder. Mateusz is also involved in projects like Paddler, a llama.cpp-based load balancer. His expertise is further showcased through his active Open Source contributions.
Ex-head of product at Packhelp, where she led internal tools and applications. Built products at organizations of different sizes, giving her insights into why traditional software development practices should no longer be maintained. Co-founder at Intentee.
Luiz Miguel is a Software engineer, developing new systems from scratch to production. Currently working on Intentee as Product Engineer. He largely contributes to open source projects, with more participation on Paddler, a local infrastructure AI environment builder.
Links
External Links
Notice: The placeholder video image is licensed under CC BY-SA 4.0. The original image can be found hereChanges made to the image are: Cropped the image to a new ratio, part of the image was cut off.