Skip to main content

Live coding a streaming ChatGPT proxy with Swift OpenAPI—from scratch!

K.4.401 | Day 1 | 16:30 - 16:50 | Speakers: Si Beaumont, Honza Dvorsky

Live coding a streaming ChatGPT proxy with Swift OpenAPI—from scratch!
A picture of a devroom at FOSDEM 2024
Open in browser
Get involved in the conversation!Join the chat

Notes

Abstract

Join us as we build a ChatGPT client, from scratch, using Swift OpenAPI Generator. We’ll take advantage of Swift OpenAPI’s pluggable HTTP transports to reuse the same generated client to make upstream calls from a Linux server, providing end-to-end streaming, backed by async sequences, without buffering upstream responses.

In this session you’ll learn how to:

  • Generate a type-safe ChatGPT macOS client and use URLSession OpenAPI transport.
  • Stream LLM responses using Server Sent Events (SSE).
  • Bootstrap a Linux proxy server using the Vapor OpenAPI transport.
  • Use the same generated ChatGPT client within the proxy by switching to the AsyncHTTPClient transport.
  • Efficiently transform responses from SSE to JSON Lines, maintaining end-to-end streaming.

Speakers

Si Beaumont
Honza Dvorsky

Notice: The placeholder video image is licensed under CC BY-SA 4.0. The original image can be found hereChanges made to the image are: Cropped the image to a new ratio, part of the image was cut off.