LLM Tool use in vLLM

Tool use or function calling is one of the key features required to realize the promises of agentic applications. While it is still a novelty with a lot of experimentation and variation among model providers, protocols such as the OpenAI function calling extensions of the chat completions API have become a standard for a specific case of tool calling. In this talk we're going to explain how function calling works all the way from the API endpoint down to the actual model prompt. We're going to explore the different kinds of interaction flows that are possible with this protocol and what the current limitations are.