Skip to main content
FOSDEM PWA

WebNN and WebLLM on RISC-V: Closing the AI Acceleration Gap with RVV and Tenstorrent

UD2.120 (Chavanne) | Day 1 | 12:40 - 13:00 | Speakers: Yuning Liang, Petr Penzin

WebNN and WebLLM on RISC-V: Closing the AI Acceleration Gap with RVV and Tenstorrent
A picture of a devroom at FOSDEM 2024
Open in browser

Notes

Abstract

As AI workloads move to the browser, the lack of a unified low-level acceleration layer on Linux—equivalent to DirectML or CoreML—creates major bottlenecks. In this talk, we explore how WebNN and next-generation WebLLM can unlock efficient on-device inference on RISC-V, using Tenstorrent hardware and the emerging RVV 1.0 Variable-Length vector ISA. We cover the challenges of WebNN integration on Linux, the importance of WASM support for RVV, and demonstrate progress on running modern LLMs directly in the browser. We will also detail the RVV-enabled WASM implementation path for WebNN and what’s needed upstream.

Speakers

Yuning Liang

Yuning Liang is the Founder and CEO of DeepComputing, focusing on developing innovative technology products based on RISC-V SoMs. From the world's first RISC-V development laptop DC-ROMA to pads, workstations, remote-controlled cars, drones, and more, all are based on RISC-V chips. The world's first RISC-V laptop, the world's first RISC-V pad capable of making phone calls, and so on, are all Yuning's masterpieces. Yuning's innovation and pioneering spirit in the RISC-V field have enabled him to create several world firsts, leading DeepComputing to gain widespread recognition in the global RISC-V product commercialization field, contributing significantly to the advancement and progress of RISC-V technology. Yuning's career has taken him from the UK to Switzerland, then to South Korea, and finally to China. He has a strong practical background in embedded systems, platform APIs, and system software. In 2024, he was honored with the "RISC-V Community Contributor Award" and recognized as a "Ubuntu Summit Contributor," further solidifying his influence in the technology sector.

Petr Penzin
Notice: The placeholder video image is licensed under CC BY-SA 4.0. The original image can be found hereChanges made to the image are: Cropped the image to a new ratio, part of the image was cut off.