Skip to main content
FOSDEM PWA

os-test: Measuring POSIX compliance on every single OS

H.2215 (Ferrer) | Day 2 | 13:00 - 13:15 | Speakers: Jonas 'Sortie' Termansen

os-test: Measuring POSIX compliance on every single OS
A picture of a talk at FOSDEM 2024
Open in browser
Get involved in the conversation!Join the chat

Notes

Abstract

os-test: Measuring POSIX compliance on every single OS

What happens if you run tests on every POSIX system? You find a lot of bugs in every single OS. I parsed the new POSIX.1-2024 standard into API definitions, generated tests, and measured exactly how much of the standard is implemented. I invoked every libc function to see if they work, and began writing detailed test suites. As it turns out, if there's a sentence in POSIX, someone probably implemented it incorrectly. I found missing interfaces, incompatible declarations, namespace pollution, a lot of bugs, interesting benign differences, and many more issues. I published all tests and results as os-test. The volume of test failures makes it virtually impossible to report all individual issues to each upstream. However as all of the data is publicly available online, vendors are now beginning to incorporate os-test feedback into their development and testing process, which ultimately leads to improved POSIX compliance and software interoperability.

In this talk we will dive into the challenges of testing 16 different operating systems, survey the main findings of os-test, and finally determine which operating system takes the lead when it comes to POSIX conformance.

https://sortix.org/os-test/ https://sortix.org/blog/os-testing-posix-headers/

os-test is currently funded by Next Generation Internet Zero Commons.

Speakers

Jonas 'Sortie' Termansen

Hi, I'm Jonas Termansen, aka sortie. I've developed the Sortix operating system for the past 14 years, which is now nearing production quality and fully self-hosting its own builds, infrastructure, and web pages. I previously worked at Google for 8 years doing testing, release engineering, and supply chain security for the Dart programming language. I'm now working on Sortix full time, with NGI Zero Commons funding for Sortix os-test, which I built to help implement POSIX accurately in my operating system and other systems too, as a shared community project.

Notice: The placeholder video image is licensed under CC BY-SA 4.0. The original image can be found hereChanges made to the image are: Cropped the image to a new ratio, part of the image was cut off.