Writing an ultrafast Lua/JSON encoder+decoder as a LuaJIT module

JSON is one of the most popular data exchange formats. Parsing routines for it exist in every modern programming languages, either built-in, or included in popular libraries such as RapidJSON for C++ or json for Rust.

The task of conversion between JSON strings and Lua objects has been solved plenty of times before, but either the solutions are not focused on performance, or the parsers are too strict for the "relaxed" format we use at BeamNG.

What if we want to have the fastest Lua table <-> relaxed JSON conversion possible? We came up with a highly optimized LuaJIT code we use for handling JSONs at BeamNG since a few years. But there is a way to go further -- hacking on the C source code of the interpreter itself to add compiled built-in JSON support. How much extra performance can we squeeze out by going a level deeper?

Get ready for juicy benchmarks and an optimization story from a real usage perspective.