How we built a new powerful JSON data type for ClickHouse

Day 1 | 14:45 | 00:30 | UB5.132 | Pavel Kruglov

Note: I'm reworking this at the moment, some things won't work.

The stream isn't available yet! Check back at 14:45.

JSON has become the lingua franca for handling semi-structured and unstructured data in modern data systems. Whether it’s in logging and observability scenarios, real-time data streaming, mobile app storage, or machine learning pipelines, JSON’s flexible structure makes it the go-to format for capturing and transmitting data across distributed systems.

At ClickHouse, we’ve long recognized the importance of seamless JSON support. But as simple as JSON seems, leveraging it effectively at scale presents unique challenges. In this talk we will discuss how we built a new powerful JSON data type for ClickHouse with true column-oriented storage, support for dynamically changing data structure and ability to query individual JSON paths really fast.

ClickHouse open-source project: https://github.com/ClickHouse/ClickHouse Links related to the topic: https://github.com/ClickHouse/ClickHouse/issues/54864, https://github.com/ClickHouse/ClickHouse/pull/58047, https://github.com/ClickHouse/ClickHouse/pull/63058, https://github.com/ClickHouse/ClickHouse/pull/66444.