Huge graph analysis on your own server with WebGraph in Rust

Huge graphs, from billions to trillions of nodes and arcs, are more and more common in modern data-intensive applications. Graph compression techniques allow loading and manipulating such graphs into main memory of either high-end workstations or commercial-grade servers, depending on graph size. In this talk, we report about webgraph-rs, a recent clean-slate Rust re-implementation of WebGraph, a state-of-the-art graph compression framework, formerly implemented in Java. webgraph-rs comes as a series of interconnected Rust crates of general interest, including high-performance bit streams, zero-copy (de)serialization, constant-time measurement of heap size for large data structures, and high-performance implementation of succinct data structures . Using webgraph-rs one can load graphs such as the Software Heritage Merkle DAG (about 0.6 trillion edges ) in less than 200 GiB of RAM and visit it in a few hours with a single core.