Benchmarking platform for robot localization systems

Building localization and navigation systems requires a lot of time, effort and statistical analysis over multiple scenarios. There are multiple sparse tools out there to create specific KPIs around datasets but almost none that provide end to end integration test scenarios, KPIs and automatic report generation for localization systems. Last year, at FOSDEM, we presented beluga - https://github.com/Ekumen-OS/beluga - a Monte Carlo Localization Framework for robotics. This year, we would like to present one of the tools we built for this project, Lambkin, the Localization and Mapping BenchmarKINg system that allows to orchestrate test runs of your code, data, KPIs and report generation all in one platform. You can find the code here: https://github.com/Ekumen-OS/lambkin