LAMBKIN: the Localization And Mapping BenchmarKINg Toolkit

🌐 Overview

LAMBKIN is a mixture of automation and conventions to facilitate reproducible benchmarking and evaluation of localization and mapping systems, featuring:

• Reproducible environments
• Declarative benchmark definitions
• Standard performance metrics
• Automatic report generation

🧩 Components

LAMBKIN aggregates a number of open-source libraries and tools, both local to the project and third-party, that can be mixed and matched as necessary:

Component	Description
core/lambkin-shepherd	A Python package for RobotFramework-based benchmark definitions and orchestration.
core/lambkin-clerk	A Python package for system introspection, useful to contextualize benchmark reports.
external/ros	ROS (1) distribution for the target platform (typically Ubuntu).
external/ros2	ROS 2 distribution for the target platform (typically Ubuntu).
external/typesetting/latex	TeX Live distribution for the target platform.
external/profiling/timemory	Source distribution of timemory, the performance analysis and logging toolkit.

These components are distributed and deployed using containers. Each container image bundles a set of components with an operating system, which ensures reproducibility (barring hardware and kernel dependencies).

📈 Benchmarks

The simplest approach to LAMBKIN is to draw inspiration from existing sample benchmarks. These are written as containerized applications, built from a base package (typically a ROS package), using RobotFramework-driven definitions and Sphinx-powered reports.

Next steps

• Run a sample benchmark or write your own!
• Read the contributing guidelines.