Rule-centric closed-loop evaluation

TrafficSignBench

High driving scores can hide
illegal behaviour

We introduce a large-scale closed-loop benchmark that jointly combines a broad traffic-sign taxonomy, automatic rule checkers, and rule-targeted scenarios for evaluating whether autonomous-driving planners obey traffic signs

Watch the overview Try the checker

34traffic signs

29'000closed-loop scenarios

17planners evaluated

Closed-loop rollout PlanT 2.0 · sign 5.19

Rule checker Violation at step 50

The metric gap. PlanT 2.0 reaches the destination with no collision and high comfort—yet fails to yield to a pedestrian.

Project video · 02:59

Overview of TrafficSignBench

From the metric gap to rule-supervised fine-tuning — a three-minute walkthrough of the benchmark, scenes, and results.

TrafficSignBench · overview Download MP4 ↓

Interactive stop-sign scenario

Think you can obey
the sign?

Take control of the ego vehicle with continuous steering, throttle, and braking. The benchmark checker runs alongside you, entirely in your browser.

Browser simulation

Stop sign · junction scene

Drive ↑ throttle · ↓ brake / reverse · ← → steer

Take control Click here or press an arrow key to drive

Real map geometry Executable rule checkers Targeted closed-loop scenes Rule-supervised fine-tuning Real map geometry Executable rule checkers Targeted closed-loop scenes Rule-supervised fine-tuning

High driving scores can hide illegal behaviour

The problem

Route progress, comfort, and collision rate describe how a vehicle moves. They do not tell us whether it understood the traffic rule that shaped the scene.

Observed rollout

Driving Score

100.00 Destination reached · no collision

Executable checker

Traffic-rule compliance

Failed Pedestrian-yield violation

CaRL closed-loop rollout failing to yield at a priority intersection. — **CaRL** CoRL 2025

Breaks the yield rule while completing the route.

PlanT 2.0 closed-loop rollout failing to yield to a pedestrian at a crosswalk. — **PlanT 2.0** arXiv 2025

Fails to yield to a pedestrian at the crosswalk.

PPO closed-loop rollout taking a prohibited direction at a mandatory-turn sign. — **PPO** lidar baseline

Ignores a mandatory-direction restriction.

High driving metrics do not mean rule compliance

Held-out evaluation

5,800 held-out episodes. SCD—Sign-Compliant Destination—requires both obeying the sign and reaching the intended goal.

PlanT 2.0 takes the prohibited branch at a direction sign. — **Direction · straight / left.** PlanT 2.0 takes the prohibited branch; the rule-supervised baseline reroutes.

Rule-supervised PlanT 2.0-FT reroutes along the allowed branch. — **Direction · straight / left.** PlanT 2.0 takes the prohibited branch; the rule-supervised baseline reroutes.

PlanT 2.0 ignores a posted speed limit. — **Speed limit.** PlanT 2.0 ignores the posted limit; the rule-supervised baseline slows down.

Rule-supervised PlanT 2.0-FT slows down under the speed limit. — **Speed limit.** PlanT 2.0 ignores the posted limit; the rule-supervised baseline slows down.

Planner results on the held-out test split
Planner	Driving score ↑	Destination ↑	Collision ↓	SCD ↑
IDM	35.7	66.4%	21.0%	7.2%
PPO	35.6	67.9%	27.5%	3.2%
CaRL	39.6	73.7%	21.0%	2.9%
PlanT 2.0	28.6	56.1%	43.3%	5.9%
PlanT 2.0-FT rule-supervised baseline	74.2	74.8%	12.0%	72.3%

Conventional metrics are episode-weighted; SCD is macro-averaged over scenario types.

Use the benchmark

Scenes, models, and evaluation code are open.

01Code & evaluationGitHub ↗ 02Benchmark scenesHugging Face ↗ 03Planner checkpointsHugging Face ↗

Cite TrafficSignBench

@misc{trafficsignbench2026,
  title     = {TrafficSignBench: Evaluating Traffic Rule Compliance
               in Autonomous Driving},
  year      = {2026},
  publisher = {GitHub},
  url       = {https://github.com/emb-ai/traffic-sign-bench}
}