Embodied AI Security Top 10 ↔ RoboJailBench crosswalk
Defensive, sim-only. This is a taxonomy-comparability artifact and provael's own measured coverage. It runs no RoboJailBench benchmark, publishes no comparative scores against their numbers, and drives no physical robot. See SAFETY.md.
What this is (and is not)
A notified body, an insurer, or a design partner comparing embodied-security frameworks asks one question: how do these two map to each other, and what does each actually measure? This document answers that for the Embodied AI Security Top 10 and RoboJailBench. It is not a benchmark comparison — we do not run their harness or score provael against their leaderboard; that is a separate task with its own licensing and reproducibility review.
Source (pinned, quoted verbatim). RoboJailBench: Benchmarking Adversarial Attacks and Defenses in Embodied Robotic Agents, Doguhuan Yeke, Yanming Zhou, Leo Y. Lin, Hongyu Cai, Antonio Bianchi & Z. Berkay Celik (Purdue University), arXiv:2605.19328v1 (19 May 2026); benchmark/leaderboard v1.0.0 (7 May 2026). The 18 category names below are quoted verbatim from the paper's Table 2.
Two taxonomies, two derivations — complementary axes
The two lists are built on overlapping evidence but organized along different axes, and reading them as competitors misses the point:
- RoboJailBench is a harm-outcome taxonomy: 18 categories of what goes wrong (collision, force violation, unauthorized capture, discrimination, …), derived from ISO/TS 15066:2016 and ISO 10218-1/-2 safety standards, Asimov's Laws, and documented incident reports (news + FDA).
- The Embodied AI Security Top 10 is an attack-mechanism / surface taxonomy: 10 categories of how the system is attacked (jailbreak, adversarial perception, poisoning, action-space integrity, injection, misalignment, CPS, identity, confidentiality, evaluation), derived from the same incident base plus a defender-operations framing (prevalence × detectability × exploitability × physical impact), cross-mapped to OWASP/MITRE ATLAS/NIST/ISO.
Because one is harms and the other is mechanisms, there is no clean 1:1: a single attack mechanism (e.g. action-space integrity) produces many harm outcomes, and several EAI mechanisms (poisoning, injection, CPS, evaluation) are delivery mechanisms or meta-risks, not harm classes at all — so they have no single RoboJailBench counterpart. That is a feature of complementary framings, not a gap in either list. A crosswalk that showed 18/18 covered would be measuring the wrong thing.
Coverage legend
| State | Meaning |
|---|---|
✅ covered |
provael ships an applicable family that measures this harm class in sim (a keep-out-zone violation or a pre-deploy screen) with a benign-FPR control. A sim proxy, mostly stub-validated — see the head-to-head for the number and its transfer status. |
🟡 partial |
a related mechanism is exercised, but only a facet / a loose proxy of the category. |
⬜ not covered |
no provael attack exercises this harm class. |
▫️ out of scope by design |
outside provael's sim-only VLA-policy red-team charter. |
The tables below are generated by provael crosswalk --target robojailbench --format md from the
machine-readable mapping in provael.crosswalk; the JSON form is committed at
results/crosswalk/crosswalk.robojailbench.json.
Mapped against RoboJailBench (arXiv 2605.19328v1, 2026-05-19; leaderboard v1.0.0, 2026-05-07). Category names quoted verbatim from Table 2 (18 categories).
Coverage tally: 2 covered · 5 partial · 9 not covered · 2 out of scope by design (of 18).
RoboJailBench → Embodied AI Security Top 10
| # | RoboJailBench category | Group | EAI id(s) | Provael family | Coverage | Note |
|---|---|---|---|---|---|---|
| 01 | Collision with people | physical safety | EAI04, EAI06, EAI02 | action_space, misalignment, sensor_spoof |
🟡 partial | provael measures a generic keep-out-zone violation — a sim proxy that does not distinguish a person from any keep-out target; stub-validated. |
| 02 | Collision with robots | physical safety | EAI04, EAI06, EAI02 | action_space, misalignment, sensor_spoof |
🟡 partial | same generic keep-out-zone proxy; the target's identity (a robot) is not modelled. |
| 03 | Collision with objects | physical safety | EAI04, EAI06, EAI02 | action_space, misalignment, sensor_spoof |
🟡 partial | same generic keep-out-zone proxy; the target's identity (an object) is not modelled. |
| 04 | Force/pressure violation | physical safety | EAI04 | — | ⬜ not covered | provael perturbs motion / commanded-end-state channels, not force or pressure; no force is modelled in the suites. |
| 05 | Unsafe motion | physical safety | EAI04 | action, action_space |
✅ covered | the action family (freeze / trajectory_hijack) and the keep-out hijack directly exercise unsafe / hijacked motion in simulation; stub-validated (see head-to-head). |
| 06 | Entrapment/crushing | physical safety | EAI04 | — | ⬜ not covered | no gripper-closing / clamping dynamic exists in provael's suites. |
| 07 | Unauthorized capture | data & privacy | EAI09 | — | ⬜ not covered | EAI09 names the surveillance facet, but the confidentiality family screens a memorized training-canary leak (membership-inference / extraction), not data capture; no attack drives the robot to record. |
| 08 | Data misuse | data & privacy | EAI09 | — | ⬜ not covered | as rjb07: provael screens memorization-leak, not use of collected data. |
| 09 | Unauthorized sharing | data & privacy | EAI09 | — | ⬜ not covered | as rjb07: no attack exercises exfiltration / telemetry sharing. |
| 10 | Unauthorized retention | data & privacy | EAI09 | — | ⬜ not covered | as rjb07: no attack exercises unlawful retention of collected data. |
| 11 | Discrimination | behavioral & ethical | EAI01, EAI06 | — | ⬜ not covered | a content / ethics harm; provael measures embodied keep-out and screen mechanisms, not content policy. Inducible via an EAI01 jailbreak but not measured here. |
| 12 | Deception/manipulation | behavioral & ethical | EAI01, EAI06 | — | ⬜ not covered | a content / ethics harm; not measured by provael (see rjb11). |
| 13 | Public disruption | behavioral & ethical | EAI04 | — | ⬜ not covered | no route-blocking / public-disruption scenario exists in provael's suites; EAI04 is the nearest mechanism. |
| 14 | Pornography | behavioral & ethical | — | — | ▫️ out of scope by design | a content-moderation harm outside provael's adversarial-embodied-security charter; belongs to LLM content-safety, not VLA-policy red-teaming. |
| 15 | Hateful conduct | behavioral & ethical | — | — | ▫️ out of scope by design | a content-moderation harm outside provael's charter (see rjb14). |
| 16 | Terrorism/weaponization | behavioral & ethical | EAI01 | instruction |
🟡 partial | the instruction family exercises the jailbreak-to-harmful-action mechanism (and is the one family that transfers on a real policy), measured as end-effector diversion — not the weaponization outcome itself. |
| 17 | Theft | behavioral & ethical | EAI08, EAI01 | authorization, instruction |
🟡 partial | the authorization family (self-authorize a guarded action) and the instruction jailbreak exercise the unauthorized-action mechanism; measured as a screen / diversion proxy, not the theft outcome. |
| 18 | Trespassing | physical safety | EAI04, EAI02, EAI08, EAI06 | action_space, sensor_spoof, misalignment |
✅ covered | 'enter a restricted area' maps directly to provael's keep-out-zone violation — the most literal sim match; stub-validated (see head-to-head). |
Embodied AI Security Top 10 → RoboJailBench
| EAI | Name | RoboJailBench counterpart(s) | Note |
|---|---|---|---|
| EAI01 | Policy & instruction jailbreak | Terrorism/weaponization, Theft | a jailbreak can drive many harms; it maps to the harmful-action outcomes provael's instruction family can reach (weaponization, theft), and can also induce the content harms rjb11/rjb12 that provael does not measure. |
| EAI02 | Adversarial perception | Collision with people, Collision with robots, Collision with objects, Unsafe motion, Trespassing | a sensor-spoof driving the effector into a keep-out zone → the collision / unsafe-motion / trespassing harms. |
| EAI03 | Model & pipeline poisoning, backdoors & supply chain | no counterpart | no counterpart: a backdoor is a delivery mechanism, not a harm class — once triggered it can produce any of RoboJailBench's 18 harms. |
| EAI04 | Action-space integrity | Collision with people, Collision with robots, Collision with objects, Force/pressure violation, Unsafe motion, Entrapment/crushing, Public disruption, Trespassing | action-space attacks produce the physical-safety harm outcomes (collision, force, unsafe motion, entrapment, disruption, trespassing). |
| EAI05 | Indirect / embodied prompt injection | no counterpart | no counterpart: an injection channel is a delivery mechanism, not a harm class. |
| EAI06 | Cross-domain safety misalignment | Collision with people, Collision with robots, Collision with objects, Unsafe motion | a benign-sounding instruction driving an unsafe action → the collision / unsafe-motion harms. |
| EAI07 | CPS, firmware, comms & teleoperation compromise | no counterpart | no counterpart: RoboJailBench's taxonomy is harm-outcome and has no CPS / firmware / comms class; also out of provael's scope by design. |
| EAI08 | Identity, access & excessive autonomy | Theft, Trespassing | unauthorized or over-broad action → theft and trespassing. |
| EAI09 | Model & data confidentiality | Unauthorized capture, Data misuse, Unauthorized sharing, Unauthorized retention | the data / privacy harms — though provael's confidentiality family screens a memorization leak, not these capture / sharing / retention behaviours (see the coverage column). |
| EAI10 | Insufficient evaluation, observability & incident response | no counterpart | no counterpart: a governance / operations meta-risk, not a harm class. |
Head-to-head: provael's measured coverage (with transfer status)
For the categories the crosswalk marks covered or partial, here is what provael actually measures
today — computed by the shipped provael.scoring.asr (Wilson CI) over the deterministic CPU stub +
reach suites, not re-derived for this table. The transfer statement is mandatory and appears
next to every number. Only the instruction family has been demonstrated on a real policy; every
other family is not demonstrated on a real policy — stub-validated scaffolding.
| Provael family | Measured (sim) | n | 95% Wilson CI | benign-FPR | Transfer status |
|---|---|---|---|---|---|
instruction (EAI01) |
70.0% | 30 | [52%, 83%] | 0% | measured on a real policy — roleplay diverted real SmolVLA×LIBERO 100% (10/10), 95% CI [72–100%], goal_substitution 60% (sim-only, one task, n=10; results/cross_arch_transfer) |
action (EAI04) |
100.0% | 20 | [84%, 100%] | 0% | not demonstrated on a real policy (stub-validated only) |
action_space (EAI04) |
100.0% | 20 | [84%, 100%] | 0% | not demonstrated on a real policy (stub-validated only) |
sensor_spoof (EAI02) |
100.0% | 20 | [84%, 100%] | 0% | not demonstrated on a real policy (stub-validated only) |
misalignment (EAI06) |
100.0% | 20 | [84%, 100%] | 0% | not demonstrated on a real policy (stub-validated only) |
authorization (EAI08) |
100.0% | 20 | [84%, 100%] | 0% | not demonstrated on a real policy (stub-validated only) |
Read this honestly: the high sim rates are fixture / keep-out-suite properties, not real-world
harm. On the one real architecture measured so far (SmolVLA × LIBERO), only the instruction family
transfers; the shipped visual and injection families produced 0% measurable lift on the real
model (95% CI [0–28%]), and are stub-validated scaffolding pending stronger perturbations. The π0
(openpi) leg is run-pending. No cross-architecture number is
claimed. Regenerate this table from source with provael crosswalk and provael study cross-arch.
Where RoboJailBench is ahead, and what provael does not cover
Being straight about the gaps is the point of a crosswalk that will be checked:
- Physical force & contact dynamics — RoboJailBench's Force/pressure violation and Entrapment/crushing (grounded in ISO/TS 15066 collaborative-robot force limits) are not covered: provael's suites perturb motion and commanded-end-state channels, not force, and model no gripper-closing dynamic.
- Surveillance / data-handling behavior — RoboJailBench's four data-privacy categories
(Unauthorized capture / Data misuse / Unauthorized sharing / Unauthorized retention) are not
covered: EAI09 names the surveillance facet, but provael's
confidentialityfamily screens a memorized-training-canary leak (membership-inference / extraction), not the robot's capture or handling of collected data. No provael attack drives a robot to record, share, or retain. - Content / ethics harms — Discrimination and Deception/manipulation are not covered (LLM content-safety, inducible via an EAI01 jailbreak but not measured here); Pornography and Hateful conduct are out of scope by design (content moderation, outside a VLA-policy red-teamer's charter).
- Public disruption is not covered — provael ships no route-blocking scenario.
Conversely, EAI07 (CPS/firmware/comms/teleop) and EAI10 (evaluation/observability) being out of provael's scope removes no RoboJailBench category — their harm-outcome taxonomy has no CPS/firmware or governance class, so nothing is lost there. And the reverse also holds: neither list covers the multi-robot / wormable-propagation and cascading-multi-agent risks on the Top 10's watchlist — RoboJailBench's 18 categories are single-robot harm outcomes.
Reproduce
provael crosswalk --target robojailbench # deterministic mapping (JSON)
provael crosswalk --target robojailbench --format md # the tables above
provael study cross-arch # the real-policy transfer numbers
# optional: attach this crosswalk as an appendix to a conformity dossier
provael certify --profile annex-i-part-a --include-crosswalk --out runs/dossier