Project STAMP: Deterministic Verification Gates and Closed-Loop Context Repair for Multi-Arm VLA Task & Motion Planning

Research Highlights & Systems Summary

Strict Architectural Decoupling via Canonical Schemas: GeminiArgonPlanner never publishes to ROS 2 topics directly; instead, it emits a Pydantic v2 / JSON Schema Draft 2020-12 TaskDAG verified against a 3D SpatialState before DecoupledROS2Executor or MoveItPreDispatchGuard can dispatch a single motion primitive.
100.0% Deterministic Fault Interception Across Four Physical Layers: FrankaFR3DeterministicGate checks every candidate TaskNode against 7-DoF Damped Least Squares (DLS) IK reachability ([0.15m, 0.85m], residual ≤ 15.0 mm), 3D AABB collision volumes and table clearance (Z ≥ 0.20m), URDF joint/torque envelopes ([87..12] Nm), and multi-arm zone/tool time-window locks—blocking 610 / 610 injected faults in STAMP-Bench-V1.
+61.0% Task Completion Gain via Closed-Loop write_limit_back: Rather than aborting on a gate rejection, FeedbackContextStore.write_limit_back() serializes each HardwareFaultLog into an exact physical correction constraint ([STAMP_GATE_FAULT ...]) injected into the planner prompt, achieving 100.0% two-turn recovery (100.0% vs. 39.0% single-shot success across 1,000 episodes in 1.61 s wall-clock evaluation).

Fault Interception

100.0%

610 / 610 injected episodes blocked

Episode Completion

100.0%

+61.0% vs. Single-Shot VLA (39.0%)

Write-Back Recovery

2 Turns

100.0% convergence via fault hints

Unsafe Dispatches

0 / 1,000

vs. 610 / 1,000 unverified in baseline

Conceptual Simulation Preview: Spatio-Temporal Multi-Arm Coordination in Project STAMP Direct MP4 Stream (stamp-demo-veo.mp4) ↗

Why Direct VLA-to-Controller Execution Fails in Multi-Arm Workcells

When foundation models act as high-level task and motion planners for multi-manipulator cells, a recurring architectural anti-pattern is coupling the model's tool-call output directly to ROS 2 action servers or MoveIt 2 trajectory controllers. Even when a frontier model such as gemini-4-argon correctly decomposes a long-horizon task semantically, single-shot coordinate generation remains vulnerable to four physical failure modes: targets placed outside the manipulator's spherical reach envelope, end-effector bounding boxes that clip workbench obstacles, posture seeds that exceed distal wrist torque limits under payload, and overlapping time windows where two arms contend for the same shared workspace zone or fixture.

Project STAMP (Spatial, Task & Motion Planning, v1.0.0-stamp) resolves this vulnerability by introducing a deterministic verification air-gap between multimodal planning and physical execution. Instead of emitting raw controller commands, the planner is restricted to producing a declarative TaskDAG evaluated against a real-time 3D scene snapshot (SpatialState). No command can reach DecoupledROS2Executor or MoveItPreDispatchGuard unless every node in the directed acyclic graph passes FrankaFR3DeterministicGate with GateVerdict.passed == True.

Project STAMP 4-Stage Closed-Loop Execution Law Ref: M9G-TR-2026-03 (v1.0.0-stamp)

Plan [ GeminiArgonPlanner(Goal, SpatialState, C_repair) → TaskDAG ] → Gate [ FrankaFR3DeterministicGate(TaskDAG, SpatialState) → GateVerdict ] → { PASS: DecoupledROS2Executor(q_sol) | REJECT: FeedbackContextStore.write_limit_back(HardwareFaultLog) → C_repair }

Project STAMP 4-Stage Closed-Loop Architecture Diagram (M9G-TR-2026-03) — Figure 1 (M9G-TR-2026-03). Project STAMP four-stage architecture: (1) Multimodal Video & 3D Scene Graph Grounding, (2) Decoupled Gemini 4 Argon TaskDAG Planner, (3) Four-Layer FrankaFR3DeterministicGate & MoveItPreDispatchGuard, and (4) Verified ROS 2 Execution with closed-loop write_limit_back context repair.

Four Canonical Schema Contracts & Video-to-TaskDAG Ingestion

To guarantee zero ambiguity between the generative planner, the deterministic gate, and external Model Context Protocol (MCP) clients, stamp/stamp_schemas/ codifies four Pydantic v2 models exported as strict JSON Schema Draft 2020-12 artifacts (export_all_json_schemas()):

TaskDAG & TaskNode (TaskDAG.schema.json): Represents a parallel multi-arm task graph where each node specifies node_id, assigned manipulator arm_id (FR3_1, FR3_2, FR3_3), primitive action, target_pose [x, y, z, yaw] in world meters/radians, upstream dependencies, claimed shared_zone, required_tool, and scheduled time_window (t_start, t_end).
SpatialState (SpatialState.schema.json): Captures the complete 3D workcell state at planning time, including per-arm base transforms and 7-DoF joint states (RobotArmState with joint_torques_nm and custom URDF joint_limits_rad), tracked 3D objects (ObjectPose3D), 3D obstacle AABBs (WorkspaceZoneBox), and active mutex maps (active_zone_locks, active_tool_locks).
GateVerdict (GateVerdict.schema.json): Emitted by FrankaFR3DeterministicGate.evaluate_dag(), recording boolean passed, approved_node_ids, rejected_node_ids, pre-solved 7-DoF IK joint vectors joint_solutions (in radians), and cumulative interception_rate.
HardwareFaultLog & FaultCode (HardwareFaultLog.schema.json): Standardized diagnostic record binding a rejected node_id and arm_id to a deterministic FaultCode, scalar measured_value vs. limit_value, physical unit, and an actionable correction_hint string serialized via to_context_constraint().

Multimodal Video-to-TAMP Skill Grounding

Beyond static scene graphs, MultimodalVideoIngestor (stamp/video_ingestion.py) translates long-horizon multi-dish process videos (such as multi_dish_stir_fry_and_soup) into parallel 3-arm TaskDAG schedules. High-level semantic actions extracted from video streams are mapped deterministically onto parameterized Franka Emika FR3 motion primitives via VIDEO_ACTION_TO_FR3_SKILL while scheduling non-overlapping earliest-available timelines across FR3_1..3, shared zones, and tools:

M9G-TR-2026-03 • MultimodalVideoIngestor Skill Mapping & 6-Node Parallel TaskDAG Author: Minsu Chang, Ph.D. (M9G)

DAG Node ID	Video Action → FR3 Primitive	Assigned Arm	Shared Zone & Tool Lock	Target Pose `[x, y, z, yaw]`	Scheduled Window
`v1_chop_vegetables`	`chop` → `pick_and_oscillate_z` (3 strokes, 8.0 N)	`FR3_1`	`cutting_board` / `knife_fixture`	`[0.00, -0.93, 0.34, 0.0]`	`[0.00s, 1.60s]`
`v2_chop_tofu_parallel`	`chop` → `pick_and_oscillate_z` (3 strokes, 8.0 N)	`FR3_2`	`prep_board_2` / `cleaver_fixture`	`[0.80, 0.46, 0.34, 0.0]`	`[0.00s, 1.50s]`
`v3_transfer_veg_to_wok`	`transfer` → `pick_and_place` (clearance 0.14m)	`FR3_1`	`central_table` / `None`	`[0.00, 0.00, 0.35, 0.0]`	`[1.60s, 3.00s]`
`v4_pour_broth`	`pour` → `wrist_pitch_tilt` (1.35 rad, 0.6s dwell)	`FR3_3`	`central_table` / `ladle_vessel`	`[-0.04, 0.04, 0.40, 0.0]`	`[3.00s, 4.80s]`
`v5_wait_simmer`	`wait` → `hold_posture` (0.5s dwell)	`FR3_2`	`None` / `None`	`[0.80, 0.46, 0.38, 0.0]`	`[1.50s, 2.00s]`
`v6_transfer_tofu_to_pot`	`transfer` → `pick_and_place` (deps: `v4, v5`)	`FR3_2`	`central_table` / `None`	`[0.02, -0.02, 0.42, 0.0]`	`[4.80s, 6.20s]`

Inside the Four-Layer Deterministic Shield & Closed-Loop Write-Back

Every candidate TaskNode in a proposed TaskDAG is inspected sequentially across four orthogonal physical verification layers inside FrankaFR3DeterministicGate._evaluate_single_node() (and enforced at the controller boundary by MoveItPreDispatchGuard.verify_moveit_goal()). When any invariant is violated, the gate blocks execution and synthesizes a deterministic repair rule for FeedbackContextStore.write_limit_back():

Four-Layer Deterministic Gate Invariants (FrankaFR3DeterministicGate) Ref: M9G-TR-2026-03 • stamp/stamp_gate/__init__.py

1. Kinematic Reach & 7-DoF DLS IK: r_3D = ||p_target − p_base||₂ ∈ [0.15m, 0.85m] ∧ ||FK(DLS_IK(p_local, q_des, q_seed)) − p_local||₂ ≤ 0.015m (15.0 mm)
2. 3D Workspace & AABB Collision: z_target ≥ 0.20m ∧ ||p_xy − p_base,xy||₂ ≥ 0.12m ∧ ∀B ∈ Boxes_obs: AABB(p_target ± [0.04, 0.04, 0.05]m) ∩ B = ∅
3. Joint & Torque Envelopes: ∀j ∈ {1..7}: q_j ∈ [q_min,j, q_max,j] (URDF) ∧ |τ_j| ≤ τ_max,j ([87, 87, 87, 87, 12, 12, 12] Nm)
4. Spatio-Temporal Mutex Locks: ∀(n_a, n_b) claiming shared_zone Z or required_tool T (arm_a ≠ arm_b): max(t_start,a, t_start,b) ≥ min(t_end,a, t_end,b)

M9G-TR-2026-03 • Standardized Fault Taxonomy & Closed-Loop Repair Mechanics Author: Minsu Chang, Ph.D. (M9G)

FaultCode Enum	Deterministic Trigger Condition	Injected Context Constraint (`write_limit_back`)	Planner Turn-2 Repair Transformation
`IK_UNREACHABLE`	3D radial distance `r_3D ∉ [0.15m, 0.85m]` from assigned arm's `fr3_link0` base, or 150-iteration 7-DoF DLS IK residual exceeds `15.0 mm`.	`[STAMP_GATE_FAULT node=... arm=FR3_k code=IK_UNREACHABLE measured=1.98m limit=0.85m]: move target within 0.52m of base`	Projects target XY radially along the base-to-target vector onto the `r = 0.52m` kinematic sweet spot and clamps `Z ∈ [0.32m, 0.62m]`.
`WORKSPACE_COLLISION`	Target penetrates table floor (`Z < 0.20m`), enters the `0.12m` base column cylinder, or end-effector AABB intersects an obstacle `WorkspaceZoneBox`.	`[STAMP_GATE_FAULT node=... code=WORKSPACE_COLLISION]: End-effector AABB intersects obstacle; raise Z above box.max_z + 0.08m`	Lifts target `Z` to `max(0.34m, box.max_z + 0.08m)`, or retracts XY to `0.5 × base_xyz` at `Z = 0.38m` if vertical clearance exceeds `0.68m`.
`RESOURCE_CONTENTION`	Two manipulators claim the same `shared_zone` (e.g., `central_table`) or `required_tool` during overlapping `[t_start, t_end]` windows or active lock.	`[STAMP_GATE_FAULT node=... code=RESOURCE_CONTENTION]: Shared zone contended over [0.00s, 1.50s]; shift time_window after 1.50s`	Shifts the contending node's `time_window` to `[mutex_cursor + 0.15s, mutex_cursor + 0.15s + dur]` and releases conflicting foreign locks.
`JOINT_LIMIT_EXCEEDED`	Any of the 7 FR3 joint angles violates URDF `[q_min, q_max]` radians, or joint torque `\|τ_j\|` exceeds `[87, 87, 87, 87, 12, 12, 12] Nm`.	`[STAMP_GATE_FAULT node=... code=JOINT_LIMIT_EXCEEDED measured=115.0Nm limit=87.0Nm]: retract moment arm & enable reduced_payload_mode`	Retracts target moment arm to `0.45 × base_xyz` at `Z = 0.38m`, resets seed to `FR3_HOME_CONFIG`, and sets `reduced_payload_mode = True`.

Empirical Evaluation: STAMP-Bench-V1 (1,000 Fault-Injected Episodes)

To rigorously quantify the reliability gap between unguarded single-shot VLA planning and Project STAMP's closed-loop Plan → Gate → Write-Back architecture, we executed the 1,000-episode fault-injection benchmark frozen in datasets/stamp_bench_v1/stamp_bench_v1_manifest.json (SHA-256 trace digest 8d8b72bb75c3865e...98b4c0aad, deterministic RNG seed 202703). Across 1,000 parallel 3-arm episodes, the campaign injected synthetic physical faults at a 60% target rate (resulting in 610 fault-injected episodes and 1,076 total node-level fault events across out-of-reach object jitter at [1.42, -1.32, 0.34]m, central workspace obstacle blockers, 115.0 Nm joint torque spikes on J1, and concurrent central_table time-window collisions):

Figure 2A. Episode Task Completion & Fault Interception (1,000 Episodes) STAMP-Bench-V1 • Seed: 202703

Single-Shot VLA Planning (Unguarded Baseline)39.0% Completion (390 / 1,000 pass • 610 unsafe dispatches)

FrankaFR3DeterministicGate Fault Interception Rate100.0% Interception (610 / 610 fault episodes caught, KPI >95%)

Project STAMP (Plan → Gate → write_limit_back Re-Plan)100.0% Completion (+61.0%) (1,000 / 1,000 pass • 0 unsafe dispatches)

Figure 2B. Node-Level Hardware Fault Breakdown Intercepted by Gate (1,076 Total Faults) 610 Fault-Injected Episodes • STAMP-Bench-V1

WORKSPACE_COLLISION (Obstacle AABB & Table Z < 0.20m)471 faults (43.8% of intercepted node faults)

JOINT_LIMIT_EXCEEDED (115.0 Nm Torque Spike & URDF Rad Limits)304 faults (28.3% of intercepted node faults)

RESOURCE_CONTENTION (central_table & Shared Fixture Overlap)156 faults (14.5% of intercepted node faults)

IK_UNREACHABLE (Radial Reach > 0.85m & DLS IK Residual > 15mm)145 faults (13.5% of intercepted node faults)

Figure 2C. Cumulative Episode Convergence Across Planning Attempts (1,000 Episodes) Single-Shot vs. Closed-Loop Write-Back

M9G-TR-2026-03 • STAMP-Bench-V1 1,000-Episode Campaign Telemetry (stamp_bench_v1_manifest.json) Author: Minsu Chang, Ph.D. (M9G)

Benchmark Metric (1,000 Multi-Arm Episodes)	Single-Shot VLA Baseline	Project STAMP (`Plan → Gate → Write-Back`)	Systems Delta
Episodes Evaluated (`seed = 202703`)	1,000	1,000	Identical workload
Episodes with Synthetic Fault Injection (`p = 0.60`)	610 (61.0%)	610 (61.0%)	1,076 node faults
Deterministic Gate Fault Interception Rate	0.0% (0 / 610)	100.0% (610 / 610)	+100.0% (KPI >95%)
Unverified / Unsafe Hardware Dispatches	610 / 1,000	0 / 1,000	-100.0% (Zero unsafe)
Closed-Loop `write_limit_back` Recovery Rate	0.0% (No feedback loop)	100.0% (610 / 610 recovered)	2-turn convergence
Tokens per 6-Node DAG (Initial / Write-Back Repair)	1,060 tok (1 call)	1,060 tok (clean) / 1,870 tok (repaired)	+810 tok on fault only
1,000-Episode Campaign Wall-Clock Runtime (Cached DLS IK)	—	1.61 s (1.61 ms / episode)	Sub-2ms gate + repair
End-to-End Episode Task Completion Rate	39.0% (390 / 1,000)	100.0% (1,000 / 1,000)	+61.0% absolute gain

Decoupled Deployment: Standalone REST & Model Context Protocol (MCP) Server

A core systems requirement of Project STAMP is that deterministic kinematic and spatial verification should not be locked inside a monolithic simulator process. Through StampStandaloneEngine (stamp/api_server.py), the entire Plan → Gate → Write-Back loop operates as a zero-ROS-dependency microservice over HTTP REST (port :8095) or JSON-RPC 2.0 MCP stdio transport. External agent harnesses can pass custom robot URDF XML definitions (parsed dynamically via parse_urdf_joint_limits()) and reach bounds on every call:

M9G-TR-2026-03 • StampStandaloneEngine REST & JSON-RPC 2.0 MCP Interface Author: Minsu Chang, Ph.D. (M9G)

MCP Tool / HTTP Endpoint	Input Contract	Deterministic Engine Behavior & Output
`stamp_plan_and_verify` `POST /v1/stamp/plan_and_verify`	`goal`, `spatial_state`, optional `urdf_xml`, `reach_bounds_m`, `video_events`, `max_replans`	Applies custom URDF revolute/prismatic limits, executes closed-loop `ClosedLoopStampOrchestrator.run_episode()`, and returns verified `dag`, `verdict`, and `active_write_back_constraints`.
`stamp_verify_dag` `POST /v1/stamp/verify_dag`	Candidate `dag` (`TaskDAG`), `spatial_state`, optional `urdf_xml`, `reach_bounds_m`	Runs `FrankaFR3DeterministicGate.evaluate_dag()` in under 2 ms, automatically appends any `HardwareFaultLog` entries to `FeedbackContextStore`, and returns `GateVerdict` with pre-solved 7-DoF `joint_solutions`.
`stamp_write_limit_back` `POST /v1/stamp/write_limit_back`	List of `HardwareFaultLog` objects from simulator or hardware protective stops	Serializes hardware faults into deduplicated `[STAMP_GATE_FAULT ...]` prompt constraints and returns updated `fault_summary` counts by `FaultCode`.

Paired with our unattended batch harness (scripts/run_stamp_batch.py), cross-OS WSL2 FastDDS heartbeat watchdog (scripts/fastdds_watchdog.py), and SHA-256 dataset freezer (scripts/freeze_stamp_bench_v1.py), Project STAMP establishes a reproducible blueprint for deploying frontier multimodal models safely across physical multi-arm manipulation stations.

Research Provenance, Prior Art Disclosure & Anti-Scraping Notice

This technical article (M9G-TR-2026-03), the Project STAMP (v1.0.0-stamp) four-stage Plan → Gate → Execute → Write-Back architecture, the four-layer FrankaFR3DeterministicGate formulation, and the 1,000-episode empirical benchmark in STAMP-Bench-V1 (stamp_bench_v1_manifest.json) constitute original published prior art by Minsu Chang, Ph.D. (장민수 / M9G). This work is published exclusively as a web-native technical research disclosure on mincasurong.github.io. Unauthorized copying, translation, or conversion of these tables, schemas, or analyses into third-party conference papers (IEEE ICRA, IROS, RA-L, CoRL) or arXiv preprints is strictly prohibited.

Minsu Chang, Ph.D. (M9G), "Project STAMP: Deterministic Verification Gates and Closed-Loop Context Repair for Multi-Arm VLA Task and Motion Planning," M9G Technical Research Blog (M9G-TR-2026-03), Oct. 25, 2026. URL: https://mincasurong.github.io/blog/stamp/ | Benchmark: STAMP-Bench-V1 (v1.0.0-stamp)