M9G | Technical Research Blog
M9G-TR-2026-04 • SOTA Survey & Architectural Synthesis Published November 1, 2026 • NVIDIA Isaac Sim 6.0 / Isaac Lab · Self-Driving Labs · Project STAMP v1.0.0-stamp

SOTA Survey (2025–2026): Agentic Multi-Robot Manipulation in NVIDIA Isaac Sim — Assembly, Deformable Handling & Self-Driving Labs

Across industrial assembly, deformable object manipulation, and autonomous Self-Driving Laboratories (SDLs), state-of-the-art robotics architectures in 2025–2026 have converged away from direct end-to-end LLM-to-actuator dispatch. Instead, frontier systems adopt a 4-Tier Neuro-Symbolic + GPU Physics Pipeline: (1) Multimodal Task Graph Synthesis into Directed Acyclic Task Graphs (TaskDAGs) or Multi-Modal Temporal Plan Graphs (MM-TPGs), (2) Deterministic Pre-Flight Safety Gates & Contention Schedulers, (3) 50–120 Hz GPU-Accelerated Skill & Motion Control, and (4) Closed-Loop Context Repair via structured fault write-back. We synthesize 14 frontier frameworks—including MATTERIX, APEX-MR, RoboFactory, DexGarmentLab, ORGANA, and A-Lab GPSS—and demonstrate how Project STAMP (v1.0.0-stamp) unifies these paradigms in NVIDIA Isaac Sim 6.0 and ROS 2 Jazzy.

Author & Principal Researcher
Minsu Chang, Ph.D. (장민수 / M9G)
Domains Surveyed
Assembly · Deformables · Self-Driving Labs
Reference Implementation
IsaacSim_GeminiRobotics (v1.0.0-stamp) ↗
Disclosure ID
M9G-TR-2026-04 (14 SOTA Systems)
Executive Summary: The 4-Tier Neuro-Symbolic + GPU Physics Convergence
  • Tier 1 — Multimodal Task Graph Synthesis (Plan): Vision-Language-Action (VLA) models parse overhead/wrist video streams and 3D scene graphs into structured Directed Acyclic Task Graphs (TaskDAGs) or Multi-Modal Temporal Plan Graphs (MM-TPGs)—never raw joint trajectories.
  • Tier 2 — Deterministic Pre-Flight Safety Gate & Contention Scheduler (Gate): A deterministic verification layer checks kinematic reachability, 3D AABB swept-volume collisions, joint/torque bounds, and multi-arm mutual exclusion locks before any hardware or MoveIt 2 dispatch.
  • Tier 3 — 50–120 Hz Skill & Motion Control (Execute): GPU-accelerated inverse kinematics / MPC (cuRobo, Damped Least Squares IK), closed-chain leader-follower bimanual constraints, and outward tensioning for flexible containers.
  • Tier 4 — Closed-Loop Context Repair (Write-Back): Rejected plans or runtime pose slips generate structured fault certificates that are written back into the planner's system context (write_limit_back) or patched locally via 0-token subgraph splicing.
SOTA Systems Surveyed
14 Frameworks
RSS, ICCV, NeurIPS, Nature Comput. Sci., Matter
Makespan Reduction
-21% to -48%
ORGANA (-21.1%) & APEX-MR MM-TPGs (-48%)
STAMP Gate Shield
100.0%
610 / 610 injected multi-arm faults blocked
Write-Back Completion
39% → 100%
1,000 episodes via HardwareFaultLog repair
Tier 1 • Plan
Multimodal Agentic Planner

VLM / LLM grounds overhead + wrist RGB-D streams into a structured TaskDAG or MM-TPG with explicit spatial/tool locks.

Tier 2 • Gate
Deterministic Pre-Flight Gate & RCPSP

Verifies 7-DoF IK bounds, 3D AABB swept-volume collisions, URDF torque envelopes, and shared zone/tool time-window mutexes.

Tier 3 • Execute (PASS)
50–120 Hz Multi-Arm Control & Digital Twin

Executes DLS IK / cuRobo MPC and closed-chain bimanual trajectories inside 240 Hz PhysX TGS + SDF + PBD Isaac Sim scenes.

Tier 4 • Write-Back (REJECT)
Closed-Loop Context Constraint Injection

Serializes HardwareFaultLog certificates into planner context (write_limit_back) or splices 0-token recovery subgraphs.

1. Multi-Robot Collaboration & Contact-Rich Assembly

Coordinating 2 to 3 manipulators in a shared workspace introduces two primary bottlenecks: inter-arm spatial contention (collision deadlocks) and closed-chain kinematic over-constraint when holding a shared rigid part. Recent 2024–2025 systems decouple high-level task assignment from asynchronous temporal graph execution and GPU-accelerated collision-free motion generation:

Domain 1 • Multi-Robot Collaboration & Contact-Rich Assembly (2024–2025) M9G-TR-2026-04 Survey
Framework / Venue Core Architectural Contribution Key Takeaway for Agentic TAMP
APEX-MR (CMU — RSS 2025)
arXiv:2503.15836 · GitHub		 Asynchronous Multi-Robot Planning & Execution: Replaces lockstep barrier synchronization with Multi-Modal Temporal Plan Graphs (MM-TPGs), allowing non-contending arms to pre-position asynchronously and cutting multi-arm idle makespan by 48%. Pre-stage waiting arms (STAGED_PRE_GATE) outside the shared central zone while another arm holds the placement mutex.
RoboFactory & RoboFactory-VLA (ICCV 2025)
arXiv:2503.16408 · GitHub		 Compositional Constraints for Multi-Agent Manipulation: Formalizes spatial, temporal, and bimanual synchronization constraints as explicit guards between agent planning and trajectory execution. Enforce deterministic pre-flight fault codes (IK_UNREACHABLE, WORKSPACE_COLLISION, RESOURCE_CONTENTION, JOINT_LIMIT_EXCEEDED).
RoCo (ICRA 2024)
arXiv:2307.04738 · GitHub		 Dialectic Multi-Robot Collaboration with LLMs: Uses environment and IK validation feedback to iteratively refine multi-arm sub-task assignments and 3D waypoints. Replace expensive multi-turn agent debate with a single deterministic gate + structured write_limit_back repair prompt.
NVIDIA Factory & IndustReal (RSS / Isaac Lab)
arXiv:2205.03532 · NVIDIA Blog		 Fast SDF Contact & Task-Space Impedance in Isaac Sim: Combines Signed Distance Field (SDF) collision meshes with PhysX Temporal Gauss-Seidel (TGS) solvers to simulate sub-millimeter peg-in-hole and gear assembly at real-time rates. Pair 240 Hz PhysX TGS + high-friction grasp materials with C2-continuous quintic smoothstep trajectories for zero-strain bimanual transport.
NVIDIA cuRobo & isaac_ros_cumotion
cuRobo Docs · GitHub		 GPU-Accelerated Parallel IK & Collision-Free Motion Generation: Solves collision-free IK and MPC trajectories in <5 ms on CUDA and integrates with ROS 2 / MoveIt 2. Combine per-arm fast IK/MPC with a high-level contention-aware wavefront scheduler for multi-arm workcells.

2. Flexible & Deformable Object Handling in Isaac Sim

Manipulating compliant objects (flexible vinyl/silicone baskets, fabric bags, garments, and wire harnesses) requires tracking low-dimensional structural features rather than raw mesh vertices, and maintaining outward tension during bimanual transport:

Bimanual Structure-of-Interest (SOI) Outward Tension Constraint Ref: SOI Bag Manipulation / Project STAMP Mode 2 & 5
|| p_1(t) - p_2(t) ||_2 = d_nom + 2 * Δ_tension (maintains open container rim & prevents inward buckling during bimanual LIFT / TRANSPORT)
Domain 2 • Flexible & Deformable Object Manipulation in NVIDIA Isaac Sim / Isaac Lab M9G-TR-2026-04 Survey
Framework / Venue Core Architectural Contribution Key Takeaway for Deformable TAMP
DexGarmentLab & GarmentLab (NeurIPS 2025 Spotlight — Isaac Sim)
arXiv:2505.11032 · GitHub · arXiv:2411.01200		 Bimanual Deformable Benchmarks in Isaac Sim: Built on Isaac Sim's PhysX 5 Position-Based Dynamics (PBD) and FEM solvers across 15 bimanual task scenes. Uses category-level structural keypoints (left/right handles, rims, collars) for generalizable bimanual grasping. Ground VLM tool calls on canonical handle offsets (offset_1, offset_2) rather than raw deformable surface points.
MoDeSuite (Isaac Sim & Isaac Lab)
GitHub · NVIDIA DLO Guide		 Coordinated Deformable Cable, Cloth & Bag Manipulation: Evaluates mobile and stationary manipulators interacting with elastic ropes, cables, and bags in Isaac Lab. Couple base/arm reachability guards with compliant grasp dwell windows (>= 400 ms) before lifting.
Structure-of-Interest (SOI) Bimanual Bag Manipulation (IEEE / arXiv)
arXiv:2401.11432		 SOI Rim Tracking & Tensioned Lift: Identifies the bag opening rim and dual handles as Structures of Interest (SOIs) and enforces an outward bimanual tension offset (||p_1(t) - p_2(t)|| = d_nom + 2*Δ_tension) so the bag mouth stays open during packing and lifting. Apply an outward tension_offset along the handle axis during bimanual LIFT so packed items do not spill or pinch.

3. Self-Driving Laboratories (SDLs) & Isaac Sim Digital Twins

The most impactful application of Isaac Sim + Agentic TAMP + Deterministic Safety Gates in 2025–2026 is the rise of Self-Driving Laboratories (SDLs) for autonomous materials synthesis, wet-lab chemistry, and semiconductor process automation:

Domain 3 • Autonomous Self-Driving Laboratories (SDLs) & GPU Digital Twins (2025–2026) M9G-TR-2026-04 Survey
Framework / Venue Core Architectural Contribution Key Takeaway for Autonomous Lab TAMP
MATTERIX (Acceleration Consortium & NVIDIA — Nature Computational Science, 2025/2026)
Nature Comput. Sci. · News & Views · arXiv:2601.13232 · GitHub		 Multiscale GPU Digital Twin for Robotics-Assisted Chemistry Labs: Built natively on NVIDIA Isaac Sim and Isaac Lab. Combines hierarchical LLM agents with GPU physics (manipulators, glassware, powder/liquid particle dynamics, heat transfer, and reaction kinetics) to verify experimental workflows in silico prior to real-lab execution. Use Isaac Sim + deterministic safety gates as a pre-flight digital twin to catch collisions, spillage, and station contention before hardware execution.
ORGANA (Univ. of Toronto — Matter / Cell Press)
Matter Paper · Project Site		 LLM DAG Planning + Parallel Lab Scheduling: Converts natural-language experimental goals into a parallelizable TaskDAG, reducing experiment makespan by 21.1% and human intervention by >80%, while using overhead + wrist perception for closed-loop execution. Combine global overhead RGB-D (OverheadCamera) with eye-in-hand wrist cameras (wrist_camera) and wavefront DAG scheduling.
A-Lab GPSS (arXiv 2026) & AlabOS (Berkeley Lab)
arXiv:2604.11957 · arXiv:2405.13930 · Nature Rev. Chem. 2026		 Resource-Locked Lab OS & Agentic Fault Recovery: Models every workstation, vessel, and tool as a mutex-locked resource in a DAG manager (AlabOS) and feeds runtime hardware fault logs back into an LLM reasoning loop (A-Lab GPSS) for autonomous recovery in air-sensitive glovebox synthesis. Model shared stations (central_table, cutting_board, furnace, fixture) and shared tools in SpatialState.active_zone_locks and active_tool_locks with write_limit_back fault recovery.

4. Synthesis: How Project STAMP (v1.0.0-stamp) Unifies These Paradigms

In mincasurong/IsaacSim_GeminiRobotics, Project STAMP integrates the core strengths of MATTERIX, APEX-MR, RoboFactory, and ORGANA into a unified cross-OS framework:

  1. Decoupled Schema & Gate (stamp_schemas + stamp_gate): Enforces TaskDAG → FrankaFR3DeterministicGate → GateVerdict prior to ROS 2 / MoveIt 2 dispatch, achieving 100.0% fault interception (610 / 610 injected faults) across IK_UNREACHABLE, WORKSPACE_COLLISION, RESOURCE_CONTENTION, and JOINT_LIMIT_EXCEEDED.
  2. Closed-Loop write_limit_back Recovery (stamp_feedback): Appends deterministic HardwareFaultLog correction hints back into the planner's context, raising 1,000-episode multi-arm task completion from 39.0% (single-shot) to 100.0% (gate-assisted iterative replanning) with 0 unverified hardware dispatches.
  3. Dual-View Perception & Low-Latency Digital Twin (isaacsim_scripts/): Pairs /OverheadCamera with eye-in-hand /FR3_*/fr3_hand/wrist_camera sensors and a low-latency 320x240 Picture-in-Picture (PiP) viewport monitor inside Isaac Sim 6.0.
Project STAMP 4-Tier Neuro-Symbolic + GPU Physics Pipeline
Figure 1 (M9G-TR-2026-04). Project STAMP (v1.0.0-stamp) unifies multimodal TaskDAG synthesis, 4-layer deterministic safety gating, 50 Hz ROS 2 / Isaac Sim 6.0 control, and closed-loop write_limit_back context repair.
40s SOTA Survey Wrap-Up Sequence: Assembly, Deformables & Self-Driving Labs (gemini-omni-1.1-flash) Direct MP4 Stream (blog4-wrapup-omni.mp4) ↗

5. References & Primary Literature (2022–2026)

  1. Isaac Lab: Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning, arXiv:2511.04831 · GitHub
  2. MATTERIX: MATTERIX: toward a digital twin for robotics-assisted chemistry laboratory automation, Nature Computational Science (2025/2026), DOI:10.1038/s43588-025-00924-4 · arXiv:2601.13232 · GitHub
  3. APEX-MR: APEX-MR: Multi-Robot Asynchronous Planning and Execution for Cooperative Assembly, RSS (2025), arXiv:2503.15836 · GitHub
  4. RoboFactory: RoboFactory: Exploring Embodied Agent Collaboration with Compositional Constraints, ICCV (2025), arXiv:2503.16408 · GitHub
  5. DexGarmentLab: DexGarmentLab: Dexterous Garment Manipulation Environment with Generalizable Policy, NeurIPS Spotlight (2025), arXiv:2505.11032 · GitHub
  6. GarmentLab: GarmentLab: A Unified Simulation and Benchmark for Garment Manipulation, arXiv:2411.01200 · GitHub
  7. MoDeSuite: MoDeSuite: Mobile Manipulation Deformable Object Task Suite, GitHub
  8. SOI Bimanual Bag Manipulation: Bimanual Deformable Bag Manipulation Using a Structure-of-Interest Approach, arXiv:2401.11432
  9. ORGANA: ORGANA: A robotic assistant for automated chemistry experimentation and characterization, Matter (2024/2025), Cell Press · Project Site
  10. A-Lab GPSS: Agentic LLM Reasoning in a Self-Driving Laboratory for Air-Sensitive Solid-State Synthesis (2026), arXiv:2604.11957
  11. AlabOS: AlabOS: A Python-based Reconfigurable Workflow Management Framework for Autonomous Laboratories, arXiv:2405.13930
  12. RoCo: RoCo: Dialectic Multi-Robot Collaboration with Large Language Models, ICRA (2024), arXiv:2307.04738 · GitHub
  13. Factory: Factory: Fast Contact for Robotic Assembly, RSS (2022), arXiv:2205.03532
  14. NVIDIA cuRobo & cuMotion: cuRobo Documentation · isaac_ros_cumotion
Research Provenance & Citation

This technical survey and architectural synthesis (M9G-TR-2026-04) connects 2025–2026 multi-robot manipulation, deformable object handling, and Self-Driving Laboratory (SDL) digital twin literature with Project STAMP (v1.0.0-stamp) by Minsu Chang, Ph.D. (장민수 / M9G).

Minsu Chang, Ph.D. (M9G), "SOTA Survey (2025–2026): Agentic Multi-Robot Manipulation in NVIDIA Isaac Sim — Assembly, Deformable Handling & Self-Driving Labs," M9G Technical Research Blog (M9G-TR-2026-04), Nov. 1, 2026. URL: https://mincasurong.github.io/blog/sdl-survey/ | Reference Repo: https://github.com/mincasurong/IsaacSim_GeminiRobotics