
- 1. Why Original DaSH Stalls and How Lazy-DaSH Fixes It: Original DaSH (IEEE T-RO 2023) pre-validates collision-free roadmaps across every feasible pick, place, and handover pair, hitting quadratic memory growth and failing on occluded Shelf-Wall and non-monotonic Stocking tasks. Lazy-DaSH starts from minimal start and goal object poses in a Task Space Hypergraph and defers Lazy-PRM edge collision checks to the active task schedule.
- 2. Millisecond Task Conflict Detection and Branch-Scoped Feedback: Before building motion roadmaps, Lazy-DaSH checks grasp, placement, and handover anchors against static geometry, frontier blockers, and transition histories. When a front item blocks a rear slot, the planner records a branch-scoped constraint along the active search branch and samples an intermediate buffer pose only for that blocker.
- 3. Mapping Four Layers onto Three Gemini Models in
IsaacSim_Gemini: Gemini 4 Carbon (1xTurn-0 call,thinking_budget=256) synthesizes and queries the Task Space Hypergraph. Gemini Robotics-ER 2 andRuleBasedTaskVerifierrun millisecond conflict checks and resample intermediate buffer or mid-air handover poses on demand. Gemini 3.5 Flash-Lite (thinking_budget=0) dispatches verified steps over 50 Hz DLS IK andcuRobo. - 4. Four-Phase Roadmap and Seven Quantitative KPIs (2026–2027): Our upgrade targets at least a 76.6% API cost cut (
$0.01972) on 9-block assembly and under$0.025on 18-object non-monotonic tasks, a 90%+ (100x) reduction in motion collision checks, 100.0% pre-dispatch fault interception across 8 fault classes, and scaling from 3 FR3 arms in Isaac Sim 6.0 to 8–16 arms and 40–60+ objects in WSL2 MuJoCo 3.x.
gemini-omni-1.1-flash)
Direct MP4 Stream (blog5-wrapup-omni.mp4) ↗
Builds a minimal Task Space Hypergraph over start and goal object poses and robot-object composites in one call (thinking_budget=256).
Checks grasp, placement, and handover anchors against frontier and history blockers, and samples intermediate buffer poses on demand.
Dispatches scheduled steps with thinking_budget=0 and evaluates Lazy-PRM and 50 Hz IK edges strictly along active hyperarcs.
Scopes conflict certificates to the active search branch and expands only the blocked task or motion subgraph without global state blowup.
1. Inside UIUC's Lazy-DaSH: Why Original DaSH Hits a Quadratic Motion-Graph Wall
Multi-Robot Task and Motion Planning (MR-TMP) must choose discrete mode switches (pick, place, handover, support) while verifying continuous collision-free paths across coupled arms. Fully coupled search explodes exponentially with robot count. Purely decoupled search fails whenever tight handovers or temporary buffer placements are needed.
In 2023, UIUC's Parasol Lab introduced DaSH (Decomposable State Space Hypergraph) (J. Motes et al., IEEE T-RO 2023), modeling coupled robot-object subsets as vertices in a directed hypergraph for a 1,000x speedup over composite mode-graph baselines. In Lazy-DaSH (arXiv:2504.05552, IEEE T-RO), Seongwon "Scott" Lee, James Motes, Isaac Ngui, Marco Morales, and Nancy M. Amato show that original DaSH still breaks on two structural limits:
- Quadratic Upfront Motion-Graph Construction: Original DaSH builds and collision-checks PRM roadmaps for every feasible robot-object transition before running its combined query. More than 95% of those collision-checked motion hyperarcs never appear in the final schedule.
- Zero Task-Level Geometric or Non-Monotonic Feedback: Because original DaSH searches only over pre-sampled motion roadmaps, it cannot detect when a front item blocks a rear shelf slot (Shelf-Wall) or when front items must move to temporary buffers and return in reverse order (Stocking).
pick, place, and handover hyperarcs without pre-sampling intermediate buffers.transit, transfer, and coupled move hyperarcs in that schedule.How Lazy-DaSH Scales to 16 Arms and 60+ Objects Across Four Layers
- Layer 1, Task Planning Layer (Task Space and Task-Extended Hypergraphs): Lazy-DaSH starts from minimal start and goal object poses under static stability rules, skipping upfront buffer sampling. A depth-first query over the Task-Extended Hypergraph extracts an optimistic candidate task schedule in milliseconds.
- Layer 2, Task Conflict Detection Layer (Frontier and History Checks): Before constructing any continuous roadmaps, Lazy-DaSH samples grasp, placement, and mid-air handover anchors along the candidate task schedule and runs per-pose collision checks. If a grasp or placement collides with a blocking object at the current frontier (Fig. 3 in Lee et al.), the layer logs a frontier constraint or history constraint and samples an intermediate stable buffer pose only for that blocker.
- Layer 3, Lazy Motion Planning Layer (Lazy-PRM Motion Hypergraph): Using verified transition anchors, Lazy-DaSH builds unvalidated roadmaps (following Bohlin and Kavraki's Lazy-PRM) strictly inside the robot-object subspaces present in the candidate task schedule.
QueryMotionPlanchecks continuous edge collisions only when evaluating that active schedule. - Layer 4, Motion Conflict Resolution and Branch-Scoped Feedback: When two arms collide along concurrent hyperarcs or a partition wall blocks a transfer, Lazy-DaSH injects motion constraints or resamples mid-air handover anchors. Both task and motion constraints attach only to the active search branch and clear whenever new pose samples enter the graph, preserving probabilistic completeness without discarding valid schedules.
| Benchmark Scenario | Physical & Algorithmic Bottleneck | Original DaSH (T-RO 2023) | Lazy-DaSH (Scott Lee et al., UIUC) |
|---|---|---|---|
| 1. Sorting (Cross-Station Handovers) |
Quadratic growth of robot-to-robot handover pairs as manipulator and object counts increase. | Caps out at 4 robots / 14 objects before motion-hypergraph memory and build timeout. | Scales to 16 robots / 60+ objects (4x robots, 4x+ objects) while shrinking the Lazy Motion Hypergraph by 100x. |
| 2. Wall (Partition Obstructions) |
Thin partition walls block direct transfers and force multi-hop relay handovers above the wall. | Caps out at 5 robots / 10 objects due to repeated motion-level replanning stalls. | Scales to 8 robots / 40+ objects by resampling mid-air handover poses through motion-conflict feedback. |
| 3. Shelf-Wall (Deep-Shelf Occlusion + Handover) |
Front shelf items block access to rear goal slots while vertical dividers block cross-side reach. | Fails (0% solved) even at 2 robots within the timeout due to missing task-level geometric feedback. | Solves up to 4 robots / 32 objects; Task Conflict Detection enforces back-to-front shelf loading order. |
| 4. Stocking (Non-Monotonic Unpack & Restock) |
Existing front items must move to temporary buffers so new stock can enter behind them, then return in reverse order. | Fails (0% solved); unguided pose pre-sampling blows up search without finding buffer sequences. | Solves 2-layer (2 robots / 28 objects) and 3-layer non-monotonic stocking (2 robots / 18 objects) via targeted Task Space Hypergraph expansion. |
| 5. Lab (Wet-Lab Inspection) (Multi-Stage Uncap & Lid Mutex) |
Two 3-axis gantry robots uncap sealed specimen jars, park lids in a tight sanitization zone, inspect, and recap. | Solves small object counts; stalls once sanitization-zone lid placement collisions accumulate. | Solves more than 2x the object count across uncap → sanitize → inspect → recap cycles via lid buffer resampling. |
2. Mapping Lazy-DaSH onto Gemini 4 Carbon, Robotics-ER 2, and Flash-Lite in IsaacSim_Gemini
Comparing our multi-robot stack in D:/git/IsaacSim_Gemini, Project STAMP (M9G-TR-2026-03), and our 3-Arm VLA Benchmark (M9G-TR-2026-01) against UIUC's results makes the engineering parallel explicit. Computational TAMP and foundation-model VLA planners hit the exact same bottleneck from two angles:
- In computational TAMP (DaSH): The planner wastes 95%+ of its compute collision-checking motion roadmaps for transitions the final schedule never uses.
- In foundation-model VLA TAMP: A monolithic loop wastes latency and input tokens calling a frontier model (
gemini-4-carbonwith highthinking_budgetand dense camera frames) on every routine step and unverified candidate move.
Lee et al. note in their IEEE T-RO conclusion that Lazy-DaSH's conflict resolution layer pairs naturally with specialized schedulers and online execution in dynamic scenes. By mapping Lazy-DaSH's four layers onto Gemini 4 Carbon, Gemini Robotics-ER 2, and Gemini 3.5 Flash-Lite, IsaacSim_Gemini runs as an online, closed-loop Lazy-DaSH engine across NVIDIA Isaac Sim 6.0 and WSL2 MuJoCo 3.x:

Model and Module Allocation Across Lazy-DaSH's Four Layers
| Lazy-DaSH Layer | Assigned Model & Module in IsaacSim_Gemini |
Input, Thinking Budget & Call Rate | Algorithmic Responsibility |
|---|---|---|---|
| Layer 1: Task Space Hypergraph | Gemini 4 Carbon ( gemini-4-carbon in gemini_robotics_node.py) |
Symbolic 3D /tf start and goal poses + conflict log · thinking_budget=256 · 1× Call per Episode or Expansion |
Synthesizes the minimal Task Space Hypergraph over start and goal poses using pipelined pick, place, and handover hyperarcs. Adds intermediate buffer vertices only when Layer 2 emits a task-conflict certificate. |
| Layer 2: Task Conflict Detection & Buffer Sampling | Gemini Robotics-ER 2 + RuleBasedTaskVerifier( rule_based_verifier.py & stamp_gate) |
Deterministic 3D AABB & SDF + Overhead/Wrist PiP on conflict · < 5 ms Gate · On-Demand ER-2 | RuleBasedTaskVerifier checks grasp, place, handover, and SDL lid anchors in under 5 ms. When a frontier blocker or non-monotonic occlusion appears, Gemini Robotics-ER 2 samples collision-free intermediate buffer or mid-air handover poses. |
| Layer 3: Lazy Motion Hypergraph & Step Dispatch | Gemini 3.5 Flash-Lite + 50 Hz DLS IK / cuRobo( multi_robot_controller.py) |
Verified candidate task schedule · thinking_budget=0 · 10× Step Calls + 50 Hz Control |
Dispatches active hyperarcs in the candidate task schedule with zero thinking-token overhead. Evaluates Lazy-PRM and 50 Hz Damped Least-Squares IK trajectories strictly for the arms and objects in that active schedule. |
| Layer 4: Motion Conflict Resolution & Scoped Write-Back | stamp_feedback + RCPSP Wavefront Scheduler( center_occupied_by mutex) |
HardwareFaultLog + active search branch path · 0-Token Local Splice or Scoped Escalation |
Clears inter-arm timing contentions with time-window mutexes (STAGED_PRE_GATE) or zero-token subgraph splices. If a geometric handover or buffer block persists, write_limit_back sends a branch-scoped conflict certificate to Layer 1 or Layer 2. |
3. Four-Phase Upgrade Roadmap and Seven Measurable KPIs for IsaacSim_Gemini
To turn Lazy-DaSH's hypergraph architecture into verified runtime gains inside D:/git/IsaacSim_Gemini, our Four-Phase Upgrade Roadmap (Nov 2026 – Q3 2027) tracks seven quantitative Key Performance Indicators (KPIs):
| KPI Dimension | Legacy Baseline (Eager Single-Model Loop) | Current IsaacSim_Gemini (v1.0.0-stamp · Stage 3) |
Lazy-DaSH × Multi-Model Target (v2.0 Roadmap) |
|---|---|---|---|
| KPI-1: Multi-Arm & Object Scale | 2–3 FR3 arms · 6–9 rigid blocks (monotonic tabletop only) | 3 FR3 arms · 9–12 objects (Tower, Conveyor, Deformable Bag, SDL Vial) | 3–4 FR3 arms in Isaac Sim 6.0 (18–32 objects) & 8–16 arms in WSL2 MuJoCo 3.x (40–60+ objects) |
| KPI-2: Motion Validation & IK Query Load | 100% eager pre-validation across candidate robot-object pairs (quadratic growth) | Per-turn RuleBasedTaskVerifier + 50 Hz DLS IK on active step |
90%+ reduction (100x fewer collision checks) by restricting Lazy-PRM edge checks to the active task schedule |
| KPI-3: Non-Monotonic & Multi-Hop Handover Rate | 0% on occluded Shelf-Wall or multi-layer Stocking (no buffer discovery) | Single-hop central-jig handoff (Z2_PrecisionAssemblyJig) |
95%+ completion on 2–3 layer non-monotonic Stocking & cross-wall multi-hop handovers via frontier and history buffer expansion |
| KPI-4: VLA Token & API Cost per Episode | 50,190 tok ($0.07860 on ER-2) · 38,713 tok ($0.08437 on Pure Carbon) |
38,277 tok ($0.01972, -76.6% cost) on 9-block tower (1C + 10F) |
Under $0.025 per 18-object non-monotonic episode (1× Carbon hypergraph + up to 2× ER-2 buffer resamples + 0-think Flash-Lite steps) |
| KPI-5: Turn-0 & Scoped Replan Latency | 8,870 ms Turn-0 plan + 13.31 s/turn step latency (Pure Carbon) | 4,670 ms Turn-0 plan + 4.17 s/turn step latency (-68.7%) |
Under 5.0 ms Layer-2 Task Conflict Detection + under 1.5 s branch-scoped replanning on conflict trigger |
| KPI-6: Pre-Dispatch Safety & SDL Invariant Shield | 39.0% single-shot episode completion under injected faults | 100.0% fault interception (610/610) & 100.0% write-back completion |
100.0% interception across 8 fault classes (adding FRONTIER_OCCLUSION, HANDOVER_IK_INVALID, and SDL_LID_VERTEX_CONFLICT) |
| KPI-7: Cross-Simulator Parity (Isaac Sim 6.0 & MuJoCo 3.x) | Windows Isaac Sim only (16–24 GB GPU VRAM required) | Dual bridge (three_robot_tower.py + WSL2 mujoco_sim_node.py) |
100% identical ROS 2 /tf & Hypergraph API: run 1,000-episode Lazy-DaSH sweeps in headless WSL2 MuJoCo at 10x real-time, then verify top schedules in 240 Hz Isaac Sim 6.0 |
Four-Phase Engineering Implementation Schedule in D:/git/IsaacSim_Gemini
- Phase 1, Declarative Task Space Hypergraph and Pose-Typed Object Vertices (Nov–Dec 2026):
Extend
RuleBasedTaskGeneratorandstamp_schemasso each movable part inWorkcellSpec(ChamferedPeg_M12,SiliconeSampleBag_DualHandle,ReagentVial_20mL,HexagonalStock_Prism) compiles into start and goal pose vertices linked by typedpick,place,handover, anduncap_lidtask hyperarcs. Gemini 4 Carbon solves this compact hypergraph in one Turn-0 call (thinking_budget=256). - Phase 2, Sub-5 ms Task Conflict Detector and ER-2 Buffer Resampler (Jan–Feb 2027):
Add Lazy-DaSH's
TaskConflictDetectionpass toRuleBasedTaskVerifier(rule_based_verifier.py). When a candidate schedule grasps an item occluded by a front blocker (Stocking or Shelf-Wall) or parks two SDL container lids in overlapping sanitization slots (Lab vertex conflict), the verifier emits a branch-scoped conflict certificate and invokes Gemini Robotics-ER 2 to sample a collision-free intermediate buffer or mid-air handover pose. - Phase 3, Lazy Motion Hypergraph Cache with
Gemini 3.5 Flash-Liteand 50 Hz DLS IK (Mar–May 2027): Pair Gemini 3.5 Flash-Lite (thinking_budget=0) with a Lazy-PRM and 50 Hz Damped Least-Squares IK motion cache insideMultiRobotController. The controller evaluates swept-volume AABB clearances only along transitions in the active task schedule, cutting motion-validation checks by over 90%. - Phase 4, 8-to-16 Arm MuJoCo 3.x Sweeps and Isaac Sim 6.0 Digital Twin Transfer (Jun–Sep 2027):
Run our WSL2 MuJoCo 3.x bridge (
wsl_ws/launcher_mujoco.sh) to benchmark 8-to-16 manipulator Sorting and Wall handover topologies alongside 3-arm non-monotonic Stocking and SDL Wet-Lab inspection in NVIDIA Isaac Sim 6.0, streaming live hypergraph frontier states to the React 19 and@xyflow/reactv12 UI.
4. References & Primary Literature
- Lazy-DaSH (Primary Paper Analyzed): Seongwon Lee, James Motes, Isaac Ngui, Marco Morales, and Nancy M. Amato (Parasol Lab, University of Illinois Urbana-Champaign), "Lazy-DaSH: Lazy Approach for Hypergraph-based Multi-robot Task and Motion Planning," IEEE Transactions on Robotics (T-RO) · arXiv:2504.05552v2 (2025/2026) · Hardware Validation Video ↗
- DaSH: James Motes, Tan Chen, Timothy Bretl, Marco Morales Aguirre, and Nancy M. Amato, "Hypergraph-based Multi-Robot Task and Motion Planning," IEEE Transactions on Robotics (T-RO), vol. 39, no. 5, pp. 4166–4186, 2023.
- Lazy-PRM: R. Bohlin and L. E. Kavraki, "Path Planning Using Lazy PRM," in Proc. IEEE Int. Conf. Robotics and Automation (ICRA), 2000, pp. 521–528.
- Adaptive Robot Coordination (ARC): I. Solis, J. Motes, M. Qin, M. Morales, and N. M. Amato, "Adaptive Robot Coordination: A Subproblem-Based Approach for Hybrid Multi-Robot Motion Planning," IEEE Robotics and Automation Letters (RA-L), vol. 9, no. 8, pp. 7238–7245, 2024.
- APEX-MR: P. Huang, R. Liu, C. Liu, and J. Li, "APEX-MR: Multi-Robot Asynchronous Planning and Execution for Cooperative Assembly," Robotics: Science and Systems (RSS), arXiv:2503.15836, 2025.
- Project STAMP (M9G-TR-2026-03): Minsu Chang, Ph.D., "Project STAMP: Decoupling Multimodal VLA Reasoning from Robot Execution via Deterministic Pre-Dispatch Gating and Fault Write-Back," https://mincasurong.github.io/blog/stamp/, 2026.
- Symbolic 3D Grounding & VLA Task Planning (M9G-TR-2026-01): Minsu Chang, Ph.D., "Symbolic 3D Grounding and Software-Pipelined VLA Task Planning: ER-2 vs. Gemini 4 Carbon," https://mincasurong.github.io/blog/VLA/, 2026.
This technical report and architectural roadmap (M9G-TR-2026-05) connects UIUC Parasol Lab's Lazy-DaSH (Seongwon "Scott" Lee et al., IEEE T-RO, arXiv:2504.05552) with the four-phase multi-model VLA upgrade plan for IsaacSim_Gemini by Minsu Chang, Ph.D. (장민수 · M9G).