WorldClaw: Agentic 3D Open-World Generation at Scale
06:00 · August 7, 2026 · arXiv cs.AI RSS
Generating large-scale, freely explorable 3D worlds from open-ended text remains challenging because a system must jointly maintain global spatial coherence, rich local content, and explicit assets suitable for downstream editing and reuse. We present WorldClaw, a fully agentic, coarse-to-fine framework for open-world 3D scene generation. Planning agents translate a text prompt into a structured specification of regions, terrain, assets, materials, and spatial relations. WorldClaw then builds a globally coherent terrain foundation from semantic layouts, reusable assets, generative or procedural materials, and a region-aware height field. For detail-demanding regions, it generates terrain-conditioned compositions, reconstructs editable textured meshes, and recovers their placement on the terrain; render-based agents further refine terrain, objects, appearance, and contacts. Across diverse open-world prompts, WorldClaw produces large-scale scenes with coherent spatial organization, visually compelling local content, and editable instance-level assets while preserving a consistent global terrain structure.
Summary
WorldClaw addresses the longstanding difficulty of turning open-ended text prompts into large-scale, freely explorable 3D environments that remain spatially coherent, visually detailed, and explicitly editable. Unlike prior approaches that either rely on rigid procedural rules, lift images or videos into 3D with limited consistency, or diffuse directly over 3D representations constrained by scarce training data, the system adopts a fully agentic, coarse-to-fine pipeline. Planning agents first translate a user prompt into a structured specification that defines regions, terrain characteristics, object categories, materials, and spatial relations.
A global terrain stage then instantiates this specification as a region-aware height field augmented by semantic layouts, reusable assets, and generative or procedural materials. Render-based agents subsequently refine geometry, appearance, and surface contacts across the entire foundation, establishing a consistent spatial scaffold without attempting to synthesize every detail at once.
In regions that require richer content, the framework renders local terrain views, employs image-editing models conditioned on both global context and terrain cues to compose object layouts, reconstructs the inserted elements as textured meshes via image-to-3D conversion, and recovers their placement on the terrain. A final agentic refinement loop corrects scale, pose, and object–terrain contacts. The resulting scene is represented as independently editable textured meshes with explicit terrain placements, supporting free-viewpoint navigation and conventional downstream pipelines in gaming, simulation, and virtual environments.
Why it matters
This research is highly relevant for Dutch AI researchers and practitioners in the creative industries, gaming (e.g., Guerrilla Games), and digital twin sectors. It provides a novel, scalable approach to 3D environment generation using LLM agents and foundation models, offering actionable methodologies for advanced simulation development.
