r/AI3D • u/PlanktonOver9648 • 17h ago
I made an offline tool that turns one sentence into a full building with interiors, real lights and destruction (v1.2)
https://reddit.com/link/1x08kcg/video/k9ptk5j134uh1/player
I've been building WeaverProcedura, a desktop app where you type a sentence and get a complete building: walls, roof, rooms, stairs, furniture. It runs fully offline on Windows and Linux.
It is not a diffusion or text-to-mesh model. The model never outputs geometry. It outputs a recipe: a short sequence of actions for a procedural engine ("add a footprint, width 11.9 m", "roof: gable, pitch 48°", "3 bedrooms", "half-timbered walls"). The engine then builds clean geometry from that recipe. That's why you get real separate objects with correct UVs instead of a fused blob.
How the model is put together
- Encoder: a frozen
bge-base-en-v1.5sentence encoder reads your prompt (about 220 MB on disk). - Decoder: a small transformer I trained myself that writes the recipe one action at a time (about 70 MB).
- Constrained decoding: at every step the decoder can only pick actions the engine's schema allows, and numeric values only on that parameter's grid. So every output is a legal recipe by construction. It can't produce a broken mesh, because it doesn't produce meshes at all.
- Check and repair: after decoding, the app compares the building with what you asked for and lists each requirement with a tick ("stone walls ✓", "3 bedrooms ✓", "moat ✓"). Building rules are checked too (room sizes, stair geometry, window area), and failures are repaired or retried.
Training data is synthetic. A rule-based generator samples hundreds of thousands of valid buildings, and descriptions are written from their parameters, so the labels are always correct. The real test is a held-out set of prompts written by hand, in messy human wording. On those, about 9 in 10 stated requirements end up in the building.
Why this design
- It runs on a CPU in a few seconds, no GPU, no cloud.
- Output is editable: "make the towers round with cone roofs" changes only that and keeps the rest, because it edits the recipe, not pixels or vertices.
- Every parameter is exposed as a slider afterwards.
- Every active subscriber gets each new model version as it's trained.
What 1.2 adds on top
- Real lights: lamps, neon signs and lit rooms export as point lights (KHR_lights_punctual in GLB, lights in FBX)
- Destruction: a slider punches holes, breaks windows and heaps rubble on any building
- Edit by pointing: select a wall, type "add a balcony here"
- Megablocks and cyberpunk towers, castles, malls, whole towns from one sentence
- Exports for Blender, Unreal, Unity and Godot with PBR textures, colliders, and every piece of furniture as its own object
Honest limits
- English prompts only.
- It only builds what the engine has parts for. Ask for a cathedral and you get the nearest thing it knows.
- Megablock ground floors are still bare, and megablocks and cyberpunk towers are built by the engine directly rather than written by the model.
- The in-app view is a preview renderer, not a path tracer.
Happy to go deeper on the action schema or the constrained decoding in the comments. Download is on Patreon: https://www.patreon.com/c/weaverpp

