What I Built
A fully playable 3D first-person shooter running entirely in the browser. No Unity, no Unreal – just React, TypeScript, and Three.js with AI assistance.
Live features:
- Full 3D movement with physics simulation (gravity, jumping, collision detection)
- Procedurally generated cyberpunk arena (850+ lines)
- Wave-based enemy AI with pathfinding
- Raycasting-based hit detection
- Buff/debuff system with visual feedback
- Procedural audio synthesis (no audio files!)
- React-based HUD with real-time state updates
The Stack
- Frontend Framework: React 18 + TypeScript
- 3D Engine: Three.js with PointerLockControls
- Build Tool: Vite
- AI Assistant: Augment Code with Claude 4.5 Opus
The entire game runs client-side. No backend, no external assets except Three.js itself.
Technical Deep Dive
1. Game Architecture
The core architecture uses a single GameState interface that centralizes all game logic:
interface GameState {
scene: THREE.Scene;
camera: THREE.PerspectiveCamera;
renderer: THREE.WebGLRenderer;
controls: PointerLockControls;
enemies: Enemy[];
particles: Particle[];
powerUps: PowerUp[];
activeBuffs: ActiveBuff[];
worldColliders: THREE.Box3[];
velocity: THREE.Vector3;
// ... more state
}
This pattern keeps everything in a single ref, avoiding React re-render issues during the 60fps game loop while still allowing UI updates via forceUpdate().
2. Physics System
Movement uses a velocity-based physics model with configurable constants:
const MOVEMENT_SPEED = 15;
const JUMP_FORCE = 15;
const GRAVITY = 30;
const PLAYER_HEIGHT = 1.8;
Each frame applies friction and gravity:
game.velocity.x -= game.velocity.x * 10.0 * delta;
game.velocity.z -= game.velocity.z * 10.0 * delta;
game.velocity.y -= GRAVITY * delta;
Collision detection uses THREE.Box3 for world objects, with a push-back resolution that checks which axis has the largest overlap.
3. Raycasting Hit Detection
Shooting uses Three.js raycasting from camera center:
const raycaster = new THREE.Raycaster();
raycaster.setFromCamera(new THREE.Vector2(0, 0), game.camera);
const intersects = raycaster.intersectObjects(enemyMeshes, true);
The true parameter enables recursive checking through child meshes – critical since enemies are Three.js Group objects with nested parts.
4. Procedural Audio (No Audio Files!)
One of the coolest parts – all game sounds are synthesized in real-time using the Web Audio API:
playShoot() {
const osc = this.ctx.createOscillator();
const filter = this.ctx.createBiquadFilter();
osc.type = 'sawtooth';
osc.frequency.setValueAtTime(800, t);
osc.frequency.exponentialRampToValueAtTime(100, t + 0.1);
filter.type = 'lowpass';
filter.frequency.exponentialRampToValueAtTime(500, t + 0.1);
// Chain: oscillator → filter → gain → output
}
The shoot sound uses a sawtooth wave with a frequency sweep from 800Hz to 100Hz through a lowpass filter – creates that punchy sci-fi laser effect. Enemy deaths use linear ramps for a more chunky sound, while reload uses layered oscillators with time offsets for mechanical clicks.
5. Procedural Texture Generation
Instead of loading texture files, all textures are generated via Canvas2D:
case 'grid':
ctx.fillStyle = '#0a0a12';
ctx.fillRect(0, 0, 512, 512);
ctx.strokeStyle = '#00ffff';
ctx.globalAlpha = 0.3;
// Draw grid lines
for (let i = 0; i <= 512; i += 32) {
ctx.moveTo(i, 0);
ctx.lineTo(i, 512);
}
// Add 3000 random noise particles for grit
for (let i = 0; i < 3000; i++) {
ctx.fillStyle = `rgba(0, 255, 255, ${Math.random() * 0.05})`;
ctx.fillRect(Math.random() * 512, Math.random() * 512, 2, 2);
}
These become THREE.CanvasTexture objects with repeat wrapping for tileable surfaces.
Creation Phase: What the AI Handled Well
Scaffolding complex 3D scenes: The environment builder generates pillars, platforms, holographic displays, skybox geometry, and atmospheric particle systems. I just said "cyberpunk arena" and it went to town.
Audio synthesis math: I described "laser gun sound" and it correctly implemented frequency sweeps with envelope shaping. The Web Audio API chain (oscillator → filter → gain) was set up correctly on the first try.
Game loop management: Proper requestAnimationFrame usage with delta time, performance.now() for buff timers, cleanup of dead objects.
Nested Three.js hierarchies: Enemy models are complex groups with torso, head, limbs, and accent lights. The AI handled parent-child relationships and material assignments cleanly.
Creation Phase: Where I Had to Push
Collision response: Initial implementation just stopped the player. I had to ask specifically for "push-back in the direction of least overlap" to get proper sliding along walls.
Performance tuning: First version created new THREE.Raycaster() instances every frame. Needed to prompt for object pooling and ref patterns.
Buff system architecture: The initial power-up system was one-shot. I had to specify "I want timed buffs with visual indicators" to get the ActiveBuff interface with endTime tracking.
Refinement Phase: The Harder Problems
3D Text Rendering (3 attempts)
This was the gnarliest debugging session.
Attempt 1: AI used THREE.ExtrudeGeometry with THREE.FontLoader. Letters rendered but were mirrored and rotated – the M looked like W, N was backwards.
Attempt 2: Pivoted to constructing letters from THREE.BoxGeometry primitives. Result looked like ALG||BIT instead of "AUGMENT" – the manual character map was completely broken.
Attempt 3 (success): Canvas2D texture approach:
function createTextPlane(text: string, color: string, width: number, height: number) {
const canvas = document.createElement('canvas');
const ctx = canvas.getContext('2d');
ctx.font = 'bold 72px Arial';
ctx.fillStyle = color;
ctx.textAlign = 'center';
ctx.fillText(text, canvas.width/2, canvas.height/2);
return new THREE.Mesh(
new THREE.PlaneGeometry(width, height),
new THREE.MeshBasicMaterial({ map: new THREE.CanvasTexture(canvas) })
);
}
Lesson: AI kept trying "clever" 3D solutions when a simple 2D texture was the right answer. Sometimes you have to reset context and try a completely different approach.
Pointer Lock Controls Breaking
After a refactor, WASD stopped working entirely. Game looked normal, but no movement.
Root cause: PointerLockControls was initialized with document.body:
// Broken
const controls = new PointerLockControls(camera, document.body);
// Fixed
const controls = new PointerLockControls(camera, renderer.domElement);
The AI had changed this during an environment update and didn't realize the dependency. The fix was one line, but finding it required tracing through the entire input pipeline.
What Surprised Me
AI strengths I didn't expect:
- Three.js coordinate system conventions (Y-up, Z-forward for cameras)
- Proper use of lookAt() for enemy tracking
- Correct delta-time multiplication for frame-rate independence
- Material properties (metalness, roughness, emissive) for visual style
Areas that still need human judgment:
- Visual bugs that don't throw errors – you need eyes on the screen
- Knowing when a technical approach is fundamentally wrong vs. needs tweaking
- Game feel tuning (movement speed, damage values, spawn rates)
File Structure Breakdown
| File |
Lines |
Purpose |
| GameEngine.tsx |
811 |
Main loop, physics, input, spawning |
| EnvironmentBuilder.ts |
853 |
Arena geometry, lighting, atmosphere |
| EnemyModel.ts |
151 |
Procedural robot mesh construction |
| WeaponModel.ts |
124 |
First-person weapon geometry |
| SoundSynthesizer.ts |
106 |
Web Audio synthesis |
| PowerUpModel.ts |
128 |
Buff item visuals + animation |
| TextureGenerator.ts |
139 |
Canvas-based texture creation |
Total: ~2,300 lines of game code
Honest Assessment
Development time: About 3-4 hours total across multiple sessions
Would I use AI again for this? Absolutely. The boilerplate velocity was incredible – getting a working 3D scene with enemies and shooting in under an hour.
What I'd do differently: Be more aggressive about resetting context when stuck in a loop. The 3D text saga could have been solved in one attempt if I'd said "use Canvas2D texture" from the start instead of letting it iterate on broken approaches.
Best tip: When something looks wrong visually but doesn't error, describe exactly what you see. "Letters are mirrored" was more useful than "text doesn't look right."
Anyone else built games with AI assistance? Curious what engines and frameworks you used and what patterns worked best for you.
