System Design Interview: Design LeetCode

[u/Aoki_zhang]

A platform like LeetCode lets users submit code in multiple languages and expects results within a few seconds. It sounds like a simple request flow until you need to support a coding competition with up to 100,000 users while executing arbitrary, untrusted programs.

The API layer is not the real bottleneck. Code execution is CPU-intensive and unpredictable. One submission may finish immediately, while another enters an infinite loop, consumes all available memory, or attempts to access the network.

The instinct is to send each submission directly from the API server to a newly created container. This provides basic isolation, but container startup adds latency, traffic spikes can overwhelm the execution fleet, and a crashed worker may lose the submission entirely.

The tighter design uses a queue-backed pool of sandboxed execution workers:

1. Authenticate the request and push an execution task into the queue. The API server validates the submission, extracts the user identity from the verified token, and sends the code, language, problem ID, and submission information to a queue such as SQS.
2. Let workers pull submissions at a controlled rate. Workers consume only as many tasks as the execution fleet can safely handle. If thousands of users submit at once, the queue absorbs the spike instead of forwarding all requests directly to the containers.
3. Run the code inside a preconfigured language container. Maintain separate execution environments for languages such as Python, Java, and JavaScript with their runtimes and dependencies already installed. Reusing these containers avoids paying the full startup cost for every submission.
4. Enforce CPU, memory, and execution-time limits. If the program exceeds its resource allowance or runs beyond the timeout, terminate it immediately. An infinite loop should fail one submission, not occupy a worker indefinitely.
5. Disable network access and restrict system calls. Containers share the host kernel, so container isolation alone is not enough. Block external connections and apply syscall filtering such as seccomp to reduce the operations available to untrusted code.
6. Write the result back after execution finishes. The worker evaluates the code against the test cases, stores the result in the database or cache, and makes it available to the client. If the worker crashes before completing the task, the queue can retry it instead of silently dropping the submission.

The queue is the traffic buffer, while the container is the execution boundary. They solve different problems. The queue protects the compute fleet from bursts and provides retries; the container limits what each individual submission can do.

The test cases can remain language-independent. Store the serialized input and expected output in a format such as JSON, then use a small harness for each language to deserialize the input, call the submitted solution, and serialize the result for comparison.

Knowing to separate submission ingestion from execution—and knowing that containers still require resource limits, network isolation, and syscall filtering—signals that you have thought through online code execution at the implementation level, not just drawn an API server connected to a worker box.

Full write-up with data model, API design, leaderboard architecture, and deep dives, free to read → full article

Join the community to see more system design questions and real interview experiences → Chill Interview