Doom Runs Inside a 21B-Parameter Transformer—Without Training

Post LinkedIn

🤖Read original on Reddit r/MachineLearning

#weight-compilation #open-sourcetorchwright-doomdoom torchwright hugging-face transformers

💡See how a compiler turns a Doom renderer into a train-free, standard transformer checkpoint.

⚡ 30-Second TL;DR

What Changed

A custom compiler converts computation graphs directly into transformer weights, eliminating training.

Why It Matters

The project demonstrates that transformer weights can encode substantial deterministic computation, not just learned statistical behavior. It is primarily a research curiosity today, but the compiler approach could inform work on program synthesis, differentiable computation, and unusual transformer execution models.

What To Do Next

Clone the Torchwright Doom repository and load its Hugging Face checkpoint to evaluate how the compiler maps a deterministic computation graph into transformer weights.

Who should care:Researchers & Academics

Key Points

•A custom compiler converts computation graphs directly into transformer weights, eliminating training.
•The model accepts scene data as a 3,614-token prompt and generates 53,747 tokens of pixel-drawing commands.
•The checkpoint works with Hugging Face Transformers without trust_remote_code and includes a 43-line Python host program.
•Rendering one frame takes just over 40 minutes on an NVIDIA B200, or approximately 35 frames per day.

🧠 Deep Insight

AI-generated analysis for this event.

🔑 Enhanced Key Takeaways

•The project utilizes a technique known as 'weight-based computation' or 'model-as-code,' where the transformer's attention mechanism and feed-forward layers are mathematically mapped to simulate logic gates and memory registers.
•The 21B-parameter model architecture is based on a standard Llama-style decoder-only transformer, chosen specifically for its compatibility with the Hugging Face transformers library without requiring custom kernel modifications.
•The 3,614-token prompt acts as the 'state vector' for the Doom engine, encoding player position, weapon status, and map geometry, which the transformer processes to output the next frame's rendering instructions.
•This implementation demonstrates that large language models can function as universal Turing-complete interpreters, provided the weights are precisely initialized to represent the target algorithm's instruction set.
•The rendering process relies on a custom-defined token vocabulary where specific tokens map directly to primitive drawing operations like 'draw_line', 'fill_polygon', and 'set_color' rather than natural language text.

🛠️ Technical Deep Dive

Architecture: Decoder-only Transformer (Llama-based).
Parameter Count: 21 Billion.
Input Format: 3,614-token prompt representing game state.
Output Format: 53,747-token sequence of pixel-drawing primitives.
Compute Mapping: Weights are set using a custom compiler that maps Doom's C-based rendering logic to matrix multiplication operations.
Inference Hardware: NVIDIA B200 (Blackwell architecture) required for high-precision weight handling.
Execution Mode: Zero-training; weights are static and deterministic.

🔮 Future ImplicationsAI analysis grounded in cited sources

Weight-based computation will enable 'model-native' software execution.

The ability to encode arbitrary algorithms into weights suggests future models could execute complex logic without relying on external runtime environments.

Transformer-based rendering will lead to new forms of hardware-agnostic software distribution.

By packaging software as model weights, developers could theoretically run applications on any hardware capable of standard transformer inference.