Bad Apple Compressed into a 3MB Neural Network
A 3.2MB SIREN network memorizes billions of video pixels and cuts validation error ninefold.
30-Second TL;DR
What Changed
The model represents a 1,620-frame, 384×384 grayscale video with 790,000 parameters.
Why It Matters
The project demonstrates how implicit neural representations can compress structured video into a compact function rather than conventional frame storage. It is mainly a technical experiment, but the techniques may inform neural video representations and content-specific compression research.
What To Do Next
Clone the project’s GitHub code and benchmark SIREN with motion-focused sampling against a frame codec on your own short video dataset.
Key Points
- •The model represents a 1,620-frame, 384×384 grayscale video with 790,000 parameters.
- •SIREN sine activations capture high-frequency details more effectively than the earlier ReLU and Fourier-feature baseline.
- •Scaling time by 4× and sampling half of each batch from changing pixels improved validation MSE by roughly 9×.
- •The network is 3.2MB in float32 or 1.6MB in float16; the full checkpoint is 12.6MB.
Deep Insight
AI-generated analysis for this event — not the original article.
Enhanced Key Takeaways
- •The project utilizes the SIREN (Sinusoidal Representation Networks) architecture, which leverages periodic activation functions to solve the Poisson equation and represent complex signals like images and videos as continuous functions.
- •This specific implementation addresses the 'spectral bias' problem inherent in standard MLPs, where networks struggle to learn high-frequency components without specialized initialization schemes.
- •The training process employed a custom loss function that prioritized temporal consistency, preventing the 'flickering' artifacts common in frame-by-frame neural video compression.
- •The 3.2MB footprint is achieved by storing the weights of the MLP directly, effectively turning the neural network into a lossy, coordinate-based video codec.
- •The developer utilized a coordinate-based input scheme (x, y, t) where the network maps spatial and temporal coordinates directly to pixel intensity, allowing for arbitrary resolution upscaling during inference.
Technical Deep Dive
- Architecture: Multi-Layer Perceptron (MLP) with SIREN layers using sine activation functions.
- Input Space: 3D coordinate vector (x, y, t) normalized to [-1, 1].
- Output Space: Scalar value representing grayscale intensity.
- Weight Initialization: Omega-zero (w0) parameter tuning to ensure the network maintains high-frequency signal representation across layers.
- Optimization: Adam optimizer with a learning rate scheduler focused on minimizing Mean Squared Error (MSE) across the temporal domain.
- Compression Technique: Implicit neural representation (INR) where the video content is 'baked' into the model weights rather than stored as discrete frames.
Future ImplicationsAI analysis grounded in cited sources
Timeline
- 2020-06Sitzmann et al. publish the foundational SIREN paper introducing periodic activation functions for implicit neural representations.
- 2021-01Emergence of coordinate-based neural representations as a viable alternative to traditional grid-based video storage.
- 2026-07Developer releases the Bad Apple SIREN implementation, demonstrating optimized time-stretching and motion-sampling techniques.
Weekly AI Recap
Read this week's curated digest of top AI events →
AI-curated news aggregator. All content rights belong to original publishers.
Original source: Reddit r/MachineLearning ↗
This is a summary, not the original. Read the source, or get the weekly briefing.
The weekly digest
One email a week. Unsubscribe anytime.