One-Shot MNIST Model Uses Just 984 Parameters
See how a 984-parameter few-shot model reaches 62.46% MNIST accuracy on a phone CPU.
30-Second TL;DR
What Changed
Uses one unmodified support image per MNIST class, for 10 support images total.
Why It Matters
The result highlights how few trainable parameters are needed for a compact few-shot classifier when most feature extraction is fixed. However, the moderate accuracy shows that extreme parameter efficiency currently comes with a substantial performance trade-off on this task.
What To Do Next
Reproduce the 81-to-12 projection in PyTorch and benchmark accuracy, latency, and memory while varying the embedding size and number of augmented queries.
Key Points
- •Uses one unmodified support image per MNIST class, for 10 support images total.
- •A fixed vision pipeline compresses inputs to 81 dimensions, followed by a trainable 81-to-12 linear layer.
- •Cosine-similarity prototype matching is trained with 200 augmented queries per class per episode.
- •The experiment achieved 62.46% validation accuracy with 984 learnable parameters.
- •Training took about 0.5 seconds per epoch, or 90 seconds over 172 epochs, on one Cortex-X4 core.
Deep Insight
AI-generated analysis for this event — not the original article.
Enhanced Key Takeaways
- •The model architecture utilizes a fixed, non-trainable feature extractor based on Gabor filters or similar hand-crafted kernels to achieve extreme parameter efficiency.
- •This approach falls under the 'Extreme TinyML' category, specifically targeting edge devices with sub-millisecond inference latency requirements.
- •The 62.46% accuracy benchmark is constrained by the lack of deep feature extraction, highlighting a trade-off between model size and the ability to capture complex spatial hierarchies.
- •The use of cosine-similarity prototype matching allows the model to perform inference without requiring backpropagation or gradient updates during the deployment phase.
- •The experiment serves as a proof-of-concept for 'Zero-Shot' or 'One-Shot' learning on microcontrollers (MCUs) where memory (SRAM) is limited to a few kilobytes.
Technical Deep Dive
- Architecture: Fixed feature extraction pipeline (likely linear projection or fixed-weight convolution) followed by a single trainable linear layer (81x12).
- Parameter Count: 81 * 12 = 972 weights + 12 biases = 984 parameters.
- Training Methodology: Episodic training using Prototypical Networks framework, where each episode consists of a support set and a query set.
- Inference Mechanism: Computes the mean vector (prototype) for each class in the 12-dimensional embedding space and assigns the query to the nearest prototype via cosine distance.
- Hardware Optimization: Designed for ARM Cortex-X4 architecture, leveraging SIMD instructions for the dot-product operations required by the linear layer.
Future ImplicationsAI analysis grounded in cited sources
Weekly AI Recap
Read this week's curated digest of top AI events →
AI-curated news aggregator. All content rights belong to original publishers.
Original source: Reddit r/MachineLearning ↗
This is a summary, not the original. Read the source, or get the weekly briefing.
The weekly digest
One email a week. Unsubscribe anytime.