Gemma 4 MTP Reverse Engineering Progress

Enhanced Key Takeaways

• •The .litertlm format is a specialized container introduced by Google for the LiteRT (formerly TensorFlow Lite) ecosystem, specifically designed to handle the complex memory mapping and weight-sharing requirements of on-device LLMs like Gemma 4.
• •Reverse engineering efforts are complicated by Google's use of custom 'Flex' delegates and proprietary operator fusion patterns within the TFLite graph, which often obscure standard transformer layer boundaries.
• •The community-led extraction relies on the assumption that Gemma 4 maintains architectural parity with the public Gemma 2/3 specifications, despite the heavy INT8 quantization and potential structural pruning applied for the LiteRT deployment.

Technical Deep Dive

• •Model Format: .litertlm (LiteRT Large Model), a container format optimized for memory-mapped weight loading on mobile/edge NPUs.
• •Quantization: Post-Training Quantization (PTQ) or Quantization-Aware Training (QAT) resulting in INT8 weights, likely utilizing symmetric per-channel quantization for weights and per-tensor for activations.
• •Graph Structure: The model is partitioned into multiple .tflite sub-graphs to bypass TFLite's historical 2GB file size limitation for single model files.
• •Inference Engine: Relies on the LiteRT runtime, which utilizes custom kernels for FlashAttention-like operations optimized for mobile GPU/NPU backends.

Future ImplicationsAI analysis grounded in cited sources

Timeline