Tag: #llm-efficiency12 results
TurboQuant algorithm adapted from KV-cache to model weights, offering drop-in nn.Linear replacement with near-optimal 4-bit quantization and lossless 8-bit residual. Benchmarks on Qwen2.5-0.5B show zero PPL increase at 50% size; 4B tests confirm minimal degradation. GitHub includes Triton kernels and full docs.
LDP is a new AI-native protocol for multi-agent LLM systems, featuring delegate identity cards, progressive payloads, governed sessions, provenance tracking, and trust domains. Implemented as a JamJet plugin, it reduces latency by 12x on easy tasks, cuts token usage by 37%, and improves security in simulations. Evaluations highlight efficiency gains but note limitations in small delegate pools and noisy provenance.