Search

Few direct matches — filled in with the latest updates.

Tag: #entropy-gate1 results

AMOR: Entropy-Gated SSM-Attention Hybrid

AMOR: Entropy-Gated SSM-Attention Hybrid

AMOR is a hybrid model inspired by dual-process cognition theories, dynamically activating sparse attention only when SSM predictions show high entropy uncertainty. It projects Ghost KV from SSM states for O(n) efficiency, outperforming SSM-only and Transformer baselines on retrieval tasks with perfect accuracy using just 22% attention positions. Prediction entropy reliably detects retrieval needs with a 1.09 nats gap.

ArXiv AIResearchFeb 17#research#amor#architecture