微軟警示生成AI記憶中毒攻擊

• Attack Delivery Mechanisms: Malicious URLs pre-populate AI assistant prompts using query string parameters (e.g., copilot.microsoft.com/?q=) that execute automatically upon clicking 'Summarize with AI' buttons[2] • Memory Injection Vectors: External actors inject unauthorized instructions or 'facts' into AI assistant memory, which the AI then treats as legitimate user preferences in future conversations[1] • Persistence Model: Once poisoned, memory entries persist across multiple conversations, allowing a single injection to influence recommendations indefinitely until manually removed[4] • Detection Keywords: Organizations can identify poisoning attempts by hunting for URLs containing keywords like 'remember,' 'trusted source,' 'in future conversations,' 'authoritative source,' and 'cite or citation'[3] • MITRE Classification: The technique is classified as AML.T0080 (Memory Poisoning) and AML.T0051 in the MITRE ATLAS knowledge base[2] • Microsoft Mitigations: Copilot implements prompt filtering, content separation between user instructions and external content, memory controls with user visibility, and continuous monitoring for emerging attack patterns[2]

AI Recommendation Poisoning represents a fundamental threat to the trustworthiness and neutrality of AI-assisted decision-making systems. As AI assistants become embedded in critical business processes—particularly in finance, healthcare, and security domains—the ability to silently manipulate recommendations without user detection creates systemic risk. The ease of deployment and widespread adoption across 14 industries suggests this will become a standard competitive tactic unless industry-wide defenses mature rapidly. Organizations will face pressure to implement memory auditing capabilities, and users may develop skepticism toward AI recommendations, potentially undermining adoption of beneficial AI tools. Regulators may eventually mandate transparency requirements around AI memory sources and manipulation detection. The discovery also highlights a broader vulnerability class: as AI systems become more autonomous and memory-dependent, the attack surface expands beyond traditional prompt injection to include persistent state manipulation.