AI Hacking Gets Dangerous With Human Expertise

Read original on Wired AI

#ai-security#human-in-the-loop#red-teaming#cybersecurityai-assisted-hackingjames-kettlewired

AI hacking is most dangerous when skilled humans guide it—an essential warning for agent builders.

30-Second TL;DR

What Changed

James Kettle conducted experiments to assess how far AI can advance hacking activities.

Why It Matters

AI security assessments should account for hybrid human-AI workflows, not just autonomous agents. Organizations may face greater risk from skilled attackers who use AI to accelerate reconnaissance, exploitation, and decision-making.

What To Do Next

Use a red-team framework such as Inspect AI to evaluate your agents against human-guided attack scenarios before deployment.

Who should care:Researchers & Academics

Key Points

•James Kettle conducted experiments to assess how far AI can advance hacking activities.
•The most effective approach combines AI capabilities with human expertise rather than relying on full automation.
•Human operators may amplify AI-driven security risks by guiding, validating, or refining its actions.

Deep Insight

AI-generated analysis for this event — not the original article.

Enhanced Key Takeaways

•James Kettle, a prominent researcher at PortSwigger, demonstrated that AI agents can successfully chain together multiple vulnerabilities to achieve complex exploits that single-purpose scanners miss.
•The research highlights the 'human-in-the-loop' paradigm as a force multiplier, where AI handles the high-volume reconnaissance and pattern matching while humans focus on high-level strategy and bypass techniques.
•Kettle's experiments utilized custom-built AI agents integrated with Burp Suite, allowing the AI to interact directly with web application traffic and stateful authentication flows.
•The study identified that AI-driven hacking tools are particularly adept at discovering 'business logic' flaws, which are traditionally difficult for automated tools to detect because they require understanding the application's intent.
•Security experts warn that this hybrid approach significantly lowers the barrier to entry for sophisticated cyberattacks, as it allows less experienced attackers to leverage the strategic guidance of AI models.

Technical Deep Dive

The research involved the development of autonomous agents capable of maintaining state across multi-step HTTP request sequences.
The agents utilized iterative feedback loops where the AI analyzed HTTP response codes and body content to adjust its payload generation in real-time.
Implementation relied on fine-tuning Large Language Models (LLMs) to understand specific web security protocols and common vulnerability patterns (e.g., SQLi, XSS, IDOR).
The framework employed a 'chain-of-thought' reasoning process to evaluate the success of previous requests before proceeding to the next stage of an exploit chain.

Future ImplicationsAI analysis grounded in cited sources

Automated vulnerability remediation will become a standard requirement for enterprise security.

As AI-human hybrid attacks become more efficient, manual patching cycles will be unable to keep pace with the speed of AI-discovered exploit chains.

The market for AI-powered offensive security tools will face increased regulatory scrutiny.

The dual-use nature of these tools, which can be used for both penetration testing and malicious exploitation, will likely trigger government oversight regarding their distribution.

Timeline

2023-05

James Kettle presents research on AI-assisted web security at major industry conferences.

2024-02

PortSwigger integrates advanced AI features into Burp Suite to assist in automated vulnerability discovery.

2025-11

Kettle publishes findings on the efficacy of human-AI hybrid models in complex exploit chains.

2023-05
James Kettle presents research on AI-assisted web security at major industry conferences.
2024-02
PortSwigger integrates advanced AI features into Burp Suite to assist in automated vulnerability discovery.
2025-11
Kettle publishes findings on the efficacy of human-AI hybrid models in complex exploit chains.

Read original article on Wired AI

Weekly AI Recap

Read this week's curated digest of top AI events →

Post LinkedIn