Comparing AI Agents to Cybersecurity Professionals in Real-World Penetration Testing
2025 · System or technique · Empirical study / experience
Catalog
Browse catalog
Find AI pentesting tools, security agent harnesses, papers and benchmarks. Filter cybersecurity research by topic, target and contribution.
189 papers.
Relevant papers
PATCHEVAL: A New Benchmark for Evaluating LLMs on Patching Real-World Vulnerabilities
2025 · Benchmark
Cybersecurity AI Benchmark (CAIBench): A Meta-Benchmark for Evaluating Cybersecurity AI Agents
2025 · Benchmark
BlueCodeAgent: A Blue Teaming Agent Powered by Automated Red Teaming for CodeGen AI
2025 · System or technique
HackWorld: Evaluating Computer-Use Agents on Exploiting Web Application Vulnerabilities
2025 · Benchmark
PACEbench: A Framework for Evaluating Practical AI Cyber-Exploitation Capabilities
2025 · Benchmark
AutoPentester: An LLM Agent-based Framework for Automated Pentesting
2025 · System or technique
PentestMCP: A Toolkit for Agentic Penetration Testing
2025 · System or technique
ATLANTIS: AI-driven Threat Localization, Analysis, and Triage Intelligence System
2025 · System or technique
xOffense: An Autonomous Multi-Agent Framework for Penetration Testing with Domain-Adapted Large Language Models
2025 · System or technique
Shell or Nothing: Real-World Benchmarks and Memory-Activated Agents for Automated Penetration Testing
2025 · System or technique · Benchmark · Empirical study / experience
Guided Reasoning in LLM-Driven Penetration Testing Using Structured Attack Trees
2025 · System or technique
All You Need Is A Fuzzing Brain: An LLM-Powered System for Automated Vulnerability Detection and Patching
2025 · System or technique · Benchmark
From CVE Entries to Verifiable Exploits: An Automated Multi-Agent Framework for Reproducing CVEs
2025 · System or technique · Dataset
Multi-Agent Penetration Testing AI for the Web
2025 · System or technique
MultiFuzz: A Dense Retrieval-based Multi-Agent System for Network Protocol Fuzzing
2025 · System or technique
Pentest-R1: Towards Autonomous Penetration Testing Reasoning Optimized via Two-Stage Reinforcement Learning
2025 · System or technique
Towards Effective Offensive Security LLM Agents: Hyperparameter Tuning, LLM as a Judge, and a Lightweight CTF Benchmark
2025 · Empirical study / experience · Benchmark
ASTRA: Autonomous Spatial-Temporal Red-teaming for AI Software Assistants
2025 · System or technique
Cyber-Zero: Training Cybersecurity Agents without Runtime
2025 · System or technique
Running in CIRCLE? A Simple Benchmark for LLM Code Interpreter Security
2025 · Benchmark
FaultLine: Automated Proof-of-Vulnerability Generation Using LLM Agents
2025 · System or technique
AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models
2025 · Benchmark
SEC-bench: Automated Benchmarking of LLM Agents on Real-World Software Security Tasks
2025 · System or technique · Benchmark · Dataset
PoCGen: Generating Proof-of-Concept Exploits for Vulnerabilities in Npm Packages
2025 · System or technique