CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale
2025 · Benchmark · Dataset
Catalog
Browse catalog
Find AI pentesting tools, security agent harnesses, papers and benchmarks. Filter cybersecurity research by topic, target and contribution.
189 papers.
Relevant papers
Improving LLM Agents with Reinforcement Learning on Cryptographic CTF Challenges
2025 · Dataset · System or technique
Evaluating AI cyber capabilities with crowdsourced elicitation
2025 · Empirical study / experience
Dynamic Risk Assessments for Offensive Cybersecurity Agents
2025 · Empirical study / experience
BountyBench: Dollar Impact of AI Agent Attackers and Defenders on Real-World Cybersecurity Systems
2025 · Benchmark
CRAKEN: Cybersecurity LLM Agent with Knowledge-Based Execution
2025 · System or technique
ACSE-Eval: Can LLMs threat model real-world cloud infrastructure?
2025 · Benchmark
AutoPentest: Enhancing Vulnerability Management With Autonomous LLM Agents
2025 · System or technique
RedTeamLLM: an Agentic AI framework for offensive security
2025 · System or technique
CAI: An Open, Bug Bounty-Ready Cybersecurity AI
2025 · System or technique
CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities
2025 · System or technique · Benchmark
D-CIPHER: Dynamic Collaborative Intelligent Multi-Agent System with Planner and Heterogeneous Executors for Offensive Security
2025 · System or technique
Can LLMs Hack Enterprise Networks? Autonomous Assumed Breach Penetration-Testing Active Directory Networks
2025 · System or technique · Empirical study / experience
Incalmo: An Autonomous LLM-assisted System for Red Teaming Multi-Host Networks
2025 · System or technique · Benchmark
VulnBot: Autonomous Penetration Testing for A Multi-Agent Collaborative Framework
2025 · System or technique
AutoPenBench: A Vulnerability Testing Benchmark for Generative Agents
2025 · System or technique · Benchmark
Mind the Gap: Towards Generalizable Autonomous Penetration Testing via Domain Randomization and Meta-Reinforcement Learning
2024 · System or technique
Hacking CTFs with Plain Agents
2024 · Empirical study / experience
HackSynth: LLM Agent and Evaluation Framework for Autonomous Penetration Testing
2024 · System or technique · Benchmark
CKGFuzzer: LLM-Based Fuzz Driver Generation Enhanced By Code Knowledge Graph
2024 · System or technique
PentestAgent: Incorporating LLM Agents to Automated Penetration Testing
2024 · System or technique
AutoPT: How Far Are We from the End2End Automated Web Penetration Testing?
2024 · System or technique
Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilities
2024 · Benchmark
EnIGMA: Interactive Tools Substantially Assist LM Agents in Finding Security Vulnerabilities
2024 · System or technique · Dataset
Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models
2024 · System or technique · Benchmark · Dataset · Empirical study / experience