---
title: "Hacking CTFs with Plain Agents (arXiv:2412.02776) | Papers With Code Cyber"
description: "We saturate a high-school-level hacking benchmark with plain LLM agent design. Concretely, we obtain 95% performance on InterCode-CTF, a popular offensive security…"
canonical: "https://paperswithcode-cyber.com/paper/2412.02776"
type: "paper"
---

# Hacking CTFs with Plain Agents

arXiv:2412\.02776 · 2024-12-03

Authors: Rustem Turtayev, Artem Petrov, Dmitrii Volkov, Denis Volk

## Abstract

We saturate a high\-school\-level hacking benchmark with plain LLM agent design\. Concretely, we obtain 95% performance on InterCode\-CTF, a popular offensive security benchmark, using prompting, tool use, and multiple attempts\. This beats prior work by Phuong et al\. 2024 \(29%\) and Abramovich et al\. 2024 \(72%\)\. Our results suggest that current LLMs have surpassed the high school level in offensive cybersecurity\. Their hacking capabilities remain underelicited: our ReAct&Plan prompting strategy solves many challenges in 1\-2 turns without complex engineering or advanced harnessing\.

## Tasks

- [Penetration testing](https://paperswithcode-cyber.com/tasks/cyberintel-penetration-testing)

## Code repositories

- [PalisadeResearch/intercode](https://github.com/PalisadeResearch/intercode)

## Paper resources

- [Abstract page](https://arxiv.org/abs/2412.02776v1)
- [PDF](https://arxiv.org/pdf/2412.02776v1.pdf)