How to Build an AI Agent for Pentesting Without Turning It Into a Black Box
Most security specialists begin using language models as advanced reference tools. A person describes the task, receives a hypothesis, command or code snippet, verifies the answer and independently decides the next action. In this mode the boundary of responsibility remains clear: the model proposes while the specialist evaluates and executes. Even if the response proves incorrect, it is usually possible to reconstruct the workflow and identify the exact step where the error appeared.
With an AI agent the scheme changes. The agent receives a goal, plans several steps, invokes tools, analyzes their output, stores results in memory and continues work with the new context. This approach automates not only individual commands but entire sequences of actions. However, together with speed comes risk: the specialist sees only the final answer and may not understand which hypotheses were tested, which commands were executed and why the agent selected that particular path. At this point the useful assistant turns into a black box.
Agent is not a long prompt
An ordinary LLM answers a request. An agent must independently repeat the working cycle: understand the current state of the task, choose the next step, call the appropriate tool, interpret the result and adjust the plan. It is precisely the ability to act across multiple stages that distinguishes an agent from a chat in which every continuation is initiated by a human.
Define boundaries first, then connect tools
The formulation “find vulnerabilities” is too broad. It does not clarify which targets may be examined, which methods are permitted, when to stop and what constitutes confirmed evidence. A human can restore missing context from a contract, program rules or personal experience. An agent sees only the data and instructions provided to it.
Before launch it is necessary to describe the operational contour:
- goal and expected result;
- permitted nodes, applications and types of checks;
- actions that must not be performed;
- available tools and limits of their use;
- criteria for confirming a discovered issue;
- conditions for stopping;
- cases when human consultation is required;
- format of the log and final report.
Restrictions should be technical rather than purely textual. A prohibition in the system instruction is useful but does not by itself block command execution. The most reliable approach combines instructions with mechanical controls that limit commands, parameters, directories, addresses and available tools at the execution environment level.
Build a minimal working cycle
The first prototype should not attempt to do everything. The more tools and roles appear simultaneously, the harder it becomes to determine the cause of failure. For the start it is enough to implement a short, easily observable and repeatable process that includes obtaining a concrete technical task, formulating one testable hypothesis, selecting an allowed tool, proposing launch parameters, obtaining confirmation for critical actions, executing the check, saving source data and result, explaining what changed in the understanding of the task and proposing the next step.
Do not force the model to reinvent the process
If the agent must be told the working procedure from scratch every time, results will vary more widely and configuration will take longer. Repeatable instructions are best packaged as separate skills that describe the purpose of a tool, allowable parameters, verification sequence, signs of successful outcome and typical errors.
Record the path, not only the final answer
The final report does not reveal the quality of the process. An agent may reach the correct result after systematic hypothesis testing or may obtain it accidentally after dozens of irrelevant actions. From the outside both variants look identical. To distinguish methodology from coincidence it is necessary to preserve the trajectory of work, including the original goal and constraints, the agent’s hypotheses, chosen tools, parameters sent, raw command output, result interpretation, reasons for plan changes, confirmation requests, manual interventions by the specialist and final evidence.
Limit actions technically
The most dangerous configuration is an agent with broad rights and a vague goal. If it has access to a shell, file system, network tools and credentials, a textual prohibition is insufficient. A safe contour should include a separate agent account, minimal file and service rights, an allowlist of tools and operations, parameter validation before execution, prohibition of destructive commands, an isolated execution environment, request-rate limits, confirmation of critical operations and complete logging of calls.
Test refusal as thoroughly as success
A demonstration in which the agent finds a flag confirms only one scenario. Before independent operation the agent must be tested in conditions where the correct result is to stop. A minimal set of negative tests should include attempts to leave the permitted scope, requests for a forbidden tool, access to an unavailable file or secret, malicious instructions in external content, attempts to bypass operation confirmation, repetition of one unsuccessful action in a loop, contradictions between the original goal and new data, and generation of a report without sufficient evidence.
Leave high-error-cost decisions to the specialist
Human-in-the-loop does not mean the human manually controls every command. The specialist’s role is to set boundaries and make decisions where automatic choice is insufficiently reliable. The specialist must intervene if the agent loses the goal, begins to repeat itself, cannot explain the next step, proposes expanding the scope or draws a conclusion that cannot be confirmed by saved data.
AI Pentesting Challenge: from agent assembly to results analysis
CyberED and Standoff Hackbase are conducting a practical AI pentesting challenge. Participation requires basic understanding of penetration testing, web vulnerabilities and command-line work. On 10 September at 19:30 MSK an opening webinar will be held. An expert will assemble a minimal agent live and bring it to launch on the training range. From 10 to 17 September participants will asynchronously configure and improve their agents, search for and exploit vulnerabilities in dynamic tasks on the shared Standoff Hackbase range and monitor the public ranking. On 17 September the expert will demonstrate their own run with an agent, analyze working strategies, dead-end approaches and typical agent-management errors, then summarize the challenge results.
Related articles
Agent-Ops 0.4.0 Released: Methodology for Secure Human-AI Collaboration in IT Operations
Sergey Zhitinsky, founder of Git in Sky, has published the public normative candidate for Agent-Ops 0.4.0, an open industry methodology governing how engineers and AI agents jointly handle IT infrastructure tasks. The framework keeps humans firmly in the decision-making loop while using deterministic programs for data collection and approved changes. It addresses risks such as prompt injection through processed data, unverified model outputs, and unclear accountability when AI recommendations lead to incidents. The methodology divides work across eight explicit steps and three separate planes: data, governance, and independent verification performed by a Guardian role. Two additional companies have joined as maintainers following agreements at the IT Elements 2026 conference, turning the project into a multi-organization effort. Contributors are invited to help refine contracts, schemas, and operational scenarios through GitHub and GitVerse.
ProxyKey MCP: Securing API Access for AI Agents Without Exposing Credentials
ProxyKey has released an MCP server that allows AI coding agents such as Claude Code and Cursor to manage API credentials without ever reading the actual secret values. The solution addresses the risk that any key visible to an agent becomes compromised through logging, tracing, or prompt injection. Real provider keys are stored encrypted with AES-256-GCM and never returned by any API endpoint after initial entry. Agents instead receive limited virtual passes that support IP binding, rate limits, TTL, and detailed request logging. A pending-secret workflow lets agents prepare services before the real token exists, with the human entering the secret only through a web panel. The approach deliberately restricts the MCP tool contract so no operation can read or return secret values.
Shadow AI in CI/CD: Why AI Agents Must Be Modeled as Security Threats
A new analysis from the CNCF highlights the growing risks of Shadow AI within continuous integration and continuous deployment pipelines. The report argues that AI agents should be treated as potential threats rather than simple productivity tools. Starting from a developer's laptop and extending to Kubernetes clusters, these agents can introduce unauthorized access paths and data exposure risks. Security teams are urged to incorporate AI agent behavior into formal threat modeling exercises. The discussion emphasizes the need for visibility and control over autonomous AI components operating in production environments.
Detecting Lateral Movement with Neural Networks Trained Solely on Synthetic Data
A researcher generated entire corporate network histories using a 135-line configuration file to create synthetic authentication logs containing lateral movement attacks. Neural networks trained exclusively on these artificial datasets were then evaluated against 1.65 billion real authentication events from Los Alamos National Laboratory, including 749 red team events across 301 compromised machines. The best ensemble of six models flagged 3.6 million hourly machine windows and placed 16 genuine attacks among the top 23 highest-scoring entries, producing only seven false positives. In comparison, a simple threshold counter required 161,000 false alarms to reach the same detection level. The approach also demonstrated an iterative feedback loop where detector errors directly informed refinements to the synthetic world generator. The work shows that synthetic data can reach AUC performance comparable to models trained on real labeled attacks while providing full control over the underlying attack definitions.