How gbao261205/multi-agent-pentest-system works
1. What this repo is
- AutoRecon-Agent is a Python-based, multi-agent reconnaissance system for security assessment.
- Its scope is limited to reconnaissance and reporting; it does not implement exploitation.
- The architecture is built around AutoGen-style agent collaboration plus a Docker sandbox for executing network-facing tools safely.
2. Primary runtime boundary
- The key architectural boundary is between the host/orchestrator process and the Docker container.
- The host side coordinates agents and manages chat state; the container side runs tools, scripts, and network probes.
- This separation is enforced by agents/executor.py and the docker/Dockerfile.
3. Main orchestration entry point
- agents/main.py is the system entry point and starts the target-driven workflow.
- The user supplies a target such as a domain or host, then the agent pipeline runs in sequence.
- Control signals move the workflow between phases: DONE_PASSIVE, DONE_ACTIVE, and TERMINATE.
4. Agent collaboration model
- agents/core_agents.py is the coordination layer that instantiates and wires the agents together.
- The design uses a shared chat / group chat style interaction rather than isolated point-to-point calls.
- There are four logical roles:
- Orchestrator_UserProxy: execution bridge, no LLM.
- Passive_Recon_Agent: OSINT and indirect data gathering.
- Active_Recon_Agent: direct scanning and attack-surface discovery.
- Reporter_Agent: synthesis, filtering, and final report generation.
5. Passive reconnaissance subsystem
- Implemented primarily through agents/custom_recon_tools.py and invoked by the passive agent.
- This subsystem collects external intelligence without directly probing the target server.
- Typical outputs include WHOIS, DNS, certificate transparency, leaked data, archive/wayback references, and subdomain clues.
- It is designed to be self-healing: if a dependency is missing, the agent can generate code and install packages inside the sandbox.
6. Active reconnaissance subsystem
- Also centered in agents/custom_recon_tools.py, but used by the active agent.
- This phase performs direct target interaction to map open ports, services, web technologies, TLS state, and hidden paths or endpoints.
- Tools include network scanning, directory enumeration, web fingerprinting, SSL/TLS checks, and JS crawling.
- The active phase produces higher-volume raw results, so long outputs are pushed into files rather than kept only in chat context.
7. Reporting and risk synthesis
- The Reporter_Agent consumes the logs from both recon phases and turns them into a normalized Markdown report.
- Its job is not just summarization; it also filters noise and reduces false positives before assigning risk.
- The final artifact is written as report_<target>.md in the workspace via utilities exposed in agents/utils.py.
8. Log and workspace flow
- Raw tool output is an important intermediate data product in this repo.
- agents/utils.py provides read/write helpers for log persistence and retrieval.
- workspace/logs/ stores large scan outputs, while workspace/reports/ stores the final report files.
- This file-based boundary prevents the LLM context from being overwhelmed by long scan output.
9. Prompting and configuration
- agents/prompts.py contains the system prompts that define each agent’s role and constraints.
- agents/config.py holds API-related configuration such as LLM credentials or model selection.
- These files are part of the control plane: they shape behavior, but do not perform reconnaissance themselves.
10. Tool execution model
- The repository uses a hybrid execution approach:
- predefined Python tools for common recon tasks,
- plus on-the-fly code generation when a case needs a custom workaround.
- agents/executor.py is the execution adapter that runs code or commands in the sandbox and returns stdout/stderr to the chat loop.
- This is the main mechanism that lets agents act autonomously while keeping execution isolated.
11. Containerized toolchain
- docker/Dockerfile defines the runtime image with the required security and network tooling.
- The image is where tools like Nmap, Gobuster, and TLS utilities are expected to run.
- The separate Docker requirements file indicates the container has its own Python dependency set, distinct from the host.
12. Developer and validation surface
- test_tools.py is a focused validation entry point for checking recon tools independently of the full agent flow.
- docs/ contains design notes, progress tracking, and usage references that support development rather than runtime behavior.
- agent_memory.md, context.md, and note.md appear to capture project context and working notes, useful for maintenance but not central to execution.
13. External service dependencies
- The architecture depends on external intelligence or model services in two places:
- LLM providers such as Groq or Gemini for agent reasoning.
- OSINT/network data sources such as CRT logs, Shodan, Whois, DNS, and archive sources for passive recon.
- These services materially affect the data flow because the agents combine local tool output with externally sourced intelligence before reporting.
14. High-level data flow
- User target enters at main.py → orchestrator creates agents → passive agent gathers off-target intelligence → active agent performs direct scanning in Docker → logs are persisted in workspace files → reporter reads raw logs and composes a final Markdown report → termination signal ends the chat and container lifecycle.
15. Architecture diagram takeaway
- If drawing this repo, the most important boxes are:
- Entry/orchestration
- LLM-driven agent layer
- Docker execution sandbox
- Custom recon tools
- Workspace logs/reports
- External OSINT/model services
- The most important arrows are:
- target input to agent chat,
- agent requests to executor,
- executor to Docker/tooling,
- tool outputs to logs,
- logs to reporter,
- final report back to workspace.
Main entry points: core_agents.py Core Agents · main.py Main · prompts.py Prompts · executor.py Executor
Components 16
Host Orchestrator
Python control plane that creates agents, manages chat state, and coordinates workflow phases.
- Main · Starts the target-driven recon workflow and drives phase transitions.
agents/main.py - Core Agents · Creates and wires the collaborative agent roles and shared chat flow.
agents/core_agents.py - Prompts · System prompts that constrain agent behavior and role boundaries.
agents/prompts.py - Config · Model and API configuration for the agent runtime.
agents/config.py - Utils · Read/write helpers for logs and report files.
agents/utils.py - Executor · Runs code and commands in the isolated execution environment and returns output to the chat loop.
agents/executor.py - Recon Tools · Passive and active reconnaissance primitives, including scanning, fingerprinting, and OSINT collection.
agents/custom_recon_tools.py - Passive Agent · Collects indirect intelligence such as DNS, WHOIS, certificates, archives, and subdomain clues.
- Active Agent · Performs direct target probing for ports, services, web tech, TLS, and hidden paths.
- Reporter · Normalizes findings, filters noise, assigns risk, and writes the final Markdown report.
Docker Sandbox
Isolated runtime where recon commands and generated code are executed.
- Dockerfile · Defines the container image used for network-facing tool execution.
docker/Dockerfile
Workspace Data
Persistent files for logs, reports, and project notes.
- Logs · Persistent storage for large recon outputs and intermediate artifacts.
- Reports · Destination for generated Markdown assessment reports.
- Docs · Design notes, progress tracking, tutorials, and usage references.
External Services
LLM providers and OSINT/network data sources used by the agents.
- LLM APIs · Model providers used for agent reasoning and collaboration.
- OSINT Sources · External data sources such as DNS, WHOIS, certificate transparency, archives, and related recon services.
Connections 23
- Main → Core Agents: start workflow · Entry point initializes the collaborative recon session.
- Main → Config: load settings · Reads model and API configuration before agent startup.
- Main → Prompts: load prompts · Loads role prompts that define agent behavior.
- Core Agents → Passive Agent: instantiate · Creates the passive reconnaissance role.
- Core Agents → Active Agent: instantiate · Creates the active reconnaissance role.
- Core Agents → Reporter: instantiate · Creates the reporting and synthesis role.
- Core Agents → Executor: wire executor · Connects agents to the execution bridge.
- Passive Agent → Recon Tools: use tools · Invokes OSINT and indirect recon helpers.
- Active Agent → Recon Tools: use tools · Invokes direct scanning and fingerprinting helpers.
- Recon Tools → Executor: run commands · Requests sandbox execution for probes and generated code.
- Executor → Dockerfile: execute in · Runs tools inside the containerized recon environment.
- Executor → Logs: persist output · Writes large stdout/stderr and scan artifacts to files.
- Reporter → Logs: read logs · Consumes passive and active recon outputs from disk.
- Reporter → Reports: write report · Produces the final Markdown assessment artifact.
- Utils → Logs: read/write · Supports log persistence and retrieval.
- Utils → Reports: write file · Supports creation of the final report file.
- Prompts → Passive Agent: shape behavior · Defines passive role constraints and objectives.
- Prompts → Active Agent: shape behavior · Defines active role constraints and objectives.
- Prompts → Reporter: shape behavior · Defines reporting and filtering constraints.
- Config → LLM APIs: select provider · Configures which model service the agents use.
- Passive Agent → OSINT Sources: query intel · Pulls external reconnaissance intelligence.
- Active Agent → OSINT Sources: probe metadata · May consult external data for enrichment during recon.
- Main → Docs: developer reference · Documentation supports usage and maintenance rather than runtime flow.