How jhonattang/mcp-qa works
1. What this repository is
- mcp-qa is a Python 3.12 automation agent for QA workflows.
- Its core shape is: a CLI front end, a LangGraph-based orchestrator, several MCP tool servers, and a few local support services like Redis and optional Ollama.
2. Primary control flow
- The user starts in src/console.
- The console sends a request into src/orchestrator, which runs a stateful pipeline.
- The orchestrator coordinates LLM calls, cache lookup, MCP tool invocations, and final report delivery.
3. Orchestration layer
- src/orchestrator/graph.py defines the main workflow as a state machine.
- The pipeline is split into four major stages: story extraction, Jira creation, API test execution, and report upload.
- src/orchestrator/nodes/ holds the stage-specific actions, while src/orchestrator/state.py defines the shared state passed through the graph.
4. LLM and caching boundary
- src/llm/factory.py abstracts model selection between local and cloud LLM backends.
- src/cache/cag_manager.py provides the cache layer used to reduce repeated reasoning and token usage.
- From an architecture perspective, this is the intelligence layer: it sits between the orchestrator and the external tools.
5. MCP server boundary
- All external integrations are isolated under mcp_servers/.
- Each subfolder is a separate MCP server with its own server.py, tools.py, and usually its own config.json and dependencies.
- This keeps external-service logic out of the orchestrator and makes each connector independently deployable.
6. Drive, Jira, and Git integration
- mcp_servers/drive handles Google Drive access and file ingestion, especially planning transcripts.
- mcp_servers/jira creates issues and attaches artifacts in Jira Cloud.
- mcp_servers/git clones and inspects repositories, including README parsing, which is used as the source of truth for API test discovery.
7. POS API test execution subsystem
- mcp_servers/pos_api is the execution engine for API validation.
- It contains the launcher and runner used to bring up the target API, discover endpoints, and execute “happy path” HTTP tests.
- This is the part of the system that bridges the agent to the application under test, often against a locally running API rather than a hosted service.
8. Reporting subsystem
- mcp_servers/reporting generates structured QA output.
- It renders Markdown through Jinja templates and can produce PDF reports.
- The reporting stage turns raw test results into an artifact suitable for attachment back into Jira.
9. Workspace/local filesystem tools
- mcp_servers/workspace provides local machine utilities such as listing repositories and reports, plus audit-related helpers.
- This server is the agent’s controlled access point for local file-system inspection and artifact discovery.
10. Configuration and secrets
- config/settings.py and .env.example centralize environment-driven configuration.
- docs/credentials_guide.md and the per-server configs indicate that credentials are managed per integration rather than globally embedded in code.
- docker-compose.yml and the Dockerfile support the agent runtime and auxiliary services.
11. Data models and validation
- src/schemas/ defines typed structures for user stories, test cases, and test results.
- These models are the contract between orchestration, LLM output, test execution, and reporting.
- They reduce ambiguity when moving data across stages and across MCP boundaries.
12. Utilities and reliability
- src/utils/logging_config.py standardizes logs.
- src/utils/retry.py provides retry behavior for transient failures, which matters for networked test execution and external APIs.
- scripts/health_check.py suggests a lightweight operational check path for the runtime environment.
13. Testing and quality gates
- tests/unit covers core logic such as the cache, router, and schemas.
- tests/integration exists for cross-component behavior, which is important in a system that depends on MCP servers and external services.
- The repo is built to validate both pure Python behavior and orchestration-driven workflows.
14. Overall architecture in one sentence
- The system is a modular QA agent where a CLI-driven LangGraph orchestrator uses LLMs plus Redis-backed cache to coordinate specialized MCP servers for Drive, Jira, Git, API testing, reporting, and local workspace access.
Main entry points: graph.py Workflow · test_router.py Tests · extract_stories.py Extract stories · run_api_tests.py Run tests
Components 22
Core app
CLI, orchestration, LLM, cache, schemas, and shared utilities.
- Console · Command-line front end that starts the QA workflow.
src/console/__main__.py - Router · Dispatches console requests into the application flow.
src/console/router.py - App · Console application wiring and prompt handling.
src/console/app.py - Workflow · Main orchestrator graph that drives the QA pipeline.
src/orchestrator/graph.py - State · State model carried across orchestration stages.
src/orchestrator/state.py - Extract stories · Stage that turns input material into user stories.
src/orchestrator/nodes/extract_stories.py - Create Jira · Stage that creates Jira issues and attaches artifacts.
src/orchestrator/nodes/create_jira.py - Run tests · Stage that executes API validation against the target service.
src/orchestrator/nodes/run_api_tests.py - Upload report · Stage that publishes final QA output and attachments.
src/orchestrator/nodes/upload_report.py - LLM factory · Selects between local and cloud model backends.
src/llm/factory.py - Cache · Cache layer used to reduce repeated reasoning and token usage.
src/cache/cag_manager.py - Schemas · Typed contracts for stories, test cases, and results.
src/schemas/user_story.py
MCP servers
Isolated tool servers for external systems and local execution.
- Drive server · Google Drive integration for planning transcripts and ingestion.
mcp_servers/drive/server.py - Git server · Repository clone and README parsing support for test discovery.
mcp_servers/git/server.py - Jira server · Creates issues and manages Jira Cloud attachments.
mcp_servers/jira/server.py - POS API · Launches and runs API validation against the application under test.
mcp_servers/pos_api/server.py - Reporting · Generates structured Markdown and PDF QA reports.
mcp_servers/reporting/server.py - Workspace · Controlled local filesystem access for repos, reports, and audit helpers.
mcp_servers/workspace/server.py
Runtime config
Environment, container, and operational support files.
- Settings · Environment-driven configuration and secrets wiring.
config/settings.py - Compose · Container and service definitions for the agent runtime.
docker-compose.yml - Health check · Lightweight operational check for the runtime environment.
scripts/health_check.py
Quality
Tests and validation-focused assets.
- Tests · Unit and integration coverage for core orchestration behavior.
tests/unit/test_router.py
Connections 24
- Console → Router: dispatches to · Console hands requests to routing logic.
- Router → App: builds · Router connects into the console app layer.
- App → Workflow: starts · App invokes the orchestrator workflow.
- Workflow → State: uses state · Workflow shares structured state across stages.
- Workflow → LLM factory: calls · Orchestrator requests model-backed reasoning.
- Workflow → Cache: checks cache · Workflow consults cached reasoning before recomputing.
- Workflow → Extract stories: stage · First major pipeline stage.
- Workflow → Create Jira: stage · Second major pipeline stage.
- Workflow → Run tests: stage · Third major pipeline stage.
- Workflow → Upload report: stage · Final pipeline stage.
- Extract stories → Drive server: reads input · Story extraction can ingest planning material from Drive.
- Extract stories → Git server: discovers source · Extracted stories can be informed by repository README parsing.
- Create Jira → Jira server: writes issues · Issue creation is delegated to the Jira MCP server.
- Run tests → POS API: executes · API validation is delegated to the POS API MCP server.
- Run tests → Git server: inspects repo · Test execution uses repository discovery inputs.
- Upload report → Reporting: renders report · Reporting server produces the final QA artifact.
- Upload report → Jira server: attaches artifact · Final report is sent back to Jira.
- Workflow → Workspace: uses workspace · Workflow may inspect local repositories and reports.
- Settings → LLM factory: configures · Settings influence model backend selection.
- Settings → Compose: drives · Runtime configuration supports containerized deployment.
- Health check → Workspace: checks · Health check validates local runtime and file access.
- Tests → Cache: covers · Tests exercise cache behavior.
- Tests → Router: covers · Tests exercise CLI routing.
- Tests → Schemas: covers · Tests validate data contracts.