# cyberagiinc/devdocs architecture

> Architecture diagram of the GitHub repository cyberagiinc/devdocs, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cyberagiinc/devdocs
- Repository: https://github.com/cyberagiinc/devdocs

## Overview

##### Main Components
- Frontend Service: Represented by the “app” directory with Next.js pages (e.g., layout.tsx, page.tsx) and other UI components in the “components” folder. This is the user interface (accessible on http://localhost:3001).
- Backend API: Found in the “backend” folder (e.g., backend/app/main.py, crawler.py, config.py) and “app/api” routes. This provides the RESTful endpoints for handling API requests, managing crawl requests, and facilitating communication with other services. It listens on http://localhost:24125.
- Crawl4AI Service: Its scripts and API are part of the project structure (e.g., test_crawl4ai.py, various crawl debug scripts) and has its specific endpoints for web crawling (running on http://localhost:11235).
- MCP Server/Integration: Dockerfiles such as Dockerfile.mcp and integration scripts (e.g., restart_and_test_mcp.sh, check_mcp_health.sh) suggest that there is an MCP server component that works with both the backend and the LLM (Claude integration per the README).
- Fast-Markdown-MCP Module: Located in the “fast-markdown-mcp” directory, this lightweight Python project is used for processing markdown and document structure, offering fast conversion and possibly serving the MCP protocols.
- Docker & Deployment Configurations: Multiple Dockerfiles (for backend, frontend, mcp) and docker-compose.yml files are used for container orchestration. These should be shown as the deployment environment that encapsulates each service.

##### Understand Relationships and Interactions
- The Frontend interacts with the Backend API (e.g., via API endpoints under “app/api”) using RESTful calls.
- The Backend communicates with the Crawl4AI service to trigger web crawling, manage crawl results (see backend/app/crawler.py and scripts under scripts/crawl), and process content.
- The MCP Server (and related fast-markdown-mcp component) provides an interface for LLM-ready data or documentation processing pipelines, interacting with both the backend and external LLM clients.
- Docker containers ensure that these modules (Frontend, Backend, MCP, Crawl4AI) run in isolation while being networked together.
- Utility scripts and configuration files (like next.config.mjs, tailwind.config.ts, postcss.config.mjs) support the building and maintenance of the different components.

##### Architectural Patterns and Technologies
- Microservice-style separation via Docker containers with clearly delineated responsibilities.
- Frontend built with Next.js (React-based) using components and hooks (see “components/ui/”, “hooks/”).
- Backend built in Python (with Flask/FastAPI hints in backend/app/main.py, though specifics might require a closer inspection).
- Integration with MCP and intelligent crawling indicates a pipeline-based processing architecture.
- Utilization of Docker Compose to orchestrate the multiple services and scripts that manage health checks and start/stop operations.
- Clear separation between static assets, configuration files, and service-specific code.

##### Instructing the Principal Software Engineer
- Separate sections for:
- Frontend UI (showing “app”, “components”, and UI elements) and mark its port (3001).
- Backend API (from “backend” and “app/api”) and note its responsibilities (API, authentication, and orchestration), including its port (24125).
- Crawl4AI service (delineated by associated scripts and tests) and its external endpoint (port 11235).
- MCP Server, including its Docker container (Dockerfile.mcp) and the fast-markdown-mcp module for content format conversion.
- Docker orchestration: Represent docker-compose and the startup scripts to show how containers are linked.
- Frontend → Backend (for API calls).
- Backend → Crawl4AI (for crawling and fetching documentation).
- Backend/Crawl4AI → MCP (data processing, LLM integration).

##### Element-Specific Inclusions
- Clearly labeled components with names (Frontend/UI, Backend/API, Crawl4AI, MCP Server, Fast-Markdown-MCP).
- Request and response cycles between UI and API.
- Data processing pipelines from crawler output to MCP for documentation formatting.
- User-facing elements (Frontend UI) vs. internal processing services (Backend, CRO).
- Infrastructure components (Docker orchestration).

This level of detail is essential to assuring accuracy in documenting the architecture for enterprise deployments and rapid development cycles. Adjust any component details based on further insights if more specifics about frameworks or inter-service communication protocols become evident during deeper analysis or code review.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Frontend UI"
        FE1("App Pages (layout.tsx, page.tsx, http://localhost:3001)"):::frontend
        FE2("UI Components (components/)"):::frontend
        FE3("Frontend Configs (next.config.mjs, tailwind.config.ts, postcss.config.mjs)"):::frontend
    end

    subgraph "Backend API"
        BE1("Backend App (main.py, crawler.py, config.py, http://localhost:24125)"):::backend
        BE2("API Routes (app/api)"):::backend
    end

    subgraph "Crawl4AI Service"
        CR1("Crawl Main Script (test_crawl4ai.py, http://localhost:11235)"):::crawl
        CR2("Crawl Debug Script (debug_crawl4ai.sh)"):::crawl
        CR3("Crawl Health Script (check_crawl4ai.sh)"):::crawl
    end

    subgraph "MCP Server/Integration"
        MCP1("MCP Docker Config (Dockerfile.mcp)"):::mcp
        MCP2("MCP Scripts (restart_and_test_mcp.sh, check_mcp_health.sh)"):::mcp
        MCP3("MCP Config (claude_mcp_settings.json)"):::mcp
        MCP4("Fast-Markdown-MCP Module"):::mcp
    end

    subgraph "Docker & Deployment"
        DO1("Backend Docker (Dockerfile.backend)"):::docker
        DO2("Frontend Docker (Dockerfile.frontend)"):::docker
        DO3("Compose File (docker-compose.yml)"):::docker
        DO4("Docker Scripts (docker-start.sh)"):::docker
    end

    %% Inter-Service Connections
    FE1 -->|"REST"| BE1
    BE1 -->|"CrawlRequest"| CR1
    BE1 -->|"ProcessData"| MCP1
    CR2 -->|"SendResults"| MCP1

    %% Docker orchestration linking services
    DO3 --- FE1
    DO3 --- BE1
    DO3 --- CR1
    DO3 --- MCP1

    %% Click Events for Frontend Service
    click FE1 "https://github.com/cyberagiinc/devdocs/blob/main/app/app/layout.tsx"
    click FE2 "https://github.com/cyberagiinc/devdocs/tree/main/components/"
    click FE3 "https://github.com/cyberagiinc/devdocs/blob/main/next.config.mjs"

    %% Click Events for Backend API
    click BE1 "https://github.com/cyberagiinc/devdocs/blob/main/backend/app/main.py"
    click BE2 "https://github.com/cyberagiinc/devdocs/tree/main/app/api"

    %% Click Events for Crawl4AI Service
    click CR1 "https://github.com/cyberagiinc/devdocs/blob/main/backend/test_crawl4ai.py"
    click CR2 "https://github.com/cyberagiinc/devdocs/blob/main/scripts/general/debug_crawl4ai.sh"
    click CR3 "https://github.com/cyberagiinc/devdocs/blob/main/scripts/general/check_crawl4ai.sh"

    %% Click Events for MCP Server/Integration
    click MCP1 "https://github.com/cyberagiinc/devdocs/blob/main/Dockerfile.mcp"
    click MCP2 "https://github.com/cyberagiinc/devdocs/blob/main/scripts/mcp/restart_and_test_mcp.sh"
    click MCP3 "https://github.com/cyberagiinc/devdocs/blob/main/claude_mcp_settings.json"
    click MCP4 "https://github.com/cyberagiinc/devdocs/blob/main/fast-markdown-mcp/pyproject.toml"

    %% Click Events for Docker & Deployment
    click DO1 "https://github.com/cyberagiinc/devdocs/blob/main/Dockerfile.backend"
    click DO2 "https://github.com/cyberagiinc/devdocs/blob/main/Dockerfile.frontend"
    click DO3 "https://github.com/cyberagiinc/devdocs/blob/main/docker-compose.yml"
    click DO4 "https://github.com/cyberagiinc/devdocs/blob/main/scripts/docker/docker-start.sh"

    %% Class Definitions for Coloring
    classDef frontend fill:#bfd7ff,stroke:#1b53e6,stroke-width:2px;
    classDef backend fill:#c5f7c7,stroke:#2e8b57,stroke-width:2px;
    classDef crawl fill:#ffe1b3,stroke:#f08b33,stroke-width:2px;
    classDef mcp fill:#e0b3ff,stroke:#8a2be2,stroke-width:2px;
    classDef docker fill:#d3d3d3,stroke:#696969,stroke-width:2px;
```
