How cyberagiinc/devdocs works
Main Components
• Frontend Service: Represented by the “app” directory with Next.js pages (e.g., layout.tsx, page.tsx) and other UI components in the “components” folder. This is the user interface (accessible on http://localhost:3001).
• Backend API: Found in the “backend” folder (e.g., backend/app/main.py, crawler.py, config.py) and “app/api” routes. This provides the RESTful endpoints for handling API requests, managing crawl requests, and facilitating communication with other services. It listens on http://localhost:24125.
• Crawl4AI Service: Its scripts and API are part of the project structure (e.g., test_crawl4ai.py, various crawl debug scripts) and has its specific endpoints for web crawling (running on http://localhost:11235).
• MCP Server/Integration: Dockerfiles such as Dockerfile.mcp and integration scripts (e.g., restart_and_test_mcp.sh, check_mcp_health.sh) suggest that there is an MCP server component that works with both the backend and the LLM (Claude integration per the README).
• Fast-Markdown-MCP Module: Located in the “fast-markdown-mcp” directory, this lightweight Python project is used for processing markdown and document structure, offering fast conversion and possibly serving the MCP protocols.
• Docker & Deployment Configurations: Multiple Dockerfiles (for backend, frontend, mcp) and docker-compose.yml files are used for container orchestration. These should be shown as the deployment environment that encapsulates each service.
Understand Relationships and Interactions
• The Frontend interacts with the Backend API (e.g., via API endpoints under “app/api”) using RESTful calls.
• The Backend communicates with the Crawl4AI service to trigger web crawling, manage crawl results (see backend/app/crawler.py and scripts under scripts/crawl), and process content.
• The MCP Server (and related fast-markdown-mcp component) provides an interface for LLM-ready data or documentation processing pipelines, interacting with both the backend and external LLM clients.
• Docker containers ensure that these modules (Frontend, Backend, MCP, Crawl4AI) run in isolation while being networked together.
• Utility scripts and configuration files (like next.config.mjs, tailwind.config.ts, postcss.config.mjs) support the building and maintenance of the different components.
Architectural Patterns and Technologies
• Microservice-style separation via Docker containers with clearly delineated responsibilities.
• Frontend built with Next.js (React-based) using components and hooks (see “components/ui/”, “hooks/”).
• Backend built in Python (with Flask/FastAPI hints in backend/app/main.py, though specifics might require a closer inspection).
• Integration with MCP and intelligent crawling indicates a pipeline-based processing architecture.
• Utilization of Docker Compose to orchestrate the multiple services and scripts that manage health checks and start/stop operations.
• Clear separation between static assets, configuration files, and service-specific code.
Instructing the Principal Software Engineer
• Separate sections for:
• Frontend UI (showing “app”, “components”, and UI elements) and mark its port (3001).
• Backend API (from “backend” and “app/api”) and note its responsibilities (API, authentication, and orchestration), including its port (24125).
• Crawl4AI service (delineated by associated scripts and tests) and its external endpoint (port 11235).
• MCP Server, including its Docker container (Dockerfile.mcp) and the fast-markdown-mcp module for content format conversion.
• Docker orchestration: Represent docker-compose and the startup scripts to show how containers are linked.
• Frontend → Backend (for API calls).
• Backend → Crawl4AI (for crawling and fetching documentation).
• Backend/Crawl4AI → MCP (data processing, LLM integration).
Element-Specific Inclusions
• Clearly labeled components with names (Frontend/UI, Backend/API, Crawl4AI, MCP Server, Fast-Markdown-MCP).
• Request and response cycles between UI and API.
• Data processing pipelines from crawler output to MCP for documentation formatting.
• User-facing elements (Frontend UI) vs. internal processing services (Backend, CRO).
• Infrastructure components (Docker orchestration).
This level of detail is essential to assuring accuracy in documenting the architecture for enterprise deployments and rapid development cycles. Adjust any component details based on further insights if more specifics about frameworks or inter-service communication protocols become evident during deeper analysis or code review.