How unoplat/unoplat-code-confluence works
Project Type and Purpose
• This is a multi-module, full-stack, code intelligence and context-extraction engine.
• Its goal is to “extract, understand, and provide precise code context” across multiple repositories using techniques such as deterministic code grammar (via ArchGuard and Tree-sitter) and graph-based semantic models.
• The project supports deep language analysis (Python, Java, etc.), dependency parsing, and even embedding-based semantic search.
Main Components (as seen in the file tree)
• Documentation & Website Module:
• “code-confluence” directory contains Docusaurus configuration, docs, sidebars, and website content for the technical documentation and quick-start guides.
• Frontend Application:
• “unoplat-code-confluence-frontend” is a React/TypeScript-based UI (with pages like HomePage, OnboardingPage, etc.) meant for user interaction and visualization.
• CLI/Command-Line Tool:
• “unoplat-code-confluence-cli” provides a command-line interface (with configuration settings and API connectors) for driving tasks or integration into CI/CD pipelines.
• Commons Library:
• “unoplat-code-confluence-commons” encapsulates shared modules (e.g., graph models, changelogs, and common tasks) that are used across ingestion and CLI tools.
• Ingestion and Processing Service:
• “unoplat-code-confluence-ingestion” is the heavy-duty module performing the core work:
▪ It contains various subpackages under “code-confluence-flow-bridge” which include:
– Parsers (for Python, package manager metadata, linters, Tree-sitter based parsing, etc.),
– Processors (handling orchestration for codebase processing, dependency resolution, and graph database integration),
– Connectors (for GitHub and other external APIs),
– Logging and configuration, etc.
• This service is the engine that parses repositories, builds the underlying code graph, and then likely persists it (possibly interfacing with an external graph database).
Relationships and Data Flow
• External Code Repositories & Sources (like GitHub) → Ingestion Service:
• The ingestion service (inside “unoplat-code-confluence-ingestion”) pulls in data (triggered perhaps using GitHub connectors and API clients).
• Data Parsing and Processing:
• Parsed code (using multiple parser strategies and linters) feeds into a graph construction process using shared models from the commons.
• There’s a clear pipeline where “connectors” supply data, “parsers” extract structure, and “processors” apply further business logic (e.g., dependency resolution, summarization, or embedding generation).
• Data Storage & Graph Database:
• Consumption by Frontend and API layers:
• The frontend (React app) likely communicates with backend API endpoints (which might be provided via FastAPI within the ingestion module or a separate API module).
• The CLI tool offers an alternative interface for operations and possibly for triggering ingestion/processing tasks.
Architectural Patterns and Technologies
• Modular Architecture:
• The project uses clear separations of concerns. Each module (documentation, frontend, CLI, ingestion, and commons) is maintained as an independent unit.
• Microservice-Like Segmentation:
• The ingestion service’s separation from the frontend and CLI mimics microservices. They interact via APIs, configuration files, and shared libraries.
• Use of Containers & Deployment Scripts:
• Docker Compose files (local-docker-compose.yml, prod-docker-compose.yml) indicate containerized deployment for local development and production.
• CI/CD Integration:
• .github workflows for various builds and deployments emphasize automation around tests, releases, and documentation builds.
• Technology Stack:
• Frontend: React, TypeScript, Vite, Tailwind CSS.
• Ingestion & CLI: Python-based modules using FastAPI (implied from typical patterns and the presence of pyproject.toml, mypy, ruff, etc.), Tree-sitter for parsing, and graph models.
• Documentation: Docusaurus for the static website.
Elements
• Clear Labels for Each Component:
• API calls between the frontend and backend services.