# sourcebot-dev/sourcebot architecture

> Architecture diagram of the GitHub repository sourcebot-dev/sourcebot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sourcebot-dev/sourcebot
- Repository: https://github.com/sourcebot-dev/sourcebot

## Overview

##### Main Components

a) Frontend Layer (packages/web):
- Next.js web application
- Search interface components
- Repository management UI
- API client interfaces

b) Backend Layer (packages/backend):
- Search service implementation
- Repository indexing system
- VCS integrations (GitHub, GitLab, Gitea, Gerrit)
- Zoekt integration

c) Storage Layer:
- Local repository cache (.sourcebot)
- Zoekt index storage
- Configuration storage

d) External Services:
- Version Control Systems (GitHub, GitLab, Gitea, Gerrit)
- Zoekt search engine
- PostHog analytics (optional)

##### Component Relationships

**[External VCS] <---> [Backend Services] <---> [Zoekt Engine]**
- ^

- v
**[Web Frontend] <---> [API Layer] <---> [Local Storage]**

##### Specific Implementation Details

a) Frontend Components:
- Components like SearchBar, CodePreviewPanel, and FilterPanel
- API client integration
- UI component library

b) Backend Services:
- Repository management system
- Multi-VCS integration handlers
- Indexing service
- Search service implementation
- Configuration management

c) Data Flow:
- Repository cloning and caching
- Index generation and updates
- Search query processing
- Results aggregation and delivery

##### Important Architectural Patterns to Highlight

- Microservices architecture (backend services)
- API-first design
- Event-driven indexing system
- Caching strategy
- Authentication flow for private repositories

##### Additional Elements

Annotations for:
- Configuration points
- Authentication boundaries
- Scaling components
- Monitoring points (PostHog integration)
- Cache invalidation flows

##### Implementation Notes

- Docker containerization
- Volume mounting for local storage
- Port mappings (3000)
- Environment variable configuration
- Periodic index updates (hourly)

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph External["External Services"]
        GitHub["GitHub"]
        GitLab["GitLab"]
        Gitea["Gitea"]
        Gerrit["Gerrit"]
        Zoekt["Zoekt Search Engine"]
        PostHog["PostHog Analytics"]
    end

    subgraph Frontend["Frontend Layer"]
        NextApp["Next.js Application"]:::frontend
        SearchUI["Search Interface"]:::frontend
        RepoUI["Repository Management UI"]:::frontend
        APIClient["API Client"]:::frontend
        UILib["UI Component Library"]:::frontend

        subgraph SearchComponents["Search Components"]
            SearchBar["Search Bar"]:::frontend
            CodePreview["Code Preview Panel"]:::frontend
            FilterPanel["Filter Panel"]:::frontend
            SearchResults["Search Results"]:::frontend
        end

        subgraph RepoComponents["Repository Components"]
            RepoTable["Repository Table"]:::frontend
            RepoCarousel["Repository Carousel"]:::frontend
        end
    end

    subgraph Backend["Backend Layer"]
        MainService["Main Service"]:::backend
        SearchService["Search Service"]:::backend
        ZoektIntegration["Zoekt Integration"]:::backend
        
        subgraph VCSIntegration["VCS Integration"]
            GitHubInt["GitHub Handler"]:::backend
            GitLabInt["GitLab Handler"]:::backend
            GiteaInt["Gitea Handler"]:::backend
            GerritInt["Gerrit Handler"]:::backend
        end

        ConfigMgmt["Configuration Management"]:::backend
        Analytics["Analytics Service"]:::backend
    end

    subgraph Storage["Storage Layer"]
        LocalCache[".sourcebot Cache"]:::storage
        ZoektIndex["Zoekt Index Storage"]:::storage
        ConfigStore["Configuration Storage"]:::storage
        DB["Database"]:::storage
    end

    %% Relationships
    GitHub --> GitHubInt
    GitLab --> GitLabInt
    Gitea --> GiteaInt
    Gerrit --> GerritInt

    GitHubInt & GitLabInt & GiteaInt & GerritInt --> MainService
    MainService --> ZoektIntegration
    ZoektIntegration --> Zoekt
    
    NextApp --> SearchUI & RepoUI
    SearchUI --> SearchComponents
    RepoUI --> RepoComponents
    
    SearchComponents & RepoComponents --> APIClient
    APIClient --> MainService
    
    MainService --> SearchService
    SearchService --> DB
    MainService --> LocalCache
    MainService --> ZoektIndex
    
    Analytics --> PostHog
    ConfigMgmt --> ConfigStore

    %% Styles
    classDef frontend fill:#3498db,stroke:#2980b9,color:white
    classDef backend fill:#2ecc71,stroke:#27ae60,color:white
    classDef storage fill:#9b59b6,stroke:#8e44ad,color:white
    classDef external fill:#f1c40f,stroke:#f39c12,color:black

    %% Click Events
    click NextApp "https://github.com/sourcebot-dev/sourcebot/tree/main/packages/web/src/app"
    click SearchBar "https://github.com/sourcebot-dev/sourcebot/tree/main/packages/web/src/app/components/searchBar"
    click CodePreview "https://github.com/sourcebot-dev/sourcebot/tree/main/packages/web/src/app/search/components/codePreviewPanel"
    click FilterPanel "https://github.com/sourcebot-dev/sourcebot/tree/main/packages/web/src/app/search/components/filterPanel"
    click SearchResults "https://github.com/sourcebot-dev/sourcebot/tree/main/packages/web/src/app/search/components/searchResultsPanel"
    click RepoTable "https://github.com/sourcebot-dev/sourcebot/blob/main/packages/web/src/app/repos/repositoryTable.tsx"
    click RepoCarousel "https://github.com/sourcebot-dev/sourcebot/blob/main/packages/web/src/app/components/repositoryCarousel.tsx"
    click APIClient "https://github.com/sourcebot-dev/sourcebot/blob/main/packages/web/src/app/api/(client)/client.ts"
    click MainService "https://github.com/sourcebot-dev/sourcebot/blob/main/packages/backend/src/main.ts"
    click SearchService "https://github.com/sourcebot-dev/sourcebot/blob/main/packages/web/src/lib/server/searchService.ts"
    click ZoektIntegration "https://github.com/sourcebot-dev/sourcebot/blob/main/packages/backend/src/zoekt.ts"
    click GitHubInt "https://github.com/sourcebot-dev/sourcebot/blob/main/packages/backend/src/github.ts"
    click GitLabInt "https://github.com/sourcebot-dev/sourcebot/blob/main/packages/backend/src/gitlab.ts"
    click GiteaInt "https://github.com/sourcebot-dev/sourcebot/blob/main/packages/backend/src/gitea.ts"
    click GerritInt "https://github.com/sourcebot-dev/sourcebot/blob/main/packages/backend/src/gerrit.ts"
    click ConfigMgmt "https://github.com/sourcebot-dev/sourcebot/blob/main/packages/backend/src/environment.ts"
    click Analytics "https://github.com/sourcebot-dev/sourcebot/blob/main/packages/backend/src/posthog.ts"
    click DB "https://github.com/sourcebot-dev/sourcebot/blob/main/packages/backend/src/db.ts"
    click UILib "https://github.com/sourcebot-dev/sourcebot/tree/main/packages/web/src/components/ui"
```
