# ishaan1013/shadow architecture

> Architecture diagram of the GitHub repository ishaan1013/shadow, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ishaan1013/shadow
- Repository: https://github.com/ishaan1013/shadow

## Overview

##### Project type and purpose
- Shadow is a full-stack, open-source “background coding agent” platform.
- It consists of:
- A Next.js frontend (real-time chat, file explorer, terminal emulator)
- A Node.js orchestrator backend (task management, LLM integration, WebSocket, API)
- A Sidecar service (isolated file/command execution in containers)
- A PostgreSQL database (via Prisma)
- Auxiliary packages (shared types, DB client, command-security, ESLint/TS configs)

a. Client Layer
- Developer’s browser (UI)
- Next.js Frontend (apps/frontend)
– HTTP-REST endpoints (`/api/...`)
– WebSocket connection for real-time task updates
b. Orchestrator Layer (Server)
- API Gateway & Middleware
- WebSocket Server (task socket)
- Task Manager & Workflow Engine
- Agent Layer (LLM integration, “braintrust,” PR generation)
- Execution Abstraction
– LocalMode: Local-FS Git service & tool executor
– RemoteMode: Kubernetes orchestration → Kata containers
- Indexing Service
– Code chunker, semantic embedding, Pinecone vector store
– Shadow Wiki generator
- GitHub Integration (OAuth, repo/branch/issues API)
- Database Services (Prisma client, task records, user settings, memories)
- Auxiliary Services (memory cleanup, checkpoint, pr-manager, summarization)
c. Sidecar Layer (apps/sidecar)
- REST APIs for file ops, git ops, terminal, workspace manager, filesystem watcher
- Runs inside VM/container, invoked by Server’s remote execution
d. Data & External Services
- PostgreSQL (packages/db)
- Vector DB (Pinecone) for semantic search
- LLM Providers (OpenAI, Anthropic, OpenRouter)
- GitHub API

##### Relationships & data flows
- Browser ⇄ Frontend via HTTPS & WebSocket
- Frontend → Server HTTP API (task creation, GitHub queries, user settings)
- Frontend ←→ Server WebSocket (streaming chat, task status)
- Server ⇄ PostgreSQL (Prisma ORM)
- Server → Pinecone (semantic embeddings)
- Server → GitHub (REST API calls, webhooks)
- Server ↔ Sidecar (HTTPS REST for file/git/exec when in remote mode)
- Server → Kubernetes API (create/teardown Kata containers)
- Server → LLM Providers (chat completions, streaming responses)

##### Architectural patterns & principles
- Layered architecture: UI, API/Gateway, Business logic, Data layer
- Abstraction of execution modes (Strategy pattern for local vs remote)
- Micro-service-style Sidecar for isolation
- Event-driven streaming via WebSockets
- Modular packages for shared types, security, and DB client
- Kubernetes + Kata containers for hardware-level isolation

##### Detailed elements
- Frontend: Next.js, React components, API routes, Socket client
- Server: Express/Koa (or custom) HTTP server, WebSocket server, Prisma client, agent modules, indexing modules, execution modules
- Sidecar: Express REST API, file/git/terminal services, workspace manager
- Data stores: PostgreSQL, Pinecone
- External: GitHub API, LLM endpoints
- Deployment: Dockerfiles for `apps/server` and `apps/sidecar`, Kubernetes YAML for remote agents

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Client Layer"
        DeveloperBrowser["Developer Browser"]:::client
        NextJSFrontend["Next.js Frontend"]:::client
        APIRoutes["API Routes (/api)"]:::client
        WebSocketClient["WebSocket Client"]:::client
    end

    DeveloperBrowser -->|"HTTP/WebSocket"| NextJSFrontend
    NextJSFrontend -->|"HTTP REST"| APIRoutes
    NextJSFrontend -->|"WebSocket"| WebSocketClient

    click NextJSFrontend "https://github.com/ishaan1013/shadow/tree/main/apps/frontend"
    click APIRoutes "https://github.com/ishaan1013/shadow/tree/main/apps/frontend/app/api"
    click WebSocketClient "https://github.com/ishaan1013/shadow/tree/main/apps/frontend/hooks/socket"

    subgraph "Orchestrator Layer"
        direction TB
        HTTPServer["HTTP Server & API Gateway"]:::backend
        Middleware["Middleware"]:::backend
        WebSocketServer["WebSocket Server"]:::backend
        TaskManager["Task Manager & Workflows"]:::backend
        AgentLayer["Agent Layer"]:::backend
        MCPAgent["MCP Agent"]:::backend
        Execution["Execution Abstraction"]:::backend
        subgraph "Execution Strategies"
            direction TB
            LocalMode["LocalMode (Local-FS)"]:::backend
            RemoteMode["RemoteMode (K8s)"]:::backend
        end
        Indexing["Indexing Service"]:::backend
        GitHubInt["GitHub Integration"]:::backend
        DBClient["Prisma DB Client"]:::backend
        AuxServices["Auxiliary Services"]:::backend
    end

    APIRoutes -->|"HTTP"| HTTPServer
    HTTPServer --> Middleware
    Middleware --> WebSocketServer
    Middleware --> TaskManager
    TaskManager --> AgentLayer
    AgentLayer --> MCPAgent
    TaskManager --> Execution
    Execution --> LocalMode
    Execution --> RemoteMode
    AgentLayer --> Indexing
    TaskManager --> AuxServices
    AgentLayer --> GitHubInt
    HTTPServer --> DBClient

    click HTTPServer "https://github.com/ishaan1013/shadow/blob/main/apps/server/src/server.ts"
    click Middleware "https://github.com/ishaan1013/shadow/tree/main/apps/server/src/middleware"
    click WebSocketServer "https://github.com/ishaan1013/shadow/blob/main/apps/server/src/socket.ts"
    click TaskManager "https://github.com/ishaan1013/shadow/tree/main/apps/server/src/services"
    click AgentLayer "https://github.com/ishaan1013/shadow/tree/main/apps/server/src/agent"
    click MCPAgent "https://github.com/ishaan1013/shadow/tree/main/apps/server/src/agent/mcp"
    click Execution "https://github.com/ishaan1013/shadow/tree/main/apps/server/src/execution"
    click Indexing "https://github.com/ishaan1013/shadow/tree/main/apps/server/src/indexing"
    click GitHubInt "https://github.com/ishaan1013/shadow/tree/main/apps/server/src/github"
    click DBClient "https://github.com/ishaan1013/shadow/blob/main/packages/db/src/client.ts"

    subgraph "Sidecar Layer"
        SidecarAPI["Sidecar REST API"]:::sidecar
    end

    TaskManager -->|"REST"| SidecarAPI
    SidecarAPI -->|"File/Git/Exec Ops"| TaskManager

    click SidecarAPI "https://github.com/ishaan1013/shadow/tree/main/apps/sidecar/src"

    subgraph "Data & External Services"
        PostgreSQL["PostgreSQL"]:::db
        Pinecone["Pinecone Vector Store"]:::external
        LLM["LLM Providers"]:::external
        GitHubAPI["GitHub API"]:::external
    end

    DBClient --> PostgreSQL
    Indexing --> Pinecone
    AgentLayer --> LLM
    GitHubInt --> GitHubAPI

    click PostgreSQL "https://github.com/ishaan1013/shadow/blob/main/packages/db/prisma/schema.prisma"
    click Pinecone "https://github.com/ishaan1013/shadow/blob/main/apps/server/src/indexing/embedding/pineconeService.ts"
    click LLM "https://github.com/ishaan1013/shadow/blob/main/apps/server/src/agent/llm/index.ts"
    click LLM "https://github.com/ishaan1013/shadow/tree/main/apps/server/src/agent/llm/models"

    classDef client fill:#b3cde0,stroke:#333,stroke-width:1px
    classDef backend fill:#ccebc5,stroke:#333,stroke-width:1px
    classDef sidecar fill:#fdae6b,stroke:#333,stroke-width:1px
    classDef db fill:#ffffb3,stroke:#333,stroke-width:1px
    classDef external fill:#decbe4,stroke:#333,stroke-width:1px
```
