# vindyang/cs464-project architecture

> Architecture diagram of the GitHub repository vindyang/cs464-project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vindyang/cs464-project
- Repository: https://github.com/vindyang/cs464-project

## Overview

1. This repository is a full-stack distributed file-storage system: a Go backend split into multiple services, a Next.js/Bun frontend, and Docker Compose wiring that runs the whole stack from the repo root.

2. The main request path starts at the frontend and the gateway. The `frontend/` app is the user-facing UI, while `backend/services/gateway/` is the HTTP edge layer, implemented with Nginx config plus OpenAPI/Swagger assets. The gateway is the public backend entrypoint and is the boundary between browser traffic and internal services.

3. The backend is organized as service-oriented components rather than one monolith. Each service has its own `cmd/`, `internal/`, Dockerfile, and tests, which makes the service boundaries explicit and independently runnable.

4. `orchestrator` is the workflow coordinator. Its job is to execute higher-level file operations by calling other backend services through shared HTTP clients. In the architecture diagram, it should sit in the middle of upload/download flows, not as a data store.

5. `sharding` is the service that determines how files are split or distributed. It exposes an API and has a small application layer, suggesting it is responsible for shard sizing/placement logic rather than persistence.

6. `shardmap` is the metadata authority. It contains handlers, application logic, and repository implementations for files and shards, so it is the service that persists and serves the mapping between logical files, their shards, and the storage nodes/providers that hold them.

7. `adapter` is the execution layer that talks to external storage providers. Its handlers and `file_operations_service` indicate it performs the actual file I/O work, while shared adapter code includes Google Drive and S3 integrations plus OAuth handling for Drive auth. This is the boundary where backend logic leaves the system and reaches third-party storage APIs.

8. The shared package under `backend/services/shared/` is a cross-cutting code layer, not a separate runtime. It holds DTOs, middleware, transport helpers, database access, reusable clients for service-to-service calls, common models/types, and provider adapters. That layer defines the contracts between services and keeps duplication down.

9. Data persistence is centralized around the shard-mapping side. The repo includes `schema.sql`, `seed.sql`, and `docker/init-local.sql`, plus repository code inside `shardmap`, which indicates an SQL-backed metadata store initialized locally for development and testing.

10. The frontend is a rich authenticated admin console. Its route groups include dashboard, nodes, providers, and settings, with reusable components for file tables, upload dialogs, shard progress, system health, and provider cards. It consumes backend APIs through `frontend/lib/api/` and keeps UI state in dedicated stores.

11. The frontend also supports local mocked data paths. `frontend/lib/mocks/` and store modules suggest the UI can be developed or exercised without the live backend, which is useful for isolated UI work.

12. Testing is layered by service. Each backend service has unit tests, `backend/tests/integration/` contains orchestrator contract/failure-path tests, and `backend/tests/e2e/` provides end-to-end coverage. There are also local test utilities like `cmd/mockserver` and `cmd/test-db` for dependency simulation.

13. Deployment is container-first. Docker Compose profiles separate `backend` and `full` runs, so the backend services, gateway, and frontend are intended to be composed together as a system, with the gateway sitting at the ingress and the other services forming the internal control plane and storage workflow.

Main entry points: [`docker-compose.yml`](https://github.com/vindyang/cs464-project/blob/main/docker-compose.yml) (Compose), [`backend/services/shared`](https://github.com/vindyang/cs464-project/tree/main/backend/services/shared) (Shared Code), [`backend/services/orchestrator`](https://github.com/vindyang/cs464-project/tree/main/backend/services/orchestrator) (Orchestrator), [`backend/services/gateway`](https://github.com/vindyang/cs464-project/tree/main/backend/services/gateway) (Gateway)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_frontend["Frontend"]
  node_frontend_ui["Admin Console<br/>Next.js UI"]
  node_frontend_api["API Client<br/>Frontend data access"]
  node_frontend_store["UI State<br/>Frontend stores"]
end

subgraph group_backend_services["Backend Services"]
  node_gateway["Gateway<br/>HTTP edge"]
  node_orchestrator["Orchestrator<br/>workflow coordinator"]
  node_sharding["Sharding<br/>split policy service"]
  node_shardmap[("Shard Map<br/>metadata service")]
  node_adapter["Adapter<br/>storage executor"]
  node_metadata_db[("SQL Store<br/>metadata persistence<br/>[schema.sql]")]
end

subgraph group_shared["Shared Layer"]
  node_shared_layer["Shared Code<br/>cross-cutting layer"]
end

subgraph group_tests["Tests"]
  node_integration_tests["Integration Tests<br/>backend contracts"]
  node_e2e_tests["E2E Tests<br/>backend e2e"]
  node_mockserver["Mock Server<br/>test harness"]
  node_testdb["Test DB<br/>test harness"]
end

subgraph group_deploy["Deploy"]
  node_compose["Compose<br/>deployment wiring<br/>[docker-compose.yml]"]
end

node_frontend_ui -->|"reads data from"| node_frontend_api
node_frontend_ui -->|"uses state"| node_frontend_store
node_frontend_api -->|"calls"| node_gateway
node_gateway -->|"routes workflow"| node_orchestrator
node_gateway -->|"exposes metadata"| node_shardmap
node_orchestrator -->|"plans shards"| node_sharding
node_orchestrator -->|"updates metadata"| node_shardmap
node_orchestrator -->|"executes storage ops"| node_adapter
node_sharding -.->|"uses contracts"| node_shared_layer
node_shardmap -->|"persists to"| node_metadata_db
node_adapter -.->|"uses providers"| node_shared_layer
node_shared_layer -.->|"shared transport"| node_gateway
node_shared_layer -.->|"shared clients"| node_orchestrator
node_shared_layer -.->|"shared clients"| node_sharding
node_shared_layer -.->|"shared contracts"| node_shardmap
node_shared_layer -.->|"shared adapters"| node_adapter
node_integration_tests -->|"verifies"| node_orchestrator
node_e2e_tests -->|"runs against"| node_compose
node_mockserver -.->|"supports"| node_integration_tests
node_testdb -.->|"supports"| node_integration_tests
node_compose -->|"starts"| node_gateway
node_compose -->|"starts"| node_orchestrator
node_compose -->|"starts"| node_sharding
node_compose -->|"starts"| node_shardmap
node_compose -->|"starts"| node_adapter
node_compose -->|"starts"| node_frontend_ui

click node_frontend_ui "https://github.com/vindyang/cs464-project/tree/main/frontend/app"
click node_frontend_api "https://github.com/vindyang/cs464-project/tree/main/frontend/lib/api"
click node_frontend_store "https://github.com/vindyang/cs464-project/tree/main/frontend/lib/store"
click node_gateway "https://github.com/vindyang/cs464-project/tree/main/backend/services/gateway"
click node_orchestrator "https://github.com/vindyang/cs464-project/tree/main/backend/services/orchestrator"
click node_sharding "https://github.com/vindyang/cs464-project/tree/main/backend/services/sharding"
click node_shardmap "https://github.com/vindyang/cs464-project/tree/main/backend/services/shardmap"
click node_adapter "https://github.com/vindyang/cs464-project/tree/main/backend/services/adapter"
click node_shared_layer "https://github.com/vindyang/cs464-project/tree/main/backend/services/shared"
click node_metadata_db "https://github.com/vindyang/cs464-project/blob/main/backend/schema.sql"
click node_integration_tests "https://github.com/vindyang/cs464-project/tree/main/backend/tests/integration"
click node_e2e_tests "https://github.com/vindyang/cs464-project/tree/main/backend/tests/e2e"
click node_mockserver "https://github.com/vindyang/cs464-project/tree/main/backend/cmd/mockserver"
click node_testdb "https://github.com/vindyang/cs464-project/tree/main/backend/cmd/test-db"
click node_compose "https://github.com/vindyang/cs464-project/blob/main/docker-compose.yml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_frontend_ui,node_frontend_api,node_frontend_store toneBlue
class node_gateway,node_orchestrator,node_sharding,node_shardmap,node_adapter,node_metadata_db toneAmber
class node_shared_layer toneMint
class node_integration_tests,node_e2e_tests,node_mockserver,node_testdb toneRose
class node_compose toneIndigo
```

## Components

### Frontend

Next.js/Bun admin console and UI state

- [Admin Console](https://github.com/vindyang/cs464-project/tree/main/frontend/app): Next.js UI · Authenticated browser UI for dashboard, nodes, providers, and settings · `frontend/app`
- [API Client](https://github.com/vindyang/cs464-project/tree/main/frontend/lib/api): Frontend data access · Typed client layer used by the UI to call backend APIs · `frontend/lib/api`
- [UI State](https://github.com/vindyang/cs464-project/tree/main/frontend/lib/store): Frontend stores · Client-side stores and mock data paths for local UI workflows · `frontend/lib/store`

### Backend Services

Go runtime services for ingress, workflow, sharding, metadata, and storage execution

- [Gateway](https://github.com/vindyang/cs464-project/tree/main/backend/services/gateway): HTTP edge · Public ingress layer with Nginx config and OpenAPI/Swagger assets · `backend/services/gateway`
- [Orchestrator](https://github.com/vindyang/cs464-project/tree/main/backend/services/orchestrator): workflow coordinator · Coordinates upload and download workflows by calling backend services · `backend/services/orchestrator`
- [Sharding](https://github.com/vindyang/cs464-project/tree/main/backend/services/sharding): split policy service · Determines shard sizing and distribution logic through its API and app layer · `backend/services/sharding`
- [Shard Map](https://github.com/vindyang/cs464-project/tree/main/backend/services/shardmap): metadata service · Persists and serves file, shard, and storage-node mappings · `backend/services/shardmap`
- [Adapter](https://github.com/vindyang/cs464-project/tree/main/backend/services/adapter): storage executor · Executes file operations against external storage providers such as Drive and S3 · `backend/services/adapter`
- [SQL Store](https://github.com/vindyang/cs464-project/blob/main/backend/schema.sql): metadata persistence · Schema and seed SQL backing the shard-mapping and workflow metadata · `backend/schema.sql`

### Shared Layer

Cross-cutting DTOs, clients, transport, auth, and provider adapters

- [Shared Code](https://github.com/vindyang/cs464-project/tree/main/backend/services/shared): cross-cutting layer · Reusable DTOs, middleware, clients, transport helpers, models, and provider adapters · `backend/services/shared`

### Tests

Integration and end-to-end coverage plus local test harnesses

- [Integration Tests](https://github.com/vindyang/cs464-project/tree/main/backend/tests/integration): backend contracts · Orchestrator contract and failure-path tests for service interaction · `backend/tests/integration`
- [E2E Tests](https://github.com/vindyang/cs464-project/tree/main/backend/tests/e2e): backend e2e · End-to-end coverage for the composed backend system · `backend/tests/e2e`
- [Mock Server](https://github.com/vindyang/cs464-project/tree/main/backend/cmd/mockserver): test harness · Local dependency simulator for backend testing · `backend/cmd/mockserver`
- [Test DB](https://github.com/vindyang/cs464-project/tree/main/backend/cmd/test-db): test harness · Local database bootstrap utility for tests and development · `backend/cmd/test-db`

### Deploy

Repo-level container orchestration and initialization

- [Compose](https://github.com/vindyang/cs464-project/blob/main/docker-compose.yml): deployment wiring · Container-first system assembly for backend and full-stack runs · `docker-compose.yml`

## Connections

- Admin Console → API Client: reads data from · The UI uses the API client layer for backend communication
- Admin Console → UI State: uses state · The UI relies on local stores and mock paths for interaction state
- API Client → Gateway: calls · Browser traffic is routed through the public gateway
- Gateway → Orchestrator: routes workflow · The edge layer forwards core file workflows to the orchestrator
- Gateway → Shard Map: exposes metadata · Gateway-backed APIs surface shard and file metadata operations
- Orchestrator → Sharding: plans shards · Workflow coordination depends on sharding decisions
- Orchestrator → Shard Map: updates metadata · Workflow execution reads and writes shard-map state
- Orchestrator → Adapter: executes storage ops · The orchestrator drives storage actions through the adapter service
- Sharding → Shared Code: uses contracts · Sharding shares DTOs and transport helpers from the common layer
- Shard Map → SQL Store: persists to · Shard mapping service stores file and shard metadata in SQL
- Adapter → Shared Code: uses providers · Adapter code reuses shared provider integrations and OAuth helpers
- Shared Code → Gateway: shared transport · Common middleware and transport helpers support service boundaries
- Shared Code → Orchestrator: shared clients · Orchestrator uses shared HTTP clients and DTOs
- Shared Code → Sharding: shared clients · Sharding uses common contracts and transport helpers
- Shared Code → Shard Map: shared contracts · Shard-map service depends on common models and middleware
- Shared Code → Adapter: shared adapters · Adapter service reuses common provider and transport code
- Integration Tests → Orchestrator: verifies · Contract tests focus on orchestrator behavior and failure handling
- E2E Tests → Compose: runs against · End-to-end tests target the composed stack
- Mock Server → Integration Tests: supports · Mock services simulate backend dependencies for tests
- Test DB → Integration Tests: supports · Local test database bootstrap supports integration scenarios
- Compose → Gateway: starts · Compose wires the public ingress and internal services together
- Compose → Orchestrator: starts · Compose includes the workflow coordinator service
- Compose → Sharding: starts · Compose includes the shard decision service
- Compose → Shard Map: starts · Compose includes the metadata authority service
- Compose → Adapter: starts · Compose includes the storage execution service
- Compose → Admin Console: starts · Compose includes the browser UI for full-stack runs
