# neerajrao23/mini-pipeline architecture

> Architecture diagram of the GitHub repository neerajrao23/mini-pipeline, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/neerajrao23/mini-pipeline
- Repository: https://github.com/neerajrao23/mini-pipeline

## Overview

##### Project type and purpose
- This is a containerized, dual-microservice video-processing pipeline built with Node.js and Docker Compose.
- One service (“api”) handles video ingestion, processing, and asynchronous callbacks. The other (“interface”) exposes a web UI, handles callbacks, stores results, and renders them via EJS.

##### File structure
- Top-level:
- docker-compose.yml – orchestrates both services
- .env examples – environment variable templates
- api/
- Dockerfile, .env, package.json, server.js
- Exposes REST endpoints on port 8000
- interface/
- Dockerfile, package.json, index.js
- public/ (static assets), views/ (EJS templates)
- Exposes port 4000, handles incoming callback posts, writes received.json, serves UI

##### README for key insights
- Features: file upload & remote-URL input, webhook callbacks, persistent storage (received.json), health checks, Docker volume mapping

##### Detail
- Every REST endpoint and method
- Environment variables usage via .env files
- Volume mounts under each service
- Host access (host.docker.internal for callbacks in URL mode)
- Both file-upload and URL-fetch flows

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Clients
    CLI["CLI / HTTP Client"]:::client
    Browser["Browser"]:::client
    RemoteSource["Remote Video Source"]:::external
    Monitoring["Monitoring"]:::monitor

    %% Configuration files
    Config["docker-compose.yml"]:::config
    EnvTemplate[".env.example"]:::config
    README["README.md"]:::config

    %% Docker Compose boundary
    subgraph "Docker Compose" 
        direction TB
        subgraph "Containers"
            direction TB
            API["API Service<br/>(Node.js, port 8000)"]:::api
            Interface["Interface Service<br/>(Node.js/EJS, port 4000)"]:::interface
        end
        NETWORK["Docker Bridge Network"]:::network
        Storage["Persistent Storage<br/>(received.json)"]:::storage
    end

    %% Flows
    CLI -->|"POST /process-video\n(file or JSON)"| API
    API -->|"HTTP GET video_url"| RemoteSource
    API -->|"POST /callbacks/steps"| Interface
    Interface -->|"write/read JSON"| Storage
    Browser -->|"GET /guides/:id\nrenders EJS"| Interface
    Monitoring -.->|"GET /health"| API
    Monitoring -.->|"GET /health"| Interface
    API --- NETWORK
    NETWORK --- Interface

    %% Click Events
    click Config "https://github.com/neerajrao23/mini-pipeline/blob/main/docker-compose.yml"
    click EnvTemplate "https://github.com/neerajrao23/mini-pipeline/blob/main/.env.example"
    click README "https://github.com/neerajrao23/mini-pipeline/blob/main/README.md"
    click API "https://github.com/neerajrao23/mini-pipeline/tree/main/api/Dockerfile"
    click API "https://github.com/neerajrao23/mini-pipeline/blob/main/api/.env"
    click API "https://github.com/neerajrao23/mini-pipeline/blob/main/api/.env.example"
    click API "https://github.com/neerajrao23/mini-pipeline/blob/main/api/package.json"
    click API "https://github.com/neerajrao23/mini-pipeline/blob/main/api/server.js"
    click Interface "https://github.com/neerajrao23/mini-pipeline/tree/main/interface/Dockerfile"
    click Interface "https://github.com/neerajrao23/mini-pipeline/blob/main/interface/package.json"
    click Interface "https://github.com/neerajrao23/mini-pipeline/blob/main/interface/index.js"
    click Interface "https://github.com/neerajrao23/mini-pipeline/blob/main/interface/received.json"
    click Interface "https://github.com/neerajrao23/mini-pipeline/blob/main/interface/public/style.css"
    click Interface "https://github.com/neerajrao23/mini-pipeline/blob/main/interface/views/guide.ejs"

    %% Styles
    classDef api fill:#ffdddd,stroke:#aa0000,stroke-width:1px;
    classDef interface fill:#ddddff,stroke:#0000aa,stroke-width:1px;
    classDef client fill:#ddffdd,stroke:#008800,stroke-width:1px;
    classDef external fill:#ffffcc,stroke:#888800,stroke-width:1px;
    classDef storage fill:#ffccdd,stroke:#aa0055,stroke-width:1px,shape:cylinder;
    classDef monitor fill:#f0f0f0,stroke:#666666,stroke-width:1px,stroke-dasharray: 5 5;
    classDef network fill:#e0e0e0,stroke:#000000,stroke-width:1px,stroke-dasharray: 2 2;
    classDef config fill:#ffffff,stroke:#333333,stroke-width:1px,stroke-dasharray: 2 2;
```
