# matthew-ngzc/travel-info-extractor architecture

> Architecture diagram of the GitHub repository matthew-ngzc/travel-info-extractor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-02-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/matthew-ngzc/travel-info-extractor
- Repository: https://github.com/matthew-ngzc/travel-info-extractor

## Overview

##### Project Type and Purpose
- This is a modular, containerized media-processing pipeline composed of discrete microservices.
- Each service (media-processor, transcription, ocr) performs one responsibility:
- media-processor: downloads or ingests raw media (audio/video) and extracts segments or metadata
- transcription: runs Whisper-based transcription on extracted audio
- ocr: (based on README) performs optical character recognition on image or video frames
- A shared “data” directory holds job artifacts and intermediate files.

##### File-Structure Analysis
Top level:
- README.md – describes folder layout
- .gitignore
Sub-folders (each is a standalone containerized service):
- media-processor/
– Dockerfile, docker-compose.yml, main.py, pyproject.toml, etc.
- transcription/
– Dockerfile, docker-compose.yml, app/ (main.py, models.py, transcription.py), pyproject.toml
- (ocr/) – similar pattern, not shown in tree
- data/ (git-ignored) – shared volume for artifacts
Patterns:
- Each service is in Python, uses Poetry or pyproject, runs in its own container.
- Docker Compose files in each service; likely orchestrated together at root via another docker-compose or script.

##### Key Architectural Insights from README
- A pipeline of services, loosely coupled and communicating via a shared filesystem (“data” folder) or potentially message queues.
- Each step picks up “jobs” from the shared directory, processes, and writes outputs back.

a. Main Components
- Ingress (job submission client or scheduler)
- media-processor service (container)
- transcription service (container)
- ocr service (container)
- Shared Data Volume (persistent storage)
- Optional Orchestrator (root-level docker-compose or workflow engine)
- External dependencies (e.g., Whisper model artifacts, external media sources)
b. Relationships & Data Flow
1. Ingress submits media download request → media-processor
2. media-processor pulls media, extracts audio/video segments → writes to Shared Data Volume
3. transcription watches Volume → picks up audio segments → runs Whisper → writes transcript JSON to Volume
4. ocr watches Volume → picks up image frames → runs OCR → writes text/artifacts to Volume
5. Downstream consumer (reporter or UI) reads final artifacts
c. Architectural Patterns & Principles
- Microservices: each service is isolated, single responsibility
- Containerization: each service in its own Docker container, with service-specific Dockerfile + Compose
- Shared-volume integration or message-driven pipeline (filesystem as queue)
- Clear separation of concerns: ingestion, processing, transcription, OCR
d. Technologies & Frameworks
- Python 3.x, Poetry/pyproject.toml dependency management
- Docker, Docker Compose for container orchestration
- Whisper (OpenAI) for transcription
- OCR library (e.g., Tesseract) for ocr service
- FastAPI or simple Python scripts (main.py) as service entrypoints

##### Final Advice
- Every component—even scripts like main.py can be shown as the service entrypoint.
- Don’t over–generalize: show both Dockerfiles and docker-compose.yml as part of each service’s boundary.
- Make data flows explicit, showing how jobs move through media-processor → transcription → ocr.

## Diagram (Mermaid)

```mermaid
graph TD
    %% Orchestration and Ingress
    subgraph "Orchestrator" 
        ORC["Root docker-compose"]:::orches
    end

    subgraph "Ingress / Scheduler" 
        IN["Job Submission Client"]:::external
    end

    %% Media Processor Service
    subgraph "Media-Processor Service" 
        MP["Container"]:::service
        DF1["Dockerfile"]:::file
        DC1["docker-compose.yml"]:::file
        M1["main.py"]:::file
        P1["pyproject.toml"]:::file
        R1["README.md"]:::file
    end

    %% Shared Data Volume
    DV["Shared Data Volume"]:::storage

    %% Transcription Service
    subgraph "Transcription Service"
        TS["Container"]:::service
        DF2["Dockerfile"]:::file
        DC2["docker-compose.yml"]:::file
        M2["main.py"]:::file
        M3["models.py"]:::file
        T1["transcription.py"]:::file
        P2["pyproject.toml"]:::file
        R2["README.md"]:::file
    end

    %% OCR Service
    subgraph "OCR Service"
        OCRC["Container"]:::service
    end

    %% Downstream Consumer
    EXT["Reporter / UI"]:::external

    %% Data Flows
    IN -->|submit job| ORC
    ORC -->|start container| MP
    MP -->|audio/video segments| DV
    DV -->|audio segments| TS
    TS -->|transcript.json| DV
    DV -->|image frames| OCRC
    OCRC -->|ocr_text.json| DV
    DV -->|final artifacts| EXT

    %% Click Events for Media-Processor
    click DF1 "https://github.com/matthew-ngzc/travel-info-extractor/tree/main/media-processor/Dockerfile"
    click DC1 "https://github.com/matthew-ngzc/travel-info-extractor/blob/main/media-processor/docker-compose.yml"
    click M1 "https://github.com/matthew-ngzc/travel-info-extractor/blob/main/media-processor/main.py"
    click P1 "https://github.com/matthew-ngzc/travel-info-extractor/blob/main/media-processor/pyproject.toml"
    click R1 "https://github.com/matthew-ngzc/travel-info-extractor/blob/main/media-processor/README.md"

    %% Click Events for Transcription
    click DF2 "https://github.com/matthew-ngzc/travel-info-extractor/tree/main/transcription/Dockerfile"
    click DC2 "https://github.com/matthew-ngzc/travel-info-extractor/blob/main/transcription/docker-compose.yml"
    click M2 "https://github.com/matthew-ngzc/travel-info-extractor/blob/main/transcription/app/main.py"
    click M3 "https://github.com/matthew-ngzc/travel-info-extractor/blob/main/transcription/app/models.py"
    click T1 "https://github.com/matthew-ngzc/travel-info-extractor/blob/main/transcription/app/transcription.py"
    click P2 "https://github.com/matthew-ngzc/travel-info-extractor/blob/main/transcription/pyproject.toml"
    click R2 "https://github.com/matthew-ngzc/travel-info-extractor/blob/main/transcription/README.md"

    %% Styles
    classDef orches fill:#8dd3c7,stroke:#333,stroke-width:2px
    classDef external fill:#fb8072,stroke:#333,stroke-width:2px
    classDef service fill:#80b1d3,stroke:#333,stroke-width:2px
    classDef storage fill:#fdb462,stroke:#333,stroke-width:2px
    classDef file fill:#e0e0e0,stroke:#666,stroke-dasharray: 2 2
    class ORC orches
    class IN,EXT external
    class MP,TS,OCRC service
    class DV storage
    class DF1,DC1,M1,P1,R1,DF2,DC2,M2,M3,T1,P2,R2 file
```
