# samilboyalikli/hospitalstream architecture

> Architecture diagram of the GitHub repository samilboyalikli/hospitalstream, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/samilboyalikli/hospitalstream
- Repository: https://github.com/samilboyalikli/hospitalstream

## Overview

##### Main Components
- Docker Compose Orchestrator
- Apache Kafka Cluster
- Producer Services (“Hospitals” Spark Batch Generators)
- 13 containers (one per hospital script) encapsulated under a shared “hospitals” service.
- Each uses Spark Batch to generate fake patient records.
- Distributor Service
- A separate container reading from the raw_stream topic.
- Routes messages by “Age” to three child topics.
- Analytics Services (Topics Modules)
- Three containerized Spark Structured-Streaming consumers:
– children
– adult
– senior
- Each reads from its respective Kafka topic, flags records, writes to analyzed_stream.
- Consumer Service

##### Interactions & Data Flows
- Labels: “route by age”

##### Architectural Patterns
- Streaming pipeline pattern: producer → message bus → consumer/processor
- Micro-service separation: each logical stage lives in its own container
- Topic-based pub/sub segregation by age group
- Batch generation (Spark) vs. real-time streaming (Spark Structured Streaming)

##### Call Out Key Technologies
- Docker & Docker Compose
- All services defined in docker-compose.yml
- Apache Spark
- Used in producers (batch) and analytics (streaming)
- Apache Kafka
- Five topics: raw_stream, children, adult, senior, analyzed_stream
- Python
- All code for producers, distributor, topics, and consumer
- EntryPoint & Dockerfiles
- Each service has its own Dockerfile (and entrypoint.sh for topics)

##### Extra Details to Capture
- That “hospitals” use Spark batch modules but run continuously under Docker.
- Optionally, show the docker-compose.yml file linking all services over a common network.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Docker Compose Network"
        DockerCompose["docker-compose.yml"]:::config
        Producer["Hospitals Spark Batch Generators"]:::producer
        KafkaRaw["Kafka Cluster\n(raw_stream topic)"]:::kafka
        Distributor["Distributor (age router)"]:::processor
        subgraph "Age-based Analytics"
            Children["Children Analytics\n(Spark Structured Streaming)"]:::processor
            Adult["Adult Analytics\n(Spark Structured Streaming)"]:::processor
            Senior["Senior Analytics\n(Spark Structured Streaming)"]:::processor
        end
        KafkaAnalyzed["Kafka Cluster\n(analyzed_stream topic)"]:::kafka
        Consumer["Consumer Service"]:::consumer
    end

    DockerCompose --> Producer
    Producer -->|"produce raw patient records"| KafkaRaw
    KafkaRaw -->|"consume raw_stream"| Distributor
    Distributor -->|"route by age to children"| ChildrenTopic["children topic"]:::kafka
    Distributor -->|"route by age to adult"| AdultTopic["adult topic"]:::kafka
    Distributor -->|"route by age to senior"| SeniorTopic["senior topic"]:::kafka

    ChildrenTopic -->|"consume children"| Children
    AdultTopic -->|"consume adult"| Adult
    SeniorTopic -->|"consume senior"| Senior

    Children -->|"flag & write analyzed records"| KafkaAnalyzed
    Adult -->|"flag & write analyzed records"| KafkaAnalyzed
    Senior -->|"flag & write analyzed records"| KafkaAnalyzed

    KafkaAnalyzed -->|"consume & display"| Consumer

    click DockerCompose "https://github.com/samilboyalikli/hospitalstream/blob/main/docker-compose.yml"
    click Producer "https://github.com/samilboyalikli/hospitalstream/tree/main/hospitals/"
    click Producer "https://github.com/samilboyalikli/hospitalstream/tree/main/hospitals/Dockerfile"
    click Producer "https://github.com/samilboyalikli/hospitalstream/blob/main/hospitals/requirements.txt"
    click Producer "https://github.com/samilboyalikli/hospitalstream/blob/main/hospitals/atlanta.py"
    click Producer "https://github.com/samilboyalikli/hospitalstream/blob/main/hospitals/boston.py"
    click Producer "https://github.com/samilboyalikli/hospitalstream/blob/main/hospitals/chicago.py"
    click Producer "https://github.com/samilboyalikli/hospitalstream/blob/main/hospitals/dallas.py"
    click Producer "https://github.com/samilboyalikli/hospitalstream/blob/main/hospitals/detroit.py"
    click Producer "https://github.com/samilboyalikli/hospitalstream/blob/main/hospitals/houston.py"
    click Producer "https://github.com/samilboyalikli/hospitalstream/blob/main/hospitals/jersey.py"
    click Producer "https://github.com/samilboyalikli/hospitalstream/blob/main/hospitals/miami.py"
    click Producer "https://github.com/samilboyalikli/hospitalstream/blob/main/hospitals/newyork.py"
    click Producer "https://github.com/samilboyalikli/hospitalstream/blob/main/hospitals/phoenix.py"
    click Producer "https://github.com/samilboyalikli/hospitalstream/blob/main/hospitals/sanfrancisco.py"
    click Producer "https://github.com/samilboyalikli/hospitalstream/blob/main/hospitals/seattle.py"
    click Producer "https://github.com/samilboyalikli/hospitalstream/blob/main/hospitals/washington.py"

    click Distributor "https://github.com/samilboyalikli/hospitalstream/tree/main/distributor/"
    click Distributor "https://github.com/samilboyalikli/hospitalstream/tree/main/distributor/Dockerfile"
    click Distributor "https://github.com/samilboyalikli/hospitalstream/blob/main/distributor/requirements.txt"
    click Distributor "https://github.com/samilboyalikli/hospitalstream/blob/main/distributor/distributor.py"

    click Children "https://github.com/samilboyalikli/hospitalstream/tree/main/topics/children/"
    click Children "https://github.com/samilboyalikli/hospitalstream/tree/main/topics/children/Dockerfile"
    click Children "https://github.com/samilboyalikli/hospitalstream/blob/main/topics/children/entrypoint.sh"
    click Children "https://github.com/samilboyalikli/hospitalstream/blob/main/topics/children/main.py"

    click Adult "https://github.com/samilboyalikli/hospitalstream/tree/main/topics/adult/"
    click Adult "https://github.com/samilboyalikli/hospitalstream/tree/main/topics/adult/Dockerfile"
    click Adult "https://github.com/samilboyalikli/hospitalstream/blob/main/topics/adult/entrypoint.sh"
    click Adult "https://github.com/samilboyalikli/hospitalstream/blob/main/topics/adult/main.py"

    click Senior "https://github.com/samilboyalikli/hospitalstream/tree/main/topics/senior/"
    click Senior "https://github.com/samilboyalikli/hospitalstream/tree/main/topics/senior/Dockerfile"
    click Senior "https://github.com/samilboyalikli/hospitalstream/blob/main/topics/senior/entrypoint.sh"
    click Senior "https://github.com/samilboyalikli/hospitalstream/blob/main/topics/senior/main.py"

    click Consumer "https://github.com/samilboyalikli/hospitalstream/tree/main/consumer/"
    click Consumer "https://github.com/samilboyalikli/hospitalstream/tree/main/consumer/Dockerfile"
    click Consumer "https://github.com/samilboyalikli/hospitalstream/blob/main/consumer/requirements.txt"
    click Consumer "https://github.com/samilboyalikli/hospitalstream/blob/main/consumer/main.py"

    classDef producer fill:#3498db,stroke:#2c3e50,color:#fff;
    classDef processor fill:#f39c12,stroke:#2c3e50,color:#fff;
    classDef consumer fill:#2ecc71,stroke:#2c3e50,color:#fff;
    classDef kafka fill:#95a5a6,stroke:#2c3e50,color:#fff;
    classDef config fill:#ecf0f1,stroke:#2c3e50,color:#2c3e50,stroke-dasharray: 5 5;
```
