# codeforleipzig/lvz-viz architecture

> Architecture diagram of the GitHub repository codeforleipzig/lvz-viz, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/codeforleipzig/lvz-viz
- Repository: https://github.com/codeforleipzig/lvz-viz

## Overview

##### Project Type and Purpose
- The backend is a Spring Boot service (Java/Gradle) with embedded crawling and NLP steps and an Elasticsearch datastore.
- The frontend is an Angular SPA (in `experimental_web/`) that fetches data via REST APIs.

##### File Structure
- Top‑level components:
– `src/main/java/...` — Spring Boot application, REST controller, crawler modules, NER module, repository interfaces.
– `src/main/resources/public/` — legacy/static UI assets served by Spring Boot.
– `experimental_web/` — modern Angular application with its own package.json, build, and e2e tests.
– `docker-compose*.yml`, `Dockerfile`, Gradle wrapper — containerization and build orchestration.
– `application-*.properties` — Spring profiles for dev/local/test/prod.
- Key sub‑modules in `src/main/java`:
– `crawler/` — CrawlScheduler, police‑ticker crawlers, Nominatim geocoding.
– `ner/` — Named‑entity recognition logic.
– `repositories/` — Elasticsearch repository interfaces.
– `web/` — REST controllers exposing JSON endpoints to UIs.
– `utilities/` — shared helper classes.
- Notice use of Spring’s `@Scheduled`, dependency injection, and Spring Data Elasticsearch patterns.

##### Extract Architectural Insights from README
- Data store: Elasticsearch (spun up via `docker-compose`).
- Crawler is activated at startup and can be delayed or configured via Spring profiles or env vars.
- Deployment supports multi‑stage Docker builds and separate dev/prod compose files.
- Frontend build uses npm + Grunt; backend build uses Gradle.

##### Main Components & Their Relationships
a. Frontend
– Angular app (`experimental_web/`), built to static assets → served either by its own Node server in dev or by Spring Boot in prod.
– Fetches data via HTTP REST calls to the backend.
b. Backend (Spring Boot)
– REST API layer (`PoliceTickerController`)
– Crawler scheduler triggers glue code:
- `LvzPoliceTickerCrawler` / `LvzPoliceTickerDetailViewCrawler` scrape HTML pages
- `NominatimAsker` enriches items with geolocation
- `NER` extracts entities
– Repository layer writes models (`PoliceTicker`) into Elasticsearch via `PoliceTickerRepository`
c. Data Store
– Elasticsearch cluster (defined in `docker-compose.yml`)
d. External Services
– LVZ website (source)
– Nominatim geocoding service
e. DevOps / Deployment
– Docker / docker‑compose sets up backend + Elasticsearch (+ optional UI server)
– Spring profiles determine which property files are active
– Multi‑stage Dockerfiles for production builds

##### Architectural Patterns & Design Principles
- Layered architecture: Controller → Service (crawler/NER) → Repository → Data store
- Spring Boot’s auto‑configuration and profile‑based config
- Asynchronous scheduled jobs (crawler) decoupled from request handling
- Single Page Application pattern on the frontend
- Container‑based micro‑services style (backend + Elasticsearch; frontend can be separate)

##### Final Checklist
– [ ] Clear labels on every component
– [ ] Notes on Docker Compose orchestration and profile selection

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Frontend
    subgraph "Frontend Layer"
        Angular["Angular UI\n(Angular 15 + Grunt)"]:::compute
    end

    %% Backend
    subgraph "Spring Boot Application\n(Spring Boot 2.x + Gradle)" 
        Entry["LvzViz.java"]:::compute
        Controller["PoliceTickerController\n(REST API Layer)"]:::compute
        Scheduler["CrawlScheduler\n(Scheduled Jobs)"]:::compute
        Crawler1["LvzPoliceTickerCrawler"]:::compute
        Crawler2["LvzPoliceTickerDetailViewCrawler"]:::compute
        Geocoder["NominatimAsker\n(Geocoding Enricher)"]:::compute
        NER["NER\n(Named‑Entity Recognition)"]:::compute
        Repo["PoliceTickerRepository\n(Spring Data Elasticsearch)"]:::compute
        Model["PoliceTicker\n(Domain Model)"]:::compute
        Utils["Utils\n(Shared Utilities)"]:::compute
        subgraph "Configurations"
            PropsDev["application-dev.properties"]:::deployment
            PropsLocal["application-local.properties"]:::deployment
            PropsTest["application-test.properties"]:::deployment
            PropsDefault["application.properties"]:::deployment
            MappingJSON["mapping.json\n(ES Mapping)"]:::deployment
        end
        subgraph "Legacy Static UI"
            LegacyUI["/public/\n(Static Assets)"]:::compute
        end
    end

    %% Data Store
    EsDb["Elasticsearch Cluster\n(ES Query DSL)"]:::db

    %% External Services
    LVZWeb["LVZ Website\n(Source)"]:::cloud
    NominatimExt["Nominatim Service"]:::cloud

    %% Deployment/DevOps
    subgraph "DevOps / Deployment"
        Dockerfile["Dockerfile"]:::deployment
        DockerfileDev["Dockerfile.dev"]:::deployment
        ComposeAll["docker-compose.yml"]:::deployment
        ComposeDev["docker-compose.dev.yml"]:::deployment
        ComposeProd["docker-compose.prod.yml"]:::deployment
        GradleWrapper["gradlew / gradlew.bat"]:::deployment
        BuildGradle["build.gradle"]:::deployment
    end

    %% Data Flows
    Angular -->|"HTTP/JSON"| Controller
    Controller -->|"calls"| Repo
    Repo -->|"Elasticsearch Query DSL"| EsDb

    Scheduler -->|"triggers"| Crawler1
    Scheduler -->|"triggers"| Crawler2

    Crawler1 -->|"HTTP→HTML"| LVZWeb
    Crawler2 -->|"HTTP→HTML"| LVZWeb

    Crawler1 -->|"HTTP/JSON"| NominatimExt
    Crawler2 -->|"HTTP/JSON"| NominatimExt

    NominatimExt -->|"geocoded data"| Geocoder
    Crawler1 -->|"raw data"| NER
    Crawler2 -->|"raw data"| NER
    Geocoder -->|"enriched data"| NER

    NER -->|"entities"| Repo

    %% Click Events
    click Angular "https://github.com/codeforleipzig/lvz-viz/tree/master/experimental_web/"
    click Entry "https://github.com/codeforleipzig/lvz-viz/blob/master/src/main/java/de/codefor/le/LvzViz.java"
    click Controller "https://github.com/codeforleipzig/lvz-viz/blob/master/src/main/java/de/codefor/le/web/PoliceTickerController.java"
    click Scheduler "https://github.com/codeforleipzig/lvz-viz/blob/master/src/main/java/de/codefor/le/crawler/CrawlScheduler.java"
    click Crawler1 "https://github.com/codeforleipzig/lvz-viz/blob/master/src/main/java/de/codefor/le/crawler/LvzPoliceTickerCrawler.java"
    click Crawler2 "https://github.com/codeforleipzig/lvz-viz/blob/master/src/main/java/de/codefor/le/crawler/LvzPoliceTickerDetailViewCrawler.java"
    click Geocoder "https://github.com/codeforleipzig/lvz-viz/blob/master/src/main/java/de/codefor/le/crawler/NominatimAsker.java"
    click NER "https://github.com/codeforleipzig/lvz-viz/blob/master/src/main/java/de/codefor/le/ner/NER.java"
    click Repo "https://github.com/codeforleipzig/lvz-viz/blob/master/src/main/java/de/codefor/le/repositories/PoliceTickerRepository.java"
    click Model "https://github.com/codeforleipzig/lvz-viz/blob/master/src/main/java/de/codefor/le/model/PoliceTicker.java"
    click Utils "https://github.com/codeforleipzig/lvz-viz/blob/master/src/main/java/de/codefor/le/utilities/Utils.java"
    click PropsDev "https://github.com/codeforleipzig/lvz-viz/blob/master/src/main/resources/application-dev.properties"
    click PropsLocal "https://github.com/codeforleipzig/lvz-viz/blob/master/src/main/resources/application-local.properties"
    click PropsTest "https://github.com/codeforleipzig/lvz-viz/blob/master/src/main/resources/application-test.properties"
    click PropsDefault "https://github.com/codeforleipzig/lvz-viz/blob/master/src/main/resources/application.properties"
    click MappingJSON "https://github.com/codeforleipzig/lvz-viz/blob/master/src/main/resources/mapping.json"
    click LegacyUI "https://github.com/codeforleipzig/lvz-viz/tree/master/src/main/resources/public/"
    click Dockerfile "https://github.com/codeforleipzig/lvz-viz/tree/master/Dockerfile"
    click DockerfileDev "https://github.com/codeforleipzig/lvz-viz/blob/master/Dockerfile.dev"
    click ComposeAll "https://github.com/codeforleipzig/lvz-viz/blob/master/docker-compose.yml"
    click ComposeDev "https://github.com/codeforleipzig/lvz-viz/blob/master/docker-compose.dev.yml"
    click ComposeProd "https://github.com/codeforleipzig/lvz-viz/blob/master/docker-compose.prod.yml"
    click GradleWrapper "https://github.com/codeforleipzig/lvz-viz/tree/master/gradlew"
    click BuildGradle "https://github.com/codeforleipzig/lvz-viz/blob/master/build.gradle"

    %% Styles
    classDef compute fill:#D6EAF8,stroke:#1B4F72,color:#154360
    classDef db fill:#F9E79F,stroke:#7D6608,color:#7D6608
    classDef cloud fill:#ABEBC6,stroke:#1D8348,color:#145A32
    classDef deployment fill:#F5EEF8,stroke:#7D3C98,color:#4A235A
```
