How codeforleipzig/lvz-viz works
Project Type and Purpose
• The backend is a Spring Boot service (Java/Gradle) with embedded crawling and NLP steps and an Elasticsearch datastore.
• The frontend is an Angular SPA (in experimental_web/) that fetches data via REST APIs.
File Structure
• Top‑level components:
– src/main/java/... — Spring Boot application, REST controller, crawler modules, NER module, repository interfaces.
– src/main/resources/public/ — legacy/static UI assets served by Spring Boot.
– experimental_web/ — modern Angular application with its own package.json, build, and e2e tests.
– docker-compose*.yml, Dockerfile, Gradle wrapper — containerization and build orchestration.
– application-*.properties — Spring profiles for dev/local/test/prod.
• Key sub‑modules in src/main/java:
– crawler/ — CrawlScheduler, police‑ticker crawlers, Nominatim geocoding.
– ner/ — Named‑entity recognition logic.
– repositories/ — Elasticsearch repository interfaces.
– web/ — REST controllers exposing JSON endpoints to UIs.
– utilities/ — shared helper classes.
• Notice use of Spring’s @Scheduled, dependency injection, and Spring Data Elasticsearch patterns.
Extract Architectural Insights from README
• Data store: Elasticsearch (spun up via docker-compose).
• Crawler is activated at startup and can be delayed or configured via Spring profiles or env vars.
• Deployment supports multi‑stage Docker builds and separate dev/prod compose files.
• Frontend build uses npm + Grunt; backend build uses Gradle.
Main Components & Their Relationships
a. Frontend
– Angular app (experimental_web/), built to static assets → served either by its own Node server in dev or by Spring Boot in prod.
– Fetches data via HTTP REST calls to the backend.
b. Backend (Spring Boot)
– REST API layer (PoliceTickerController)
– Crawler scheduler triggers glue code:
• LvzPoliceTickerCrawler / LvzPoliceTickerDetailViewCrawler scrape HTML pages
• NominatimAsker enriches items with geolocation
• NER extracts entities
– Repository layer writes models (PoliceTicker) into Elasticsearch via PoliceTickerRepository
c. Data Store
– Elasticsearch cluster (defined in docker-compose.yml)
d. External Services
– LVZ website (source)
– Nominatim geocoding service
e. DevOps / Deployment
– Docker / docker‑compose sets up backend + Elasticsearch (+ optional UI server)
– Spring profiles determine which property files are active
– Multi‑stage Dockerfiles for production builds
Architectural Patterns & Design Principles
• Layered architecture: Controller → Service (crawler/NER) → Repository → Data store
• Spring Boot’s auto‑configuration and profile‑based config
• Asynchronous scheduled jobs (crawler) decoupled from request handling
• Single Page Application pattern on the frontend
• Container‑based micro‑services style (backend + Elasticsearch; frontend can be separate)
Final Checklist
– [ ] Clear labels on every component
– [ ] Notes on Docker Compose orchestration and profile selection