How promanczak-hub/anonimizator-v3 works
Project type and purpose
• This is a full-stack web application (“Anonimizator v3”) for PDF anonymization and schema extraction (Digital Twin JSON).
• Users upload or drop PDFs; the system processes them in two modes (“Unify” vs “Layout”), using AI (Vertex AI) and returns redacted PDFs or JSON.
File structure
• Top-level services orchestrated by docker-compose:
• frontend/ (React + Vite)
• backend/ (FastAPI + Celery)
• Redis (queue)
• PostgreSQL (database)
• watch folder for incoming PDFs
• backend/app/api: HTTP endpoints (documents, jobs, health)
• backend/app/services & workers:
• watcher.py monitors watch folder
• pdf_processor.py handles parsing/redaction
• vertex_ai.py invokes external AI
• Celery tasks (workers/tasks.py) for async processing
• frontend/src: React components and pages, API client module (api/client.js)
README insights
• Confirms tech stack: React 19 + Vite, FastAPI + Celery, PostgreSQL, Redis, Vertex AI
• Highlights two processing modes and “watch” folder concept
Main components
a. User & Browser (client)
b. Frontend Service (React/Vite)
c. Backend API Service (FastAPI + Uvicorn)
d. Message Broker (Redis queue)
e. Celery Worker Pool (async tasks)
f. Watch Folder (file drop area)
g. PDF Processor & AI Service (Vertex AI)
h. Database (PostgreSQL)
i. Storage (for original & processed PDFs, JSON)
j. External Services (Vertex AI)
Relationships & data flow
1. User → (HTTP) Frontend → uploads PDF or triggers processing
2. Frontend → (HTTP POST) → Backend API (/documents or /jobs)
3. Backend API
• writes metadata to PostgreSQL (models/document, job)
• publishes a Celery task to Redis
4. Celery Worker picks up task → watcher or tasks.py
• invokes pdf_processor.py to parse/redact
• calls vertex_ai.py for AI inference
• writes results (redacted PDF or Digital Twin JSON) to storage
• updates job status in PostgreSQL
5. Frontend polls or fetches job/document status via API
6. Frontend displays redacted PDF or JSON viewer
Architectural patterns & principles
• Separation of concerns: UI vs API vs background processing
• Event-driven / asynchronous: Celery + Redis for decoupling
• Microservice-style isolation: each component in its own Docker container
• Watcher pattern: auto-ingest of PDFs from file system
Additional touches
• Directories or modules next to each component
• Optional: show Docker Compose as an orchestration layer around all containers