How gilbertossoares/anonimizer works
Project Type and Purpose
• This is a full-stack Python web application (Flask) for anonymizing documents (PDF, DOCX, TXT) by detecting and removing PII via Azure AI services.
• The user interface (HTML/CSS/JS) handles file uploads and status display; the backend (app.py + utils/) orchestrates storage, AI calls, conversion, anonymization, and result retrieval.
File Structure Analysis
• Top-level:
• app.py – Flask entry point; defines routes for upload, search, modify.
• requirements.txt – Python dependencies (Flask, Azure SDKs).
• startup.sh – launch script.
• .env (not in tree) – holds Azure credentials.
• Templates: Jinja2 HTML pages for the UI.
• utils/: core modules:
• anonimizer.py – text anonymization logic.
• converter.py – extract/convert text from various formats.
• signature.py – apply digital signatures or watermarks.
• azure_blob.py – upload/download to Azure Blob Storage.
Key README Insights
• Integrates multiple Azure services: Blob Storage, Form Recognizer, Computer Vision (including Custom Vision), Cognitive Search, Text Analytics, Data Lake.
• Workflow images define how a file flows through ingestion, PII detection, replacement, and delivery.
Main Components
a. User Interface Layer
• Web Browser – HTML/JS for file upload & result view.
• Static CDN (optional) – serves CSS/JS.
b. Application Server
• Flask App (app.py)
• Template Engine (Jinja2)
• Business Logic (utils/*.py)
c. Storage Layer
• Azure Blob Storage
– Input Container (raw uploads)
– Output Container (anonymized docs)
• (Optional) Azure Data Lake for long-term archival
d. AI Services Layer
• Azure Form Recognizer – OCR & field extraction
• Azure Text Analytics – PII detection
• Azure Computer Vision – image-based PII detection
• Azure Custom Vision – domain-specific PII models
• Azure Cognitive Search – index & search redacted content
e. Conversion & Post-processing
• converter.py (text extraction/format conversion)
• anonimizer.py (PII replacement)
• signature.py (digital signature, watermarking)
f. External Clients
• Admin tools (if any) calling REST endpoints
• Automated pipelines via Azure SDK/CLI
Relationships and Data Flow
1. User uploads a document via browser → Flask upload endpoint.
2. Flask streams file to Azure Blob (Input Container).
3. Flask triggers Form Recognizer/OCR → raw text.
4. Text passed to Text Analytics & Custom Vision for PII detection.
5. converter.py normalizes format (e.g., DOCX → text).
6. anonimizer.py replaces/redacts PII tokens.
7. signature.py applies signatures/watermarks.
8. Anonymized file saved back to Azure Blob (Output Container).
9. Flask returns download link to user (via templates/search.html).
10. (Optional) Index into Cognitive Search for metadata lookup.
Architectural Patterns & Design Principles
• Layered Architecture: clear separation UI → App Logic → AI Services → Storage.
• Single-process monolith (Flask) but logically modular via utils/.
• Environment-driven configuration (.env).
• Externalize state to Azure Storage—stateless app server.
• Asynchronous or synchronous service calls over HTTPS (Azure SDK).
• Frontend (blue): browser, static assets, templates.
• Backend (green): Flask app, utils modules.
• Storage (orange): Azure Blob/Data Lake.
• AI Services (purple): Form Recognizer, Text Analytics, Computer Vision, Search.
• SDK/API calls: Flask → Azure Services.
• Data flow: Blob storage ↔ Flask.