How kshruthii/pii-detection-and-privacy-protection-system works
This is a local, policy-driven privacy proxy for unstructured objects. A client authenticates, uploads an object, and the proxy stores it before triggering Presidio analysis and saving PII spans. On retrieval, the verified identity selects a policy; the proxy sends the object, spans’ lookup key, and policy to the anonymizer, which returns transformed bytes without changing the stored original. Separate branches provide admin policy management and guarded LLM chat. The README documents the overall flow; several internal links are supported by excerpts, while unsampled implementation details are not inferred.
Main entry points: app.py HTTP proxy · pii_analyser.py PII processor · auth.py JWT authentication · policies.py Policy management API
Components 17
Proxy and identity
- HTTP proxy
proxy-server/app.py - JWT authentication
proxy-server/auth.py - Password and JWT helpers
security.py
PII analysis
- Local upload trigger
consumer_analyzer/consumer/kafka_consumer.py - PII processor
consumer_analyzer/pii_analyser/pii_analyser.py - Object text reader
consumer_analyzer/pii_analyser/pii_analyser.py - Presidio analyzer
consumer_analyzer/pii_analyser/pii_analyser.py
Policy and anonymization
- Policy management API
proxy-server/policies.py - Anonymizer gRPC server
anonymizer/anonymizer/server/server.py - Anonymizer request handler
anonymizer/anonymizer/server/handler.py - Policy transformer
anonymizer/anonymizer/pii_anonymizer.py
Local state
- Object files
proxy-server/app.py - PII metadata
consumer_analyzer/pii_analyser/pii_analyser.py - Users and policies
proxy-server/sqlite_db.py
LLM protection
- LLM prompt protection
proxy-server/app.py
Other components
- API client
- Ollama service
Connections 22
- API client → HTTP proxy: sends requests
README.md - HTTP proxy → JWT authentication: routes auth
proxy-server/app.py - JWT authentication → Password and JWT helpers: verifies tokens
proxy-server/auth.py - JWT authentication → Users and policies: reads credentials
proxy-server/auth.py - HTTP proxy → Object files: reads and writes
README.md - HTTP proxy → Local upload trigger: triggers analysis
proxy-server/app.py - Local upload trigger → PII processor: invokes processing
consumer_analyzer/consumer/kafka_consumer.py - PII processor → Object text reader: reads object text
consumer_analyzer/pii_analyser/pii_analyser.py - PII processor → Presidio analyzer: analyzes text
consumer_analyzer/pii_analyser/pii_analyser.py - PII processor → PII metadata: stores spans
consumer_analyzer/pii_analyser/pii_analyser.py - HTTP proxy → Users and policies: loads user policy
README.md - HTTP proxy → Anonymizer gRPC server: sends anonymize request
README.md - Anonymizer gRPC server → Anonymizer request handler: dispatches requests
anonymizer/anonymizer/server/server.py - Anonymizer request handler → PII metadata: reads spans
anonymizer/anonymizer/server/handler.py - Anonymizer request handler → Policy transformer: applies policy
anonymizer/anonymizer/server/handler.py - HTTP proxy → Policy management API: routes policy APIs
proxy-server/app.py - Policy management API → JWT authentication: checks admin role
proxy-server/policies.py - Policy management API → Users and policies: updates assignments
proxy-server/policies.py - HTTP proxy → LLM prompt protection: handles chat
proxy-server/app.py - LLM prompt protection → Presidio analyzer: detects prompt PII
proxy-server/app.py - LLM prompt protection → Anonymizer gRPC server: anonymizes prompt
proxy-server/app.py - LLM prompt protection → Ollama service: sends protected prompt
README.md