How satvik-sharma-05/ulpf works
ULPF normalizes heterogeneous logs into a shared schema, preserves their raw bytes, and stores records in Neo4j for investigation. Users can upload logs or preview parsing in Parser Lab, inspect analytics and alerts, or ask questions through the chat interface. Batch and Kafka-based ingestion are distinct entry paths; analytics and chatbot services read the graph for different forms of exploration. README documents parser synthesis, OCSF/ECS exports, and a tamper-evident ledger; the sampled excerpts do not establish every API-to-module call, so uncertain wiring is omitted.
Main entry points: parser.py Log parser · consumer_service.py Realtime consumer · run_pipeline.py Batch ingestion · chatbot.py Chat orchestration
Components 29
Log ingestion
- Ingestion API
app/backend/ingestion_pipeline/api.py - Batch ingestion
app/backend/ingestion_pipeline/batch/run_pipeline.py - Realtime consumer
app/backend/ingestion_pipeline/realtime/consumer_service.py
Normalization and trust
- Log parser
app/backend/ingestion_pipeline/core/parser.py - Parser registry
app/backend/ingestion_pipeline/core/parsers/registry.py - Generic fallback
app/backend/ingestion_pipeline/core/parsers/generic_fallback.py - Common schema
app/backend/ingestion_pipeline/core/graph_schema.py - Log embeddings
app/backend/ingestion_pipeline/core/embeddings.py - Tamper ledger
app/backend/analytics_pipeline/ledger/chain.py - Merkle proofs
app/backend/analytics_pipeline/ledger/merkle.py
Investigation services
- Analytics API
app/backend/analytics_pipeline/api.py - Analytics queries
app/backend/analytics_pipeline/queries.py - Export formats
app/backend/analytics_pipeline/exporters.py - Chat API
app/backend/chatbot_pipeline/api.py - Chat orchestration
app/backend/chatbot_pipeline/chatbot.py - Graph retrieval
app/backend/chatbot_pipeline/graph_rag.py - Text to Cypher
app/backend/chatbot_pipeline/text_to_cypher.py
User experience
- Web application
app/frontend/src/App.jsx - Parser Lab
app/frontend/src/pages/ParserLabPage.jsx - Chat experience
app/frontend/src/pages/ChatPage.jsx - Analytics views
app/frontend/src/pages/AnalyticsPage.jsx - Upload client
app/frontend/src/ingestApi.js - Analytics client
app/frontend/src/analyticsApi.js - Chat client
app/frontend/src/api.js
Other components
- Log engineer
- Raw logs
- Kafka
- Log graph
- Local AI services
Connections 35
- Log engineer → Web application: uses
- Raw logs → Batch ingestion: supplies logs
- Raw logs → Kafka: publishes raw events
- Kafka → Realtime consumer: delivers events
- Web application → Upload client: uploads or previews
- Upload client → Ingestion API: calls
- Ingestion API → Log parser: parses preview
- Batch ingestion → Log parser: parses logs
- Realtime consumer → Log parser: parses events
- Log parser → Parser registry: detects format
- Log parser → Generic fallback: falls back
- Log parser → Common schema: normalizes records
- Batch ingestion → Log embeddings: generates vectors
- Realtime consumer → Log embeddings: embeds events
- Batch ingestion → Log graph: writes records
- Realtime consumer → Log graph: writes records
- Realtime consumer → Kafka: publishes parsed events
- Log graph → Tamper ledger: anchors records
- Tamper ledger → Merkle proofs: builds proofs
- Analytics views → Analytics client: requests views
- Analytics client → Analytics API: calls
- Analytics API → Analytics queries: runs aggregations
- Analytics queries → Log graph: reads graph
- Analytics API → Export formats: streams exports
- Chat experience → Chat client: sends questions
- Chat client → Chat API: calls
- Chat API → Chat orchestration: dispatches request
- Chat orchestration → Text to Cypher: plans graph query
- Chat orchestration → Graph retrieval: retrieves evidence
- Text to Cypher → Log graph: queries graph
- Graph retrieval → Log graph: retrieves records
- Chat orchestration → Local AI services: uses offline AI
- Parser Lab → Upload client: requests parse preview
- Web application → Analytics views: shows
- Web application → Chat experience: shows