How prasro/security_log_analyzer works
Project type & purpose (what you’re diagramming)
This is a single-process Python CLI tool (not a distributed system) that:
Parses them into a structured representation,
3) Runs multiple threat detectors (brute force, SQLi, port scan, unauthorized access, etc.),
Aggregates findings and metrics,
5) Generates an HTML report using Jinja2 templates, likely with charts and tables (with Pandas used for analysis/aggregation).
File structure analysis (map folders → components)
From the tree:
• main.py
• Entry point / CLI orchestration. Accepts a log path and runs the pipeline.
• wrappers/
• “Glue” / orchestration layer (pipeline stages).
• wrappers/log_parser.py: log ingestion + parsing into structured events.
• wrappers/event_analyzer.py: likely runs detectors and aggregates results.
• wrappers/report_gen.py: prepares data for the report, renders HTML via template.
• detectors/
• Independent detection modules (core business logic).
• access_detect.py: unauthorized/sensitive resource access detection.
• brute_force_detect.py: brute-force & time-window brute-force patterns.
• sql_injection_detect.py: SQL injection pattern detection.
• port_scan.py: port scanning pattern detection.
• ip_attack_counts.py: shared aggregation utilities (counts per IP, etc.).
• templates/report_template.html
• Jinja2 HTML template used by report generator.
• sample_logs/test_log.log
• Example input.
• requirements.txt
• External dependencies (explicitly: Pandas, Jinja2; possibly chart libs).
Architecturally, the key boundary is:
wrappers = pipeline & integration, detectors = pluggable analytical modules, templates = presentation, main = entrypoint.
What the README implies (important runtime relationships)
• Multiple threat types → detectors likely run in parallel conceptually (or sequentially in code) over the same parsed event stream / dataframe.
• HTML report “interactive” with charts → there’s a render stage that transforms results into report-ready structures and injects them into report_template.html.
• Pandas → likely an internal canonical data structure is a DataFrame (or at least intermediate tabular data).
CLI / Entry Point
• main.py
Log Ingestion & Parsing
• wrappers/log_parser.py
• Input: *.log file path (e.g., sample_logs/test_log.log)
• Output: “Parsed Events” (list/dict) and/or Pandas DataFrame
Event Analysis Orchestrator
• wrappers/event_analyzer.py
• Responsibility: call detectors, merge/normalize outputs, compute summaries
Detection Modules (Pluggable Detectors)
• detectors/brute_force_detect.py
• detectors/sql_injection_detect.py
• detectors/access_detect.py
• detectors/port_scan.py
• detectors/ip_attack_counts.py (utility / shared aggregation)
• Output: findings (alerts), per-IP stats, time-series stats, etc.
Report Generation
• wrappers/report_gen.py
• Inputs: aggregated findings + metrics
• Uses: templates/report_template.html (Jinja2)
• Output: report.html (or similar) as an artifact
• Pandas (data manipulation)
• Jinja2 (templating)
Output Artifact
• “HTML Security Report” (file)
A clean baseline flow:
main.py
→ calls Log Parser (wrappers/log_parser.py)
→ returns Parsed Events / DataFrame
Parsed Events / DataFrame
→ passed into Event Analyzer (wrappers/event_analyzer.py)
Event Analyzer
→ collects Detector Findings (alerts + metrics)
Aggregated Findings
→ passed to Report Generator (wrappers/report_gen.py)
Report Generator
→ reads Jinja2 Template (templates/report_template.html)
→ uses Jinja2 to render
→ produces HTML Report artifact
Additionally:
Architectural patterns / design principles
• Pipeline architecture: Parse → Detect → Aggregate → Render.
• Plugin-like modular detection: each detector is an independent module under detectors/ (easy to extend by adding a new detector file and wiring it in the analyzer).
• Separation of concerns:
• Parsing isolated from detection
• Detection isolated from presentation
• Presentation driven by a template
Relevant technologies
• Python 3.8+
• Pandas (tabular computation / grouping / aggregation)
• Jinja2 (HTML templating)
• (Optional) “Filesystem I/O” for logs + report output
“Separate into as many components as possible” (without overthinking)
To be maximally detailed for this repo, split into these subcomponents inside the pipeline:
• Log Parser
• “Reader” (filesystem open/read)
• “Tokenizer/Regex parser” (extract IP, timestamp, endpoint, status, port, etc.)
• “Normalizer” (standard event schema)
• Analyzer
• “Detector runner” (invokes detectors)
• “Result normalizer” (common schema for alerts)
• “Aggregator” (summary tables, per-IP counts, time buckets)
• Detectors
• Brute force detection
• Time-window brute force detection (if distinct from brute force; README implies both)
• SQL injection detection
• Unauthorized access detection
• Port scanning detection
• Shared IP metrics utility (ip_attack_counts.py)
• Report Generator
• “Data preparation for template” (tables + chart series)
• “Template render” (Jinja2)
• “Write HTML output”
• Shows runtime pipeline from log file to HTML report, with detectors branching off the analyzer stage.
• main.py depends on wrappers/*
• wrappers/event_analyzer.py depends on detectors/*
• wrappers/report_gen.py depends on templates/report_template.html, Jinja2, and possibly Pandas