How tsanford01/superlogs works
Project Type & Purpose
• SuperLogs is a standalone Python application offering both CLI and GUI (tkinter‐based) for self-learning log analysis.
• It ingests log files, learns regex patterns, evolves them over time, and produces HTML reports and Markdown documentation.
Top-Level Components
a. Presentation Layer
• CLI (superlogs/cli.py, __main__.py, entry points: superlogs command, .bat/.ps1 wrappers)
• GUI (superlogs/gui.py using tkinter)
b. Core Orchestration
• Analyzer (superlogs/analysis/*.py) – drives end-to-end analysis and correlation
• Scanner (superlogs/core/scanner.py) – efficient file reader (gzip, plain text)
• Parser (superlogs/parsing/log_format_parser.py) – breaks logs into structured tokens
• Learner (superlogs/core/learner.py & superlogs/learning/*.py) – clusters error lines, generates/refines patterns
• Pattern Manager (superlogs/patterns/*.py) – stores, retrieves, tags, and shares regex patterns
• Config Module (superlogs/config/*.py) – centralized parsing of sample_config YAML and CLI flags
• Health & Upgrade (superlogs/health/*.py, superlogs/upgrade/*.py) – specialized analyses for deployment health and upgrades
c. Reporting & Documentation
• Deployment Reporter (superlogs/reporting/deployment_reporter.py) – HTML dashboards
• Auto-Documentation (superlogs/learning/documenter.py) – generates Markdown for each pattern
d. Data Storage
• Patterns JSON (data/patterns/) & cache (data/cache/)
• Sample configurations (sample_config/)
• Tag library & correlations (data/tags/)
e. Utilities & Scripts
• Helper functions (superlogs/utils/*.py)
• Shell wrappers & installation scripts (install.bat, SuperLogs.ps1, setup.py)
• Extraction & analysis automation (extract_and_analyze.ps1)
Component Interactions & Data Flow
• User invokes via CLI or GUI → Config Module loads settings → Orchestrator triggers Scanner → Parser tokenizes logs
• Parsed records → passed to Analyzer & Correlation Engine → Learner clusters events → Pattern Generation
• New/updated patterns → persisted to data/patterns, indexed by Pattern Manager → feedback loop for Pattern Evolution
• Analysis results & pattern stats → pushed to Reporter (HTML) & Documenter (Markdown) → output to filesystem
• Tag Manager enriches analysis by symptom tags (connectivity, performance, FIM, upgrade, etc.)
Architectural Patterns & Principles
• Modular Package Layout (separation by function: config, core, parsing, analysis, learning, patterns, reporting, gui, cli)
• Pipeline / Orchestrator Pattern (scan → parse → analyze/learn → report)
• Plugin-style Config Parsers (each log type or analyzer registers a parser/analyzer in superlogs/config)
• Data-Driven Design (JSON pattern store, YAML config)
• Single Process, Layered Architecture (Presentation → Application Services → Domain Logic → Data)
Essential Elements to Capture
• CLI vs GUI entry points
• Orchestration pipeline and modules invoked at each stage
• Pattern library as central shared datastore
• Reporting/documentation outputs
• Config parsing plugins
• Tag-based analysis overlay