# tsanford01/superlogs architecture

> Architecture diagram of the GitHub repository tsanford01/superlogs, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tsanford01/superlogs
- Repository: https://github.com/tsanford01/superlogs

## Overview

##### Project Type & Purpose
- SuperLogs is a standalone Python application offering both CLI and GUI (tkinter‐based) for self-learning log analysis.
- It ingests log files, learns regex patterns, evolves them over time, and produces HTML reports and Markdown documentation.

##### Top-Level Components
a. Presentation Layer
- CLI (superlogs/cli.py, __main__.py, entry points: `superlogs` command, .bat/.ps1 wrappers)
- GUI (superlogs/gui.py using tkinter)
b. Core Orchestration
- Analyzer (superlogs/analysis/*.py) – drives end-to-end analysis and correlation
- Scanner (superlogs/core/scanner.py) – efficient file reader (gzip, plain text)
- Parser (superlogs/parsing/log_format_parser.py) – breaks logs into structured tokens
- Learner (superlogs/core/learner.py & superlogs/learning/*.py) – clusters error lines, generates/refines patterns
- Pattern Manager (superlogs/patterns/*.py) – stores, retrieves, tags, and shares regex patterns
- Config Module (superlogs/config/*.py) – centralized parsing of sample_config YAML and CLI flags
- Health & Upgrade (superlogs/health/*.py, superlogs/upgrade/*.py) – specialized analyses for deployment health and upgrades
c. Reporting & Documentation
- Deployment Reporter (superlogs/reporting/deployment_reporter.py) – HTML dashboards
- Auto-Documentation (superlogs/learning/documenter.py) – generates Markdown for each pattern
d. Data Storage
- Patterns JSON (data/patterns/) & cache (data/cache/)
- Sample configurations (sample_config/)
- Tag library & correlations (data/tags/)
e. Utilities & Scripts
- Helper functions (superlogs/utils/*.py)
- Shell wrappers & installation scripts (install.bat, SuperLogs.ps1, setup.py)
- Extraction & analysis automation (extract_and_analyze.ps1)

##### Component Interactions & Data Flow
- User invokes via CLI or GUI → Config Module loads settings → Orchestrator triggers Scanner → Parser tokenizes logs
- Parsed records → passed to Analyzer & Correlation Engine → Learner clusters events → Pattern Generation
- New/updated patterns → persisted to data/patterns, indexed by Pattern Manager → feedback loop for Pattern Evolution
- Analysis results & pattern stats → pushed to Reporter (HTML) & Documenter (Markdown) → output to filesystem
- Tag Manager enriches analysis by symptom tags (connectivity, performance, FIM, upgrade, etc.)

##### Architectural Patterns & Principles
- Modular Package Layout (separation by function: config, core, parsing, analysis, learning, patterns, reporting, gui, cli)
- Pipeline / Orchestrator Pattern (scan → parse → analyze/learn → report)
- Plugin-style Config Parsers (each log type or analyzer registers a parser/analyzer in superlogs/config)
- Data-Driven Design (JSON pattern store, YAML config)
- Single Process, Layered Architecture (Presentation → Application Services → Domain Logic → Data)

##### Essential Elements to Capture
- CLI vs GUI entry points
- Orchestration pipeline and modules invoked at each stage
- Pattern library as central shared datastore
- Reporting/documentation outputs
- Config parsing plugins
- Tag-based analysis overlay

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Presentation Layer
    subgraph "Presentation Layer"
        direction TB
        CLI["CLI Entry Point"]:::presentation
        Main["CLI Main Module"]:::presentation
        Bat["Windows Wrapper (BAT)"]:::presentation
        PS1["Windows Wrapper (PS1)"]:::presentation
        GUI["GUI (tkinter)"]:::presentation
    end

    %% Core Services
    subgraph "Core Services"
        direction TB
        Config["Config Parsers"]:::core
        Scanner["Scanner"]:::core
        Parser["Log Format Parser"]:::core
        Orchestrator["Analyzer Orchestrator"]:::core
        subgraph "Analysis Engines"
            direction TB
            CorrEng["Correlation Engine"]:::core
            FlowEng["Data Flow Analyzer"]:::core
        end
        Learner["Core Learner"]:::core
        subgraph "Pattern Management"
            direction TB
            Library["Pattern Library"]:::core
            PatternModel["Pattern Model"]:::core
            TagMgr["Tag Manager"]:::core
            Workflows["Troubleshooting Workflows"]:::core
        end
    end

    %% Reporting & Documentation
    subgraph "Reporting & Documentation"
        direction TB
        Reporter["Deployment Reporter (HTML)"]:::core
        DocGen["Auto-Documentation"]:::core
    end

    %% Health & Upgrade
    subgraph "Health & Upgrade"
        direction TB
        HealthCheck["Health Checker"]:::core
        DepHealth["Deployment Health Analyzer"]:::core
        Upgrade["Upgrade Analyzer"]:::core
    end

    %% Learning Modules
    subgraph "Learning Modules"
        direction TB
        CompDet["Component Detector"]:::core
    end

    %% Utilities & Scripts
    subgraph "Utilities & Scripts"
        direction TB
        Utils["Log File Utilities"]:::utils
        Extract["extract_and_analyze.ps1"]:::utils
        InstallBat["install.bat"]:::utils
        Shortcut["create_shortcuts.ps1"]:::utils
        Setup["setup.py"]:::utils
        Req["requirements.txt"]:::utils
    end

    %% Data Storage
    subgraph "Data Storage"
        direction TB
        PatternsStore["Patterns Store"]:::data
        Cache["Cache"]:::data
        TagsStore["Tag Library"]:::data
        SampleConf["Sample Config Files"]:::data
    end

    %% External
    subgraph "External Interfaces"
        direction TB
        FS["File System (Logs & Outputs)"]:::external
        ExtTools["External Tools"]:::external
        PyRuntime["Python Runtime & Dependencies"]:::external
    end

    %% Flow
    CLI -->|invoke| Config
    Main -->|invoke| Config
    Bat -->|wrapper| Main
    PS1 -->|wrapper| Main
    GUI -.->|trigger| Orchestrator
    Config -->|loads settings| Scanner
    Scanner -->|reads logs| Parser
    Parser -->|tokenizes| Orchestrator
    Orchestrator -->|calls| CorrEng
    Orchestrator -->|calls| FlowEng
    CorrEng -->|feeds events| Learner
    FlowEng -->|feeds events| Learner
    Learner -->|generates patterns| Library
    Learner -->|generates patterns| PatternModel
    Learner -->|tags patterns| TagMgr
    Learner -->|uses workflows| Workflows
    Library -->|persist| PatternsStore
    PatternModel -->|persist| PatternsStore
    TagMgr -->|persist| TagsStore
    Workflows -->|persist| PatternsStore
    Orchestrator -->|analysis results| Reporter
    DocGen -->|generate docs| FS
    Reporter -->|generate HTML| FS
    Config -->|loads samples| SampleConf
    HealthCheck -->|analyzes health| Orchestrator
    DepHealth -->|analyzes deployment| Orchestrator
    Upgrade -->|analyzes upgrades| Orchestrator
    CompDet -->|detects components| Learner
    Utils -->|assist| Scanner
    Extract -->|automate| Orchestrator
    InstallBat -->|install| FS
    Shortcut -->|install| FS
    Setup -->|setup| FS
    Req -->|dependencies| PyRuntime
    FS -->|stores| SampleConf
    ExtTools -->|import/export| PatternsStore
    PyRuntime -->|hosts| GUI
    PyRuntime -->|hosts| CLI

    %% Click Events
    click CLI "https://github.com/tsanford01/superlogs/blob/master/superlogs/cli.py"
    click Main "https://github.com/tsanford01/superlogs/blob/master/superlogs/__main__.py"
    click Bat "https://github.com/tsanford01/superlogs/blob/master/superlogs.bat"
    click PS1 "https://github.com/tsanford01/superlogs/blob/master/SuperLogs.ps1"
    click GUI "https://github.com/tsanford01/superlogs/blob/master/superlogs/gui.py"
    click Scanner "https://github.com/tsanford01/superlogs/blob/master/superlogs/core/scanner.py"
    click Parser "https://github.com/tsanford01/superlogs/blob/master/superlogs/parsing/log_format_parser.py"
    click Orchestrator "https://github.com/tsanford01/superlogs/blob/master/superlogs/core/analyzer.py"
    click CorrEng "https://github.com/tsanford01/superlogs/blob/master/superlogs/analysis/correlation_engine.py"
    click FlowEng "https://github.com/tsanford01/superlogs/blob/master/superlogs/analysis/data_flow_analyzer.py"
    click Learner "https://github.com/tsanford01/superlogs/blob/master/superlogs/core/learner.py"
    click CompDet "https://github.com/tsanford01/superlogs/blob/master/superlogs/learning/component_detector.py"
    click DocGen "https://github.com/tsanford01/superlogs/blob/master/superlogs/learning/documenter.py"
    click Library "https://github.com/tsanford01/superlogs/blob/master/superlogs/patterns/library.py"
    click PatternModel "https://github.com/tsanford01/superlogs/blob/master/superlogs/patterns/pattern.py"
    click TagMgr "https://github.com/tsanford01/superlogs/blob/master/superlogs/patterns/tag_manager.py"
    click Workflows "https://github.com/tsanford01/superlogs/blob/master/superlogs/patterns/troubleshooting_workflows.py"
    click Config "https://github.com/tsanford01/superlogs/tree/master/superlogs/config/"
    click HealthCheck "https://github.com/tsanford01/superlogs/blob/master/superlogs/health/checker.py"
    click DepHealth "https://github.com/tsanford01/superlogs/blob/master/superlogs/health/deployment_health.py"
    click Upgrade "https://github.com/tsanford01/superlogs/blob/master/superlogs/upgrade/analyzer.py"
    click Reporter "https://github.com/tsanford01/superlogs/blob/master/superlogs/reporting/deployment_reporter.py"
    click PatternsStore "https://github.com/tsanford01/superlogs/tree/master/data/patterns/"
    click Cache "https://github.com/tsanford01/superlogs/tree/master/data/cache/"
    click TagsStore "https://github.com/tsanford01/superlogs/tree/master/data/tags/"
    click SampleConf "https://github.com/tsanford01/superlogs/tree/master/sample_config/"
    click Utils "https://github.com/tsanford01/superlogs/blob/master/superlogs/utils/log_file_utils.py"
    click Extract "https://github.com/tsanford01/superlogs/blob/master/extract_and_analyze.ps1"
    click InstallBat "https://github.com/tsanford01/superlogs/blob/master/install.bat"
    click Shortcut "https://github.com/tsanford01/superlogs/blob/master/create_shortcuts.ps1"
    click Setup "https://github.com/tsanford01/superlogs/blob/master/setup.py"
    click Req "https://github.com/tsanford01/superlogs/blob/master/requirements.txt"

    %% Styles
    classDef presentation fill:#D0E6FF,stroke:#009,stroke-width:1px;
    classDef core fill:#DFF2BF,stroke:#080,stroke-width:1px;
    classDef data fill:#FFF2CC,stroke:#CC0,stroke-width:1px;
    classDef utils fill:#F0F0F0,stroke:#666,stroke-width:1px;
    classDef external fill:#EEEEEE,stroke:#999,stroke-width:1px;
```
