# devadripta/agentic-training-copilot architecture

> Architecture diagram of the GitHub repository devadripta/agentic-training-copilot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-02-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/devadripta/agentic-training-copilot
- Repository: https://github.com/devadripta/agentic-training-copilot

## Overview

This repo is a **single-file, CLI-based observability tool** for deep learning training runs. It **does not run as a service** and has **no UI**; it attaches to **training log files**, interprets them via a **deterministic state machine**, predicts **ETA**, and emits **narrative alerts** to **Console and optionally Telegram**.

From the file tree + README, the core “system” is essentially:
- A **local Python runtime** (`python copilot.py ...`)
- Reading from the **filesystem** (log discovery + tailing)
- Optional outbound calls to **Telegram Bot API** via `requests`

##### What the file tree implies architecturally
Top-level:
- `copilot.py` — the primary executable (v2.1 per README)
- `README.md`, `QUICKSTART.md` — documentation, usage contracts (CLI flags / env vars)

##### Main components to show (decompose aggressively)
Use the README’s architecture block as your baseline, then expand it into explicit subcomponents:

##### Entry & Configuration
- **CLI Entrypoint**: `python copilot.py`
- **Argument parsing**: `--auto`, `--log`, `--wait`, `--base-dir`, `--from-start`, `--token`, `--chat`
- **Config sources**:
- CLI flags
- Environment variables (`TELEGRAM_TOKEN`, `TELEGRAM_CHAT`)

##### Log Discovery & Attachment Layer (platform-agnostic)
- **LogDiscovery**
- Builds `search_paths` (current dir + common dirs + optional platform hints)
- Recursive glob search (`**/*.log`)
- Filters by heuristics:
- size > 1KB
- contains training indicators (“epoch”, “loss”, “train”)
- Picks “best candidate” by recency/fitness
- **Wait Mechanism**
- If no log exists yet, sleeps and retries up to `--wait` seconds

##### Log Reader / Tailer
- **File tailing loop**
- Follow appended lines (like `tail -f`)
- Optional “read from beginning” (`--from-start`)
- Produces a stream of new log lines/events into the pipeline

##### Parser (Extract structured signals)
- **MMEngine Log Parser**
- Extracts metrics: loss, lr, val mAP, epoch/iter counters, timestamps
- Detects anomalies: NaN/Inf, exploding loss patterns (for DIVERGING)

##### Training State Machine (Interpretation)
- **Phase/state classifier**
- States listed in README: INITIALIZING, WARMUP, LEARNING_FAST, STABLE_CONVERGENCE, PLATEAU, OVERFITTING_RISK, DIVERGING, CONVERGED
- **Transition logic**
- Deterministic thresholds (improvement per epoch, divergence between train/val, etc.)
- Emits “phase transition” events with reason strings

##### ETA Engine (Predictive)
- **ETA prediction**
- EMA-smoothed progress rate
- Confidence band / qualitative confidence
- “Plateau-aware” ETA adjustments (per README narrative)

##### Decision / Recommendation Engine
- **Early termination recommendation**
- Probability-of-gain heuristics
- Savings estimate in GPU-hours (needs runtime assumptions or inferred epoch time)

##### Notifier / Output Adapters
- **Intelligent Notifier**
- Cooldown / spam suppression
- Event formatting: narrative messages (phase transitions, early stop, divergence)
- **Sinks**
- Console sink (stdout)
- Telegram sink (Telegram Bot API via `requests`)

##### External Systems (must be explicit
- **Training job / framework** (MMEngine) producing logs
- **Filesystem** where logs are written
- **Telegram Bot API** (external SaaS endpoint)

##### CLI/Config decides mode
- `--log` explicit path OR `--auto` discovery

##### Control plane (configuration & gating)
- CLI flags/env vars configure:
- search locations (`--base-dir`)
- waiting (`--wait`)
- reading mode (`--from-start`)
- notification endpoints (`--token`, `--chat`)
- cooldown behavior (if implemented)

##### Architectural patterns to highlight (what makes this project “itself”)
- **Single-file deployable monolith** with **internal modular boundaries**
- **Pipeline architecture** (Reader → Parser → Interpreters → Notifier)
- **State machine** as the interpretability primitive (key innovation)
- **Adapter pattern** for notification sinks (Console, Telegram)
- **Platform-agnostic filesystem discovery** (no hard dependency on Kaggle/Colab APIs)

- User
→ (runs) Copilot CLI (`copilot.py`)
- Config (CLI/env)
- LogDiscovery (+ Wait loop)
- Tailer (follow file)
- Parser (MMEngine patterns)
- State Machine (8 phases)
- ETA Engine (EMA + confidence)
- Decision Engine (early stop/overfit/diverge)
- Notifier (cooldown + formatting)
- Console sink
- Telegram sink (`requests`)
↔ Filesystem (log file)
- Training job (MMEngine) → Filesystem
- Copilot → Telegram Bot API → User phone

## Diagram (Mermaid)

```mermaid
flowchart TD
  %% =========================
  %% Diagram 1 — System Context
  %% =========================
  subgraph "Diagram 1 — System Context"
    U["User (ML engineer)"]:::external

    TR["Training Process (MMEngine / training script)"]:::external
    FS[("Filesystem\n(log directories + .log files)")]:::storage
    TG["Telegram Bot API"]:::external

    CP["Agentic Training Observability\n(CLI: copilot.py)"]:::app

    TR -->|"writes Training logs"| FS
    U -->|"runs CLI command"| CP
    CP -->|"reads/tails Log file"| FS
    CP -->|"prints Narrative alerts"| U
    CP -->|"sends Notification message"| TG
    TG -->|"pushes Alert to user device"| U
  end

  %% ==========================================
  %% Diagram 2 — Container/Runtime (single proc)
  %% ==========================================
  subgraph "Diagram 2 — Container/Runtime View (Single Python Process)"
    subgraph "Python Process"
      CLI["CLI Entrypoint (v2.1)\nargparse + mode selection"]:::entry
      CFG["Config Resolution\nCLI flags + env vars\n(TELEGRAM_TOKEN/CHAT)"]:::entry

      DISC["LogDiscovery\nsearch_paths + heuristics\nrecursive glob + select best"]:::core
      WAIT["Wait/Retry Loop\nsleep + retry up to --wait"]:::core
      TAIL["Log Tailer\nfollow appended lines\n--from-start support"]:::core

      PARSE["MMEngine Log Parser\nmetrics extraction + anomaly detect"]:::core

      SM["Deterministic Training State Machine\n8 phases + transitions w/ reasons"]:::state
      ETA["ETA Engine\nEMA rate + confidence\nplateau-aware adjustments"]:::core
      DEC["Decision/Recommendation Engine\nearly-stop + savings estimate"]:::decision

      NOTIF["Intelligent Notifier\ncooldown + narrative formatting"]:::decision
      CONS["Console Sink\nstdout"]:::sink
      TELE["Telegram Sink\nrequests -> Bot API"]:::sink

      LEG["Alternative/Legacy Entrypoint\n(agentic_copilot_v2.py)"]:::variant
    end

    FS2[("Filesystem\n(log files)")]:::storage
    TG2["Telegram Bot API"]:::external
  end

  %% Data plane (solid)
  CLI -->|"selects mode (--log or --auto)"| DISC
  DISC -->|"log file path (or none)"| WAIT
  WAIT -->|"log file path"| TAIL
  TAIL -->|"log lines"| PARSE
  PARSE -->|"metric events"| SM
  PARSE -->|"progress events"| ETA
  SM -->|"phase + transition events"| DEC
  ETA -->|"ETA estimate + confidence"| DEC
  DEC -->|"recommendations (early stop/diverge)"| NOTIF
  NOTIF -->|"formatted alert"| CONS
  NOTIF -->|"formatted alert"| TELE
  TELE -->|"sendMessage"| TG2

  %% External I/O
  DISC -->|"search/scan directories"| FS2
  TAIL -->|"read/seek/follow file"| FS2

  %% Control plane (dashed)
  CLI -.->|"parses flags"| CFG
  CFG -.->|"--base-dir, --auto"| DISC
  CFG -.->|"--wait"| WAIT
  CFG -.->|"--from-start"| TAIL
  CFG -.->|"--token/--chat + env"| TELE
  CFG -.->|"cooldown policy"| NOTIF

  %% ===================================
  %% Diagram 3 — Sequence (common paths)
  %% ===================================
  subgraph "Diagram 3 — Sequence (Common + Exception Paths)"
    direction TB

    subgraph "Common Path: Auto-discover → Tail → Interpret → Notify"
      direction TB
      S0["Start: python copilot.py --auto"]:::seq
      S1["Build search_paths + recursive glob **/*.log"]:::seq
      S2{"Found candidate log?"}:::seqGate
      S3["Attach to best log (recency/fitness)"]:::seq
      S4["Tail loop: read new lines"]:::seq
      S5["Parse: extract loss/lr/mAP/epoch/iter\n+ detect NaN/Inf patterns"]:::seq
      S6["Update State Machine: classify phase\n+ emit transitions"]:::seq
      S7["Compute ETA: EMA rate + confidence\n(plateau-aware)"]:::seq
      S8["Decide: recommend early stop/diverge/overfit risk"]:::seq
      S9["Notify: cooldown + narrative message"]:::seq
      S10["Sinks: console + optional Telegram send"]:::seq

      S0 --> S1 --> S2
      S2 -->|"yes"| S3 --> S4 --> S5 --> S6 --> S7 --> S8 --> S9 --> S10
    end

    subgraph "Exception Path: No log yet → Wait → Timeout exit"
      direction TB
      E1["No log found initially"]:::seqAlt
      E2["Wait: sleep + retry until --wait seconds"]:::seqAlt
      E3{"Log appears before timeout?"}:::seqGate
      E4["Attach + continue common path"]:::seqAlt
      E5["Exit gracefully\n(print suggestion to pass --log or --base-dir)"]:::seqAlt

      E1 --> E2 --> E3
      E3 -->|"yes"| E4
      E3 -->|"no"| E5
    end
  end

  %% =========================
  %% Click Events (code paths)
  %% =========================
  click CLI "https://github.com/devadripta/agentic-training-copilot/blob/main/copilot.py"
  click CFG "https://github.com/devadripta/agentic-training-copilot/blob/main/copilot.py"
  click DISC "https://github.com/devadripta/agentic-training-copilot/blob/main/copilot.py"
  click WAIT "https://github.com/devadripta/agentic-training-copilot/blob/main/copilot.py"
  click TAIL "https://github.com/devadripta/agentic-training-copilot/blob/main/copilot.py"
  click PARSE "https://github.com/devadripta/agentic-training-copilot/blob/main/copilot.py"
  click SM "https://github.com/devadripta/agentic-training-copilot/blob/main/copilot.py"
  click ETA "https://github.com/devadripta/agentic-training-copilot/blob/main/copilot.py"
  click DEC "https://github.com/devadripta/agentic-training-copilot/blob/main/copilot.py"
  click NOTIF "https://github.com/devadripta/agentic-training-copilot/blob/main/copilot.py"
  click CONS "https://github.com/devadripta/agentic-training-copilot/blob/main/copilot.py"
  click TELE "https://github.com/devadripta/agentic-training-copilot/blob/main/copilot.py"
  click LEG "https://github.com/devadripta/agentic-training-copilot/blob/main/agentic_copilot_v2.py"
  click CP "https://github.com/devadripta/agentic-training-copilot/blob/main/copilot.py"
  click S0 "https://github.com/devadripta/agentic-training-copilot/blob/main/copilot.py"
  click S10 "https://github.com/devadripta/agentic-training-copilot/blob/main/copilot.py"
  click E5 "https://github.com/devadripta/agentic-training-copilot/blob/main/copilot.py"

  %% Docs
  D1["README (Architecture + CLI contracts)"]:::doc
  D2["QUICKSTART (Run instructions)"]:::doc
  click D1 "https://github.com/devadripta/agentic-training-copilot/blob/main/README.md"
  click D2 "https://github.com/devadripta/agentic-training-copilot/blob/main/QUICKSTART.md"

  %% Connect docs lightly (context)
  D1 -.->|"documents usage + design"| CLI
  D2 -.->|"how to run"| CLI

  %% =========
  %% Styling
  %% =========
  classDef external fill:#E5E7EB,stroke:#6B7280,color:#111827,stroke-width:1px
  classDef storage fill:#F3F4F6,stroke:#4B5563,color:#111827,stroke-width:1px
  classDef app fill:#DBEAFE,stroke:#1D4ED8,color:#0F172A,stroke-width:1.5px

  classDef entry fill:#C7D2FE,stroke:#4338CA,color:#0F172A,stroke-width:1.5px
  classDef core fill:#BFDBFE,stroke:#1E40AF,color:#0F172A,stroke-width:1.5px
  classDef state fill:#FDBA74,stroke:#9A3412,color:#0F172A,stroke-width:2px
  classDef decision fill:#E9D5FF,stroke:#7E22CE,color:#0F172A,stroke-width:1.5px
  classDef sink fill:#DDD6FE,stroke:#5B21B6,color:#0F172A,stroke-width:1.5px
  classDef variant fill:#FDE68A,stroke:#B45309,color:#0F172A,stroke-width:1.5px

  classDef doc fill:#FEF9C3,stroke:#A16207,color:#0F172A,stroke-width:1px

  classDef seq fill:#DBEAFE,stroke:#1D4ED8,color:#0F172A,stroke-width:1px
  classDef seqAlt fill:#F3E8FF,stroke:#7E22CE,color:#0F172A,stroke-width:1px
  classDef seqGate fill:#E5E7EB,stroke:#374151,color:#111827,stroke-width:1px
```
