# yuchen413/anomalyruler architecture

> Architecture diagram of the GitHub repository yuchen413/anomalyruler, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yuchen413/anomalyruler
- Repository: https://github.com/yuchen413/anomalyruler

## Overview

##### Project type and purpose
- This is an open‐source research pipeline (ECCV’24) for video anomaly detection that leverages vision‐to‐text and large language models (LLMs) to induce and deduct rules for detecting anomalies in surveillance datasets (SHTech, UBNormal, ped2, avenue).
- The pipeline is orchestrated by Python scripts (image2text.py, main.py, majority_smooth.py, etc.) and depends on PyTorch, Transformers, OpenAI API, and supporting libraries.

##### File structure
Top‐level directories & files:
- LICENSE, README.md – project metadata
- SHTech, UBNormal, avenue, ped2 – dataset folders each containing train.csv, test.csv, and subfolders of frame CSVs and descriptions
- dataset.py – generic dataset loader
- image2text.py – “Visual Perception” module (frame → text)
- generate_choices.py, llm.py, openai_api.py – LLM interaction and candidate generation
- main.py – entry for Rule Generation (induct) and Robust Reasoning (deduct)
- majority_smooth.py – Perception Smoothing module
- ram2tag.py, reasoning_test.py – evaluation / tagging utilities
- rule/ – static rule templates per dataset
- utils.py – shared utilities
- run.sh – convenience wrapper

##### README for additional insights
- Pipeline is split into two stages: Induction (visual perception → rule generation → rule aggregation) and Deduction (visual perception → smoothing → robust reasoning).
- Four sequential steps invoked by command‐line scripts.
- Dependencies include deep learning, LLM libraries, OpenAI API, and data‐processing tools.

a. Components
1) Data Layer
– Raw frames & metadata (CSV) per dataset
– dataset.py: abstract loader
2) Induction Stage
– Visual Perception Module (image2text.py)
– LLM Interface (openai_api.py, llm.py)
– Rule Generation (main.py --induct + rule/rule_*.txt)
– Rule Aggregation (generate_choices.py + voting logic)
3) Deduction Stage
– Visual Perception Module (image2text.py)
– Perception Smoothing (majority_smooth.py)
– Robust Reasoning (main.py --deduct)
4) Evaluation / Utilities
– reasoning_test.py, ram2tag.py, utils.py
5) External Services
– OpenAI API (text ↔ LLM calls)

b. Relationships & data ﬂows
- Raw frames → dataset.py → image2text → text descriptions
- During induction: descriptions + rule templates → LLM → generated rules → voting → saved rule set
- During deduction: frames → image2text → smoothed descriptions → LLM reasoning using induced rules → final anomaly predictions
- Outputs: per‐frame anomaly labels written to CSVs under each dataset folder

c. Architectural patterns & principles
– Pipeline / staging architecture (induction vs. deduction)
– Modular separation: data loading, perception, rule handling, smoothing, reasoning
– External service integration via adapter (openai_api.py)
– Reusability: utils.py for common code, rule templates in rule/

d. Key technologies
– Python 3
– PyTorch, torchvision for frame processing
– Hugging Face Transformers for vision‐language models
– OpenAI API for LLM calls
– pandas, scikit‐learn for data handling and voting

##### Elements
- Clear labels on each module/component
- Blue for data/storage (datasets, CSVs)
- Green for processing modules (Python scripts)
- Orange for external services (OpenAI API)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Storage
    subgraph "Data Storage" 
        direction TB
        SHTech["SHTech/"]:::data
        UBNormal["UBNormal/"]:::data
        avenue["avenue/"]:::data
        ped2["ped2/"]:::data
    end

    %% Data Layer
    dataset["dataset.py"]:::processing

    %% Visual Perception
    image2text["image2text.py"]:::processing

    %% Induction Stage
    subgraph "Induction Stage" 
        direction TB
        induct_entry["main.py --induct"]:::processing
        openai_api["openai_api.py"]:::processing
        llm["llm.py"]:::processing
        rule_SHTech["rule/rule_SHTech.txt"]:::data
        rule_UBNormal["rule/rule_UBNormal.txt"]:::data
        rule_avenue["rule/rule_avenue.txt"]:::data
        rule_ped2["rule/rule_ped2.txt"]:::data
        generate_choices["generate_choices.py"]:::processing
        induced_rules["Induced Rule Set"]:::data
    end

    %% Deduction Stage
    subgraph "Deduction Stage"
        direction TB
        deduct_entry["main.py --deduct"]:::processing
        majority_smooth["majority_smooth.py"]:::processing
        reasoning["Robust Reasoning"]:::processing
        predictions["Anomaly Predictions (CSV)"]:::data
    end

    %% Evaluation & Utilities
    subgraph "Evaluation & Utilities"
        direction TB
        reasoning_test["reasoning_test.py"]:::processing
        ram2tag["ram2tag.py"]:::processing
        utils["utils.py"]:::processing
    end

    %% External Services
    OpenAI["OpenAI API"]:::external

    %% Flows
    SHTech --> dataset
    UBNormal --> dataset
    avenue --> dataset
    ped2 --> dataset

    dataset --> image2text

    %% Induction flow
    image2text --> induct_entry
    induct_entry --> openai_api
    openai_api --> llm
    llm --> generate_choices
    generate_choices --> induced_rules
    rule_SHTech --> induct_entry
    rule_UBNormal --> induct_entry
    rule_avenue --> induct_entry
    rule_ped2 --> induct_entry
    OpenAI --> openai_api

    %% Deduction flow
    image2text --> majority_smooth
    majority_smooth --> deduct_entry
    deduct_entry --> reasoning
    reasoning --> predictions
    induced_rules -.-> deduct_entry
    OpenAI --> deduct_entry

    %% Evaluation
    predictions --> reasoning_test
    predictions --> ram2tag
    utils -.-> induct_entry
    utils -.-> deduct_entry

    %% Click events
    click dataset "https://github.com/yuchen413/anomalyruler/blob/master/dataset.py"
    click SHTech "https://github.com/yuchen413/anomalyruler/tree/master/SHTech/"
    click UBNormal "https://github.com/yuchen413/anomalyruler/tree/master/UBNormal/"
    click avenue "https://github.com/yuchen413/anomalyruler/tree/master/avenue/"
    click ped2 "https://github.com/yuchen413/anomalyruler/tree/master/ped2/"
    click image2text "https://github.com/yuchen413/anomalyruler/blob/master/image2text.py"
    click openai_api "https://github.com/yuchen413/anomalyruler/blob/master/openai_api.py"
    click llm "https://github.com/yuchen413/anomalyruler/blob/master/llm.py"
    click induct_entry "https://github.com/yuchen413/anomalyruler/blob/master/main.py"
    click rule_SHTech "https://github.com/yuchen413/anomalyruler/blob/master/rule/rule_SHTech.txt"
    click rule_UBNormal "https://github.com/yuchen413/anomalyruler/blob/master/rule/rule_UBNormal.txt"
    click rule_avenue "https://github.com/yuchen413/anomalyruler/blob/master/rule/rule_avenue.txt"
    click rule_ped2 "https://github.com/yuchen413/anomalyruler/blob/master/rule/rule_ped2.txt"
    click generate_choices "https://github.com/yuchen413/anomalyruler/blob/master/generate_choices.py"
    click majority_smooth "https://github.com/yuchen413/anomalyruler/blob/master/majority_smooth.py"
    click deduct_entry "https://github.com/yuchen413/anomalyruler/blob/master/main.py"
    click reasoning_test "https://github.com/yuchen413/anomalyruler/blob/master/reasoning_test.py"
    click ram2tag "https://github.com/yuchen413/anomalyruler/blob/master/ram2tag.py"
    click utils "https://github.com/yuchen413/anomalyruler/blob/master/utils.py"
    click run "https://github.com/yuchen413/anomalyruler/blob/master/run.sh"

    %% Styles
    classDef data fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef processing fill:#ccffcc,stroke:#333,stroke-width:1px
    classDef external fill:#ffddcc,stroke:#333,stroke-width:1px
```
