# ds-hub-sochi/animal_classification architecture

> Architecture diagram of the GitHub repository ds-hub-sochi/animal_classification, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ds-hub-sochi/animal_classification
- Repository: https://github.com/ds-hub-sochi/animal_classification

## Overview

##### Main System Components
- CLI Layer
– irbis_classifier/cli/...
– Sub-modules: data_addition, preprocessing, reports, training, saving, synthetic_data_generation, testing
- Core Library
– irbis_classifier/src/...
– Sub-packages: models, training, testing, synthetic_data, utils
- Configuration Store
– configs/*.json
- Data Storage
– data/raw, data/interim, data/processed, data/external
- Models Output
– models/ (traced model artifacts)
- Reports Output
– reports/figures (generated plots)
- Tests
– tests/ (unit tests and assets)

##### Relationships and Data Flows
a. Data Ingestion & Augmentation
– Raw data (data/raw) → CLI/data_addition → interim
– Synthetic data generator can inject into data/raw or interim
b. Preprocessing Pipeline
– interim → filter_broken_images → filter_duplicates → sample_from_long_series → train/val/test split → match_splits_with_markup → processed
c. Reporting
– processed (and intermediate stages) → CLI/reports → reports/figures
d. Training
– processed/train.csv → CLI/training → Core training modules (datasets, trainer, experiment, losses, transforms) → saved model artifact
e. Saving/Exporting Model
– training output → CLI/saving → models/traced_model.pt
f. Testing/Evaluation
– processed/test.csv + traced_model.pt → CLI/testing → metrics, testers → CLI/reports or command-line output

##### Architectural Patterns & Design Principles
– Layered Architecture:
- Presentation/CLI layer on top
- Core domain modules (data processing, model training, evaluation)
- Infrastructure: file-system storage for data, models, reports
– Separation of Concerns:
- Each CLI command calls into a single-purpose library function
- Configurations are externalized into JSON files under configs/
– Extensibility:
- New CLI commands can be added under irbis_classifier/cli
- New model architectures via irbis_classifier/src/models/factory

##### Relevant Technologies & Frameworks
– Python 3.x
– PyTorch (models, trainer, tracing)
– pandas / numpy (data handling)
– matplotlib / seaborn (plots)
– pytest (unit tests)
– pre-commit, pylint, tox (linting & CI)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% CLI Layer
    subgraph "CLI Layer"
        direction TB
        DataAddition["Data Addition CLI"]:::cli
        PreprocessingCLI["Preprocessing CLI"]:::cli
        ReportsCLI["Reporting CLI"]:::cli
        TrainingCLI["Training CLI"]:::cli
        SavingCLI["Saving CLI"]:::cli
        SyntheticCLI["Synthetic Data CLI"]:::cli
        TestingCLI["Testing CLI"]:::cli
    end

    %% Core Library
    subgraph "Core Library"
        direction TB
        subgraph "Models"
            ModelsFactory["factory.py"]:::core
            ModelsUtils["utils.py"]:::core
        end
        subgraph "Training Modules"
            TrainingDatasets["datasets.py"]:::core
            TrainingExperiment["experiment.py"]:::core
            TrainingLosses["losses.py"]:::core
            TrainingTrainer["trainer.py"]:::core
            TrainingTransforms["transforms.py"]:::core
        end
        subgraph "Testing Modules"
            TestingMetrics["metrics.py"]:::core
            TestingTest["test.py"]:::core
            TestingTesters["testers.py"]:::core
        end
        subgraph "Synthetic Data Utils"
            SyntheticConfigs["confings.py"]:::core
        end
        GeneralUtils["utils.py"]:::core
    end

    %% Configuration Store
    Configs["Config Files"]:::config

    %% Data Storage
    RawData["data/raw"]:::data
    InterimData["data/interim"]:::data
    ProcessedData["data/processed"]:::data
    ExternalData["data/external"]:::data

    %% Outputs
    ModelsOutput["models/ (artifacts)"]:::output
    ReportsOutput["reports/figures"]:::output

    %% Tests
    TestsDir["tests/"]:::test

    %% Data flows
    RawData -->|add_images / add_kaluzskie / add_russian| DataAddition
    DataAddition -->|writes images| InterimData
    SyntheticCLI -->|injects synthetic| RawData
    SyntheticCLI -->|injects synthetic| InterimData

    InterimData -->|filter / split| PreprocessingCLI
    PreprocessingCLI -->|produces CSVs| ProcessedData

    ProcessedData -->|stats plots| ReportsCLI
    ReportsCLI --> ReportsOutput

    ProcessedData -->|train.csv| TrainingCLI
    TrainingCLI --> ModelsOutput
    TrainingCLI -->|uses| TrainingDatasets
    TrainingDatasets --> TrainingTrainer
    TrainingTrainer --> ModelsFactory
    TrainingTrainer --> TrainingTransforms
    TrainingTrainer --> TrainingExperiment
    TrainingExperiment --> TrainingLosses

    ProcessedData -->|test.csv| TestingCLI
    ModelsOutput --> TestingCLI
    TestingCLI -->|metrics| ReportsCLI

    %% Configs connections
    Configs --> DataAddition
    Configs --> PreprocessingCLI
    Configs --> SyntheticCLI
    Configs --> TrainingCLI
    Configs --> TestingCLI

    %% CLI into Core
    DataAddition --> GeneralUtils
    PreprocessingCLI --> GeneralUtils
    ReportsCLI --> GeneralUtils
    SyntheticCLI --> SyntheticConfigs
    TestingCLI --> TestingMetrics
    TrainingCLI --> TrainingTrainer

    %% Click events for CLI
    click DataAddition "https://github.com/ds-hub-sochi/animal_classification/tree/main/irbis_classifier/cli/data_addition/"
    click PreprocessingCLI "https://github.com/ds-hub-sochi/animal_classification/tree/main/irbis_classifier/cli/preprocessing/"
    click ReportsCLI "https://github.com/ds-hub-sochi/animal_classification/tree/main/irbis_classifier/cli/reports/"
    click TrainingCLI "https://github.com/ds-hub-sochi/animal_classification/blob/main/irbis_classifier/cli/training/start_training.py"
    click SavingCLI "https://github.com/ds-hub-sochi/animal_classification/blob/main/irbis_classifier/cli/saving/save_as_traced_model.py"
    click SyntheticCLI "https://github.com/ds-hub-sochi/animal_classification/tree/main/irbis_classifier/cli/synthetic_data_gereration/"
    click TestingCLI "https://github.com/ds-hub-sochi/animal_classification/blob/main/irbis_classifier/cli/testing/start_testing.py"

    %% Click events for Core Library
    click ModelsFactory "https://github.com/ds-hub-sochi/animal_classification/blob/main/irbis_classifier/src/models/factory.py"
    click ModelsUtils "https://github.com/ds-hub-sochi/animal_classification/blob/main/irbis_classifier/src/models/utils.py"
    click TrainingDatasets "https://github.com/ds-hub-sochi/animal_classification/blob/main/irbis_classifier/src/training/datasets.py"
    click TrainingExperiment "https://github.com/ds-hub-sochi/animal_classification/blob/main/irbis_classifier/src/training/experiment.py"
    click TestingMetrics "https://github.com/ds-hub-sochi/animal_classification/blob/main/irbis_classifier/src/testing/metrics.py"
    click SyntheticConfigs "https://github.com/ds-hub-sochi/animal_classification/blob/main/irbis_classifier/src/synthetic_data/confings.py"
    click GeneralUtils "https://github.com/ds-hub-sochi/animal_classification/blob/main/irbis_classifier/src/utils.py"

    %% Click events for Configs and Tests
    click Configs "https://github.com/ds-hub-sochi/animal_classification/tree/main/configs/"
    click TestsDir "https://github.com/ds-hub-sochi/animal_classification/tree/main/tests/"

    %% Styles
    classDef cli fill:#blue,stroke:#000,color:#fff;
    classDef core fill:#0f0,stroke:#000,color:#000;
    classDef data fill:#ffa500,stroke:#000,color:#000;
    classDef config fill:#800080,stroke:#000,color:#fff;
    classDef output fill:#aaa,stroke:#000,color:#000;
    classDef test fill:#ccc,stroke:#000,color:#000;
```
