# cleanlab/cleanvision architecture

> Architecture diagram of the GitHub repository cleanlab/cleanvision, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cleanlab/cleanvision
- Repository: https://github.com/cleanlab/cleanvision

## Overview

##### Main Components
- Imagelab (Main Entry Point): Represent this as the central component. It is the primary interface that users call to run data audits.
- Dataset Modules: Show separate components such as BaseDataset, FsspecDataset, HFDataset, and TorchDataset found in src/cleanvision/dataset. These modules abstract how various image data sources are accessed.
- Issue Detection and Management: Include components for the issue managers (e.g., DuplicateIssueManager, ImagePropertyIssueManager) under src/cleanvision/issue_managers and in the utils folder (base_issue_manager, viz_manager, etc.). These parts encapsulate the logic to detect and handle specific image issues.
- Utilities: Indicate the utility functions and classes (e.g., in src/cleanvision/utils) that support common functionality like serialization, constant definitions, and helper methods.

##### Relationships and Interactions
- User Interaction Layer: At the top level, the application code uses the Imagelab class to initiate the auditing process.
- Data Flow: The Imagelab component takes a path to image files, then routes the images through one or multiple dataset modules, depending on user configuration (e.g., standard folder images, HuggingFace datasets, Torchvision datasets).
- Reporting Flow: After processing, the results are aggregated by Imagelab and passed to the reporting module to generate a clean and user-friendly output.
- Utility Support: Across the modules, utility components support common tasks (e.g., serialization, visualization, configuration) — these can be depicted as a shared service layer.

##### Architectural Patterns and Design Principles
- Separation of Concerns: The design clearly separates data ingestion (dataset modules), issue detection (issue managers), and reporting. This modularity is essential for plug-and-play improvements and updates.
- Extensibility: The modular issue manager structure indicates a design that supports new image quality checks without altering the core pipeline.
- Reusability: Utility modules provide common functionality used across multiple parts of the system.

##### Relevant Technologies, Frameworks, and Tools
- Python 3.7+ is required as mentioned in the README.
- While not explicitly visible in the file tree, external libraries (likely PIL/Pillow, OpenCV, or other image processing libraries) are used for the image analyses.
- The testing suite (in the tests directory) confirms that the architecture is unit-test friendly.
- CI/CD integration via GitHub workflows indicates an emphasis on continuous integration and code quality.

## Diagram (Mermaid)

```mermaid
flowchart TD
    User["User"]:::user

    %% Main Entry Point
    Imagelab("Imagelab (Entry Point)"):::entry

    %% Dataset Modules Subgraph
    subgraph "Dataset Modules"
        BaseDataset("BaseDataset"):::dataset
        FsspecDataset("FsspecDataset"):::dataset
        HFDataset("HFDataset"):::dataset
        TorchDataset("TorchDataset"):::dataset
    end

    %% Issue Managers Subgraph
    subgraph "Issue Managers"
        DuplicateIssueManager("DuplicateIssueManager"):::issue
        ImagePropertyIssueManager("ImagePropertyIssueManager"):::issue
    end

    %% Utilities Subgraph
    subgraph "Utilities"
        BaseIssueManager("Base Issue Manager"):::utility
        VizManager("Viz Manager"):::utility
        Constants("Constants"):::utility
        Serialize("Serialize"):::utility
        Utils("Utils"):::utility
    end

    %% Data Flow Connections
    User -->|"starts"| Imagelab

    %% Imagelab processes image datasets
    Imagelab -->|"ingest"| BaseDataset
    Imagelab -->|"ingest"| FsspecDataset
    Imagelab -->|"ingest"| HFDataset
    Imagelab -->|"ingest"| TorchDataset

    %% Dataset Modules forward data to Issue Managers
    BaseDataset -->|"analyze"| DuplicateIssueManager
    BaseDataset -->|"analyze"| ImagePropertyIssueManager
    FsspecDataset -->|"analyze"| DuplicateIssueManager
    FsspecDataset -->|"analyze"| ImagePropertyIssueManager
    HFDataset -->|"analyze"| DuplicateIssueManager
    HFDataset -->|"analyze"| ImagePropertyIssueManager
    TorchDataset -->|"analyze"| DuplicateIssueManager
    TorchDataset -->|"analyze"| ImagePropertyIssueManager

    %% Issue Managers return detected issues for reporting back to Imagelab 
    DuplicateIssueManager -->|"report"| Imagelab
    ImagePropertyIssueManager -->|"report"| Imagelab

    %% Utilities support Imagelab and its modules
    Imagelab ---|"uses"| BaseIssueManager
    Imagelab ---|"uses"| VizManager
    Imagelab ---|"uses"| Constants
    Imagelab ---|"uses"| Serialize
    Imagelab ---|"uses"| Utils

    %% Styles
    classDef entry fill:#add8e6,stroke:#000,stroke-width:2px;
    classDef dataset fill:#90ee90,stroke:#000,stroke-width:2px;
    classDef issue fill:#ffcccb,stroke:#000,stroke-width:2px;
    classDef utility fill:#ffffcc,stroke:#000,stroke-width:2px;
    classDef user fill:#d3d3d3,stroke:#000,stroke-width:2px;

    %% Click Events for Main Entry Point
    click Imagelab "https://github.com/cleanlab/cleanvision/blob/main/src/cleanvision/imagelab.py"
    click Imagelab "https://github.com/cleanlab/cleanvision/blob/main/docs/source/cleanvision/imagelab.rst"

    %% Click Events for Dataset Modules
    click BaseDataset "https://github.com/cleanlab/cleanvision/blob/main/src/cleanvision/dataset/base_dataset.py"
    click BaseDataset "https://github.com/cleanlab/cleanvision/blob/main/docs/source/cleanvision/dataset/base_dataset.rst"

    click FsspecDataset "https://github.com/cleanlab/cleanvision/blob/main/src/cleanvision/dataset/fsspec_dataset.py"
    click FsspecDataset "https://github.com/cleanlab/cleanvision/blob/main/docs/source/cleanvision/dataset/fsspec_dataset.rst"

    click HFDataset "https://github.com/cleanlab/cleanvision/blob/main/src/cleanvision/dataset/hf_dataset.py"
    click HFDataset "https://github.com/cleanlab/cleanvision/blob/main/docs/source/cleanvision/dataset/hf_dataset.rst"

    click TorchDataset "https://github.com/cleanlab/cleanvision/blob/main/src/cleanvision/dataset/torch_dataset.py"
    click TorchDataset "https://github.com/cleanlab/cleanvision/blob/main/docs/source/cleanvision/dataset/torch_dataset.rst"

    %% Click Events for Issue Managers
    click DuplicateIssueManager "https://github.com/cleanlab/cleanvision/blob/main/src/cleanvision/issue_managers/duplicate_issue_manager.py"
    click DuplicateIssueManager "https://github.com/cleanlab/cleanvision/blob/main/docs/source/cleanvision/issue_managers/duplicate_issue_manager.rst"

    click ImagePropertyIssueManager "https://github.com/cleanlab/cleanvision/blob/main/src/cleanvision/issue_managers/image_property_issue_manager.py"
    click ImagePropertyIssueManager "https://github.com/cleanlab/cleanvision/blob/main/docs/source/cleanvision/issue_managers/image_property_issue_manager.rst"
    click ImagePropertyIssueManager "https://github.com/cleanlab/cleanvision/blob/main/src/cleanvision/issue_managers/image_property.py"
    click ImagePropertyIssueManager "https://github.com/cleanlab/cleanvision/blob/main/docs/source/cleanvision/issue_managers/image_property.rst"

    %% Click Events for Utilities - Issue Utilities
    click BaseIssueManager "https://github.com/cleanlab/cleanvision/blob/main/src/cleanvision/utils/base_issue_manager.py"
    click BaseIssueManager "https://github.com/cleanlab/cleanvision/blob/main/docs/source/cleanvision/utils/base_issue_manager.rst"

    click VizManager "https://github.com/cleanlab/cleanvision/blob/main/src/cleanvision/utils/viz_manager.py"
    click VizManager "https://github.com/cleanlab/cleanvision/blob/main/docs/source/cleanvision/utils/viz_manager.rst"

    %% Click Events for Utilities - Other Utilities
    click Constants "https://github.com/cleanlab/cleanvision/blob/main/src/cleanvision/utils/constants.py"
    click Constants "https://github.com/cleanlab/cleanvision/blob/main/docs/source/cleanvision/utils/constants.rst"

    click Serialize "https://github.com/cleanlab/cleanvision/blob/main/src/cleanvision/utils/serialize.py"
    click Serialize "https://github.com/cleanlab/cleanvision/blob/main/docs/source/cleanvision/utils/serialize.rst"

    click Utils "https://github.com/cleanlab/cleanvision/blob/main/src/cleanvision/utils/utils.py"
    click Utils "https://github.com/cleanlab/cleanvision/blob/main/docs/source/cleanvision/utils/utils.rst"
```
