# iscc/iscc-core architecture

> Architecture diagram of the GitHub repository iscc/iscc-core, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/iscc/iscc-core
- Repository: https://github.com/iscc/iscc-core

## Overview

##### Main Components

a) Core Processing Units:
- Metadata Processing Unit
- Content Processing Units (Text, Image, Audio, Video, Mixed)
- Data Processing Unit
- Instance Processing Unit
- ISCC Code Generation Unit

b) Algorithm Components:
- CDC (Content-Defined Chunking)
- DCT (Discrete Cosine Transform)
- MinHash
- SimHash
- WTAHash (Winner Takes All Hash)

c) Support Components:
- Codec System
- Conformance Testing Framework
- Utility Functions
- Configuration Management

**┌─ Input Layer ─────────────────────────────────────────┐**
- Raw Data │ Metadata │ Content │ Instance Data         │
- ┴──────────┴─────────┴──────────────────────┘
- ↓
**┌─ Processing Layer ──────────────────────────────────┐**
- ┌─ Core Processing Units ─┐  ┌─ Algorithms ───────┐ │
- Meta Code Generator   │  │ - CDC              │ │
- Content Code Gen.     │←→│ - DCT              │ │
- Data Code Generator   │  │ - MinHash          │ │
- Instance Code Gen.    │  │ - SimHash          │ │
- ┘  │ - WTAHash         │ │
- ┘  │
- ┘
- ↓
**┌─ Output Layer ────────────────────────────────────┐**
- ISCC Codes │ Similarity Metrics │ Conformance Data│
- ┘

##### Important Details

- Version indicators (V0) for different components
- Bit-length indicators for different code types
- MainType indicators (META, SEMANTIC, CONTENT, DATA, INSTANCE)
- Input/Output format specifications

##### Technical Details

- Supported Python version (3.9+)
- External dependencies
- Optimization points (Cython implementations)
- Test coverage paths

Remember to maintain a balance between technical detail and clarity, as this is a reference implementation that others will use to port to different languages.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Input
        RawData[Raw Data]:::input
        Metadata[Metadata]:::input
        Content[Content]:::input
        InstanceData[Instance Data]:::input
    end

    subgraph "Core Processing Units"
        MetaProc[Metadata Processing]:::core
        ContentProc[Content Processing]:::core
        DataProc[Data Processing]:::core
        InstanceProc[Instance Processing]:::core
        ISCCGen[ISCC Code Generation]:::core

        subgraph "Content Processing Types"
            TextProc[Text Processing]:::content
            ImageProc[Image Processing]:::content
            AudioProc[Audio Processing]:::content
            VideoProc[Video Processing]:::content
            MixedProc[Mixed Processing]:::content
        end
    end

    subgraph "Algorithm Layer"
        CDC[CDC]:::algo
        DCT[DCT]:::algo
        MinHash[MinHash]:::algo
        SimHash[SimHash]:::algo
        WTAHash[WTAHash]:::algo
    end

    subgraph "Support Systems"
        Codec[Codec System]:::support
        Conformance[Conformance Testing]:::support
        Utils[Utility Functions]:::support
        Config[Configuration Management]:::support
    end

    subgraph Output
        ISCCCodes[ISCC Codes]:::output
        SimilarityMetrics[Similarity Metrics]:::output
        ConformanceData[Conformance Data]:::output
    end

    %% Input connections
    Metadata --> MetaProc
    Content --> ContentProc
    RawData --> DataProc
    InstanceData --> InstanceProc

    %% Content Processing connections
    ContentProc --> TextProc & ImageProc & AudioProc & VideoProc & MixedProc

    %% Algorithm connections
    ContentProc <--> CDC & DCT & MinHash & SimHash & WTAHash

    %% Support system connections
    MetaProc & ContentProc & DataProc & InstanceProc --> Codec
    Codec --> ISCCGen
    ISCCGen --> Conformance
    Utils --> MetaProc & ContentProc & DataProc & InstanceProc
    Config --> MetaProc & ContentProc & DataProc & InstanceProc

    %% Output connections
    ISCCGen --> ISCCCodes
    Conformance --> ConformanceData
    CDC & MinHash & SimHash & WTAHash --> SimilarityMetrics

    %% Click events for component mapping
    click MetaProc "https://github.com/iscc/iscc-core/blob/main/iscc_core/code_meta.py"
    click TextProc "https://github.com/iscc/iscc-core/blob/main/iscc_core/code_content_text.py"
    click ImageProc "https://github.com/iscc/iscc-core/blob/main/iscc_core/code_content_image.py"
    click AudioProc "https://github.com/iscc/iscc-core/blob/main/iscc_core/code_content_audio.py"
    click VideoProc "https://github.com/iscc/iscc-core/blob/main/iscc_core/code_content_video.py"
    click MixedProc "https://github.com/iscc/iscc-core/blob/main/iscc_core/code_content_mixed.py"
    click DataProc "https://github.com/iscc/iscc-core/blob/main/iscc_core/code_data.py"
    click InstanceProc "https://github.com/iscc/iscc-core/blob/main/iscc_core/code_instance.py"
    click ISCCGen "https://github.com/iscc/iscc-core/blob/main/iscc_core/iscc_code.py"
    click CDC "https://github.com/iscc/iscc-core/blob/main/iscc_core/cdc.py"
    click DCT "https://github.com/iscc/iscc-core/blob/main/iscc_core/dct.py"
    click MinHash "https://github.com/iscc/iscc-core/blob/main/iscc_core/minhash.py"
    click SimHash "https://github.com/iscc/iscc-core/blob/main/iscc_core/simhash.py"
    click WTAHash "https://github.com/iscc/iscc-core/blob/main/iscc_core/wtahash.py"
    click Codec "https://github.com/iscc/iscc-core/blob/main/iscc_core/codec.py"
    click Conformance "https://github.com/iscc/iscc-core/blob/main/iscc_core/conformance.py"
    click Utils "https://github.com/iscc/iscc-core/blob/main/iscc_core/utils.py"
    click Config "https://github.com/iscc/iscc-core/blob/main/iscc_core/options.py"

    %% Styling
    classDef input fill:#f96,stroke:#333
    classDef core fill:#9cf,stroke:#333
    classDef content fill:#acf,stroke:#333
    classDef algo fill:#9f9,stroke:#333
    classDef support fill:#c9f,stroke:#333
    classDef output fill:#fc9,stroke:#333
```
