# ipis-mjkim/caueeg-dataset architecture

> Architecture diagram of the GitHub repository ipis-mjkim/caueeg-dataset, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ipis-mjkim/caueeg-dataset
- Repository: https://github.com/ipis-mjkim/caueeg-dataset

## Overview

##### Project Type & Purpose
- This is a **research dataset repository** (“CAUEEG EEG dataset”)
- It provides:
– Raw EEG signals in EDF format
– Event descriptions (.json)
– Annotation files (.json, .xlsx)
- It is consumed by a separate Python/PyTorch library (caueeg-ceednet) for loading, benchmarking, and model training.

##### File Structure
Top-level items:
- annotation.json, dementia.json, abnormal.json, etc.
- /event/*.json (one file per recording)
- /signal/edf/*.edf (EEG signal files)
There are no code modules here—this is strictly the **data layer**.

##### Key Insights from README
- Data access requires an external **approval form** → download link & password
- Loading functions:
– load_caueeg_full_dataset(...) → returns config_data + full_eeg_dataset
– load_caueeg_task_datasets(...) → returns train/val/test splits for tasks (“dementia”, “abnormal”, “no-overlap”)
- Python/PyTorch is the target environment for downstream ML pipelines and benchmarks.

##### Main Components
a. **External Service** (“Data Request & Ethics Approval”)
– Form (Google Forms) → CAU ethics committee → download link & password
b. **Data Storage (on disk)**
– Annotation files
– Event folder
– Signal folder (EDF files)
c. **Data Access Layer**
– caueeg-ceednet Python module (DataLoader)
– Key entry points: load_caueeg_full_dataset, load_caueeg_task_datasets, load_caueeg_task_split
d. **Benchmark/Task Modules**
– CAUEEG-Dementia benchmark
– CAUEEG-Abnormal benchmark
– “No-overlap” variants
e. **ML Pipeline**
– PyTorch Dataset & DataLoader
– Model training & evaluation stages
– Metric computation, logging, visualization
f. **Client / Researcher**
– Jupyter notebooks (e.g., 01_DataSet_DataLoader.ipynb)
– Custom scripts that invoke the loader and training code

##### Relationships & Data Flows
1. Researcher submits form → receives download link → downloads dataset to local path
2. Python DataLoader (caueeg-ceednet) reads files from disk:
- Reads annotation JSON/XLSX → maps serial IDs to labels
- Reads event JSON → optional event streams
- Reads EDF files → loads raw signal arrays
3. DataLoader outputs in-memory Python dicts / PyTorch Datasets
4. Benchmark modules split data into train/val/test based on task parameters
5. ML Pipeline consumes splits → trains models → outputs metrics & plots

##### Architectural Patterns & Design Principles
- **Separation of Concerns**:
– Data storage vs. data loading vs. training/evaluation
- **Modular Benchmarks**: one loader function, multiple task-specific wrappers
- **Extensibility**: load_event flag toggles inclusion of event data
- **Standard Formats**: EDF, JSON, Excel ensure compatibility

- EDF (European Data Format) → raw EEG signals
- JSON / Excel → annotations & events
- Python / PyTorch → DataLoader & ML code
- Jupyter Notebook → exploratory analysis
- GitHub → version control & code hosting

##### Final Checklist
[ ] All major components drawn and labeled
[ ] Key technologies (, EDF, JSON, PyTorch) annotated
[ ] Benchmarks and flags (load_event, no-overlap) represented

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Service
    ExternalService(["Data Request & Ethics Approval"]):::external

    %% Data Storage
    DataStorage[(Data Storage<br/>Annotations & Signals)]:::storage
    README["README.md"]:::storage

    %% Data Access Layer
    DataLoader["DataLoader<br/>(load_caueeg_* functions)"]:::process

    %% Optional Event Stream
    OptionalEvent["Event Stream<br/>(load_event=True)"]:::process

    %% In-memory Dataset
    InMemoryDataset["In-memory Dataset<br/>(PyTorch Dataset)"]:::process

    %% Benchmark Modules
    Benchmarks["Benchmark Modules<br/>(Dementia, Abnormal, No-overlap)"]:::process

    %% ML Pipeline
    MLPipeline["ML Pipeline<br/>(Training & Evaluation)"]:::process

    %% Researcher Interface
    Researcher["Researcher / Client<br/>(Jupyter Notebooks)"]:::external

    %% Connections
    ExternalService -->|"Form → download link"| DataStorage
    DataStorage -->|"load_caueeg_*()"| DataLoader
    DataLoader -->|"read annotations, EDF files"| InMemoryDataset
    DataLoader -->|"read event JSON"| OptionalEvent
    OptionalEvent --> InMemoryDataset
    InMemoryDataset -->|"split data"| Benchmarks
    Benchmarks -->|"train & eval"| MLPipeline
    MLPipeline -->|"results & plots"| Researcher
    README --> DataStorage

    %% Click Events
    click DataStorage "https://github.com/ipis-mjkim/caueeg-dataset/blob/main/."
    click README "https://github.com/ipis-mjkim/caueeg-dataset/blob/main/README.md"

    %% Styles
    classDef storage fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef process fill:#90EE90,stroke:#333,stroke-width:1px
    classDef external fill:#ffa500,stroke:#333,stroke-width:1px
```
