# soda-inria/carte architecture

> Architecture diagram of the GitHub repository soda-inria/carte, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/soda-inria/carte
- Repository: https://github.com/soda-inria/carte

## Overview

##### Project Type and Purpose
- The README and file structure indicate that CARTE is a specialized machine learning library for tabular learning. It implements a method that treats rows as star graphs. The project provides both regressor and classifier modules (e.g., CARTERegressor, CARTEClassifier) and also includes utilities for data loading, preprocessing, graph conversion, and hyperparameter grid search.
- It is designed for research and reproducibility (with examples and a paper reference), as well as for production use by installing via PyPI.

##### File Structure
- The top-level directory contains important files (README, INSTALL, Makefile, pyproject.toml) and the main package folder “carte_ai”.
- Inside “carte_ai”:
– The “configs” folder holds configuration settings for different aspects (e.g., model_parameters, directory paths, visualization settings).
– The “data” folder includes configuration JSONs, example parquet files, and pretrained models.
– The “scripts” folder contains runnable scripts for tasks like downloading data, compiling results, evaluating models, and preprocessing.
– The “src” folder holds core modules for the library: model definitions (carte_model, carte_estimator), grid search, baseline implementations (baseline_multitable, baseline_singletable_nn), conversion from table to graph, and utility functions.
- Separate directories for examples, tests, and additional configuration files reinforce modularity and clarity of roles (e.g., examples for demonstrating use cases and tests for unit testing components).

##### README for Additional Insights
- The README clearly explains the step-by-step usage workflow:
a. Data loading and preprocessing (with functions for loading datasets).
b. Converting tables to graphs (using Table2GraphTransformer).
c. Model instantiation and prediction (using CARTERegressor and CARTEClassifier with sklearn-like interfaces).
- It shows the use of external libraries such as fasttext and huggingface_hub, indicating dependencies and integration points.

a. Main Components:
- Data Input & Configuration:
- Data Sources: raw tabular data (e.g., wine and Spotify datasets).
- Configuration Files: JSON files for dataset parameters, model parameters, directory configurations.
- Preprocessing & Transformation:
- Data Loading Scripts: functions from data/load_data.py.
- Preprocessing Modules: scripts and utilities in “src/preprocess_utils.py” and “scripts/preprocess_raw.py”.
- Table2GraphTransformer: Converts tabular data into graph representations.
- Core Model Components:
- Model Modules: carte_model, carte_estimator, baseline modules.
- Grid Search and Ensemble Methods: implemented in carte_gridsearch and baseline scripts.
- Evaluation & Prediction:
- Evaluation Components: evaluator functions and scripts (evaluate_utils, evaluate_singletable.py).
- Results and Visualization: components in “visualization_utils.py” and output files in results directories.
- External Dependencies:
- FastText for text embeddings (graph nodes).
- Huggingface_hub for downloading pretrained models.

b. Relationships and Data Flow:
- Data Loader routes the data to the Preprocessing stage, where it undergoes:
- Preprocessing (cleaning, formatting) and then transformation into graph representations via Table2GraphTransformer.
- The transformed graph representations are then fed into the Core Model Component:
- For training using CARTERegressor or CARTEClassifier with selected hyperparameters (bagging strategy, device settings, etc.).
- After model training, predictions are generated and evaluated, with outputs stored under the results directories.
- A feedback loop may exist through grid search and experimentation scripts for optimizing parameters.

c. Highlighting Architectural Patterns or Design Principles:
- Modularity: Clear separation between configuration, data handling, processing, model implementation, and evaluation.
- Reusable Components: The use of utilities and configuration modules facilitates easy adaptation for different datasets.
- Extensibility: The design supports both baseline and advanced models (single vs. multi-table processing) along with extensions via scripts and tests.

d. Relevant Technologies and Frameworks:
- Python package structure supporting modularization.
- External libraries: FastText, Huggingface Hub, and potentially sklearn for the fit/predict interface.
- Graph Transformers as the core technological approach for tabular data.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Input & Configuration Layer
    subgraph "Configuration & Data Setup"
        DC("Data Input & Configuration"):::config
        DS("Raw Data & Config Files"):::config
    end

    %% Data Loader
    subgraph "Data Loading"
        DL("Data Loader (load_data.py)"):::config
    end

    %% Preprocessing & Graph Transformation Layer
    subgraph "Preprocessing & Graph Transformation"
        PS("Preprocessing Scripts"):::preprocess
        GT("Table2GraphTransformer"):::preprocess
    end

    %% Core Modeling & Training Layer
    subgraph "Core Model Components"
        MD("Model Definitions (CARTERegressor/Classifier)"):::model
        GS("Grid Search Module"):::model
        BL("Baseline Implementations"):::model
    end

    %% Evaluation & Visualization Layer
    subgraph "Evaluation & Prediction"
        EV("Evaluation Utilities & Scripts"):::eval
        VI("Visualization Components"):::eval
    end

    %% External Dependencies
    ED("External Dependencies\n(FastText, Huggingface Hub)"):::external

    %% Data Flow Connections
    DC --> DS
    DS --> DL
    DL --> PS
    PS --> GT
    GT --> MD
    MD --> EV
    EV --> VI

    %% Additional Model Components links
    MD --> GS
    MD --> BL
    %% External dependencies input to core modeling (dashed arrow)
    ED -.-> MD

    %% Click Events for Configuration & Data Setup
    click DC "https://github.com/soda-inria/carte/tree/main/carte_ai/configs"
    click DS "https://github.com/soda-inria/carte/tree/main/carte_ai/data"
    click DL "https://github.com/soda-inria/carte/blob/main/carte_ai/data/load_data.py"

    %% Click Events for Preprocessing & Transformation
    click PS "https://github.com/soda-inria/carte/blob/main/carte_ai/scripts/preprocess_raw.py"
    click PS "https://github.com/soda-inria/carte/blob/main/carte_ai/src/preprocess_utils.py"
    click GT "https://github.com/soda-inria/carte/blob/main/carte_ai/src/carte_table_to_graph.py"

    %% Click Events for Core Model Components
    click MD "https://github.com/soda-inria/carte/blob/main/carte_ai/src/carte_model.py"
    click MD "https://github.com/soda-inria/carte/blob/main/carte_ai/src/carte_estimator.py"
    click GS "https://github.com/soda-inria/carte/blob/main/carte_ai/src/carte_gridsearch.py"
    click BL "https://github.com/soda-inria/carte/blob/main/carte_ai/src/baseline_multitable.py"
    click BL "https://github.com/soda-inria/carte/blob/main/carte_ai/src/baseline_singletable_nn.py"

    %% Click Events for Evaluation & Prediction
    click EV "https://github.com/soda-inria/carte/blob/main/carte_ai/src/evaluate_utils.py"
    click EV "https://github.com/soda-inria/carte/blob/main/carte_ai/scripts/evaluate_singletable.py"
    click VI "https://github.com/soda-inria/carte/blob/main/carte_ai/src/visualization_utils.py"

    %% Styles for Color Coding
    classDef config fill:#add8e6,stroke:#000,stroke-width:1px;
    classDef preprocess fill:#90ee90,stroke:#000,stroke-width:1px;
    classDef model fill:#ffa500,stroke:#000,stroke-width:1px;
    classDef eval fill:#ffff99,stroke:#000,stroke-width:1px;
    classDef external fill:#d3d3d3,stroke:#000,stroke-width:1px;
```
