# atifmallo/rome architecture

> Architecture diagram of the GitHub repository atifmallo/rome, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/atifmallo/rome
- Repository: https://github.com/atifmallo/rome

## Overview

RoME is a two-stage multimodal sentiment-analysis framework for CMU-MOSI and CMU-MOSEI. The principal workflow loads utterance features and labels, trains modality experts in Stage 1, then trains and evaluates a mask-aware multimodal model in Stage 2 across single-, pair-, and trimodal conditions. The model combines expert routing with CIT, MRR, RMC, and CM-DAE; training also computes masked regression losses and saves results. The README documents the model capabilities and evaluation conditions; detailed implementation of some Stage-2 internals was only partially sampled.

Main entry points: [`Rome/model.py`](https://github.com/atifmallo/rome/blob/main/Rome/model.py) (RoME model), [`Rome/train_Rome.py`](https://github.com/atifmallo/rome/blob/main/Rome/train_Rome.py) (Training and evaluation), [`Rome/utils.py`](https://github.com/atifmallo/rome/blob/main/Rome/utils.py) (Data and model setup), [`Rome/dataloader_cmumosi.py`](https://github.com/atifmallo/rome/blob/main/Rome/dataloader_cmumosi.py) (Utterance loader)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data input"]
  node_feature_data["CMU features"]
  node_config["Path configuration<br/>[config.py]"]
  node_dataset_loader["Utterance loader"]
end

subgraph group_learning["Model learning"]
  node_rome["RoME model<br/>[model.py]"]
  node_experts["Modality experts<br/>[model.py]"]
  node_attention["Expert attention"]
  node_cit["Cross-modal interaction<br/>[model.py]"]
  node_mrr["Reliability reweighting<br/>[model.py]"]
  node_cmdae["Latent reconstruction<br/>[model.py]"]
  node_rmc["Residual correction<br/>[model.py]"]
end

subgraph group_training["Training and evaluation"]
  node_batch_setup["Data and model setup<br/>[utils.py]"]
  node_loss["Masked losses<br/>[loss.py]"]
  node_trainer["Training and evaluation<br/>[train_Rome.py]"]
end

node_engineer(("Researcher"))

node_engineer -->|"starts runs"| node_trainer
node_config -->|"sets data paths"| node_dataset_loader
node_config -->|"sets output paths"| node_trainer
node_feature_data -->|"provides features"| node_dataset_loader
node_trainer -->|"requests loaders"| node_batch_setup
node_batch_setup -->|"builds dataset"| node_dataset_loader
node_batch_setup -->|"constructs model"| node_rome
node_trainer -->|"computes regression loss"| node_loss
node_trainer -->|"runs forward pass"| node_rome
node_rome -->|"routes modalities"| node_experts
node_experts -->|"uses attention blocks"| node_attention
node_rome -->|"interacts across modalities"| node_cit
node_rome -.->|"scales by reliability"| node_mrr
node_rome -->|"reconstructs latent features"| node_cmdae
node_rome -->|"refines fused output"| node_rmc
node_trainer -->|"saves checkpoints"| node_rome
node_trainer -->|"evaluates conditions"| node_trainer

click node_feature_data "https://github.com/atifmallo/rome/tree/main/dataset"
click node_config "https://github.com/atifmallo/rome/blob/main/config.py"
click node_dataset_loader "https://github.com/atifmallo/rome/blob/main/Rome/dataloader_cmumosi.py"
click node_batch_setup "https://github.com/atifmallo/rome/blob/main/Rome/utils.py"
click node_rome "https://github.com/atifmallo/rome/blob/main/Rome/model.py"
click node_experts "https://github.com/atifmallo/rome/blob/main/Rome/model.py"
click node_attention "https://github.com/atifmallo/rome/blob/main/Rome/modules/Attention_softmoe.py"
click node_cit "https://github.com/atifmallo/rome/blob/main/Rome/model.py"
click node_mrr "https://github.com/atifmallo/rome/blob/main/Rome/model.py"
click node_cmdae "https://github.com/atifmallo/rome/blob/main/Rome/model.py"
click node_rmc "https://github.com/atifmallo/rome/blob/main/Rome/model.py"
click node_loss "https://github.com/atifmallo/rome/blob/main/Rome/loss.py"
click node_trainer "https://github.com/atifmallo/rome/blob/main/Rome/train_Rome.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_feature_data,node_config,node_dataset_loader toneBlue
class node_rome,node_experts,node_attention,node_cit,node_mrr,node_cmdae,node_rmc toneAmber
class node_batch_setup,node_loss,node_trainer toneMint
class node_engineer toneIndigo
```

## Components

### Data input

- [CMU features](https://github.com/atifmallo/rome/tree/main/dataset): `dataset`
- [Path configuration](https://github.com/atifmallo/rome/blob/main/config.py): `config.py`
- [Utterance loader](https://github.com/atifmallo/rome/blob/main/Rome/dataloader_cmumosi.py): `Rome/dataloader_cmumosi.py`

### Model learning

- [RoME model](https://github.com/atifmallo/rome/blob/main/Rome/model.py): `Rome/model.py`
- [Modality experts](https://github.com/atifmallo/rome/blob/main/Rome/model.py): `Rome/model.py`
- [Expert attention](https://github.com/atifmallo/rome/blob/main/Rome/modules/Attention_softmoe.py): `Rome/modules/Attention_softmoe.py`
- [Cross-modal interaction](https://github.com/atifmallo/rome/blob/main/Rome/model.py): `Rome/model.py`
- [Reliability reweighting](https://github.com/atifmallo/rome/blob/main/Rome/model.py): `Rome/model.py`
- [Latent reconstruction](https://github.com/atifmallo/rome/blob/main/Rome/model.py): `Rome/model.py`
- [Residual correction](https://github.com/atifmallo/rome/blob/main/Rome/model.py): `Rome/model.py`

### Training and evaluation

- [Data and model setup](https://github.com/atifmallo/rome/blob/main/Rome/utils.py): `Rome/utils.py`
- [Masked losses](https://github.com/atifmallo/rome/blob/main/Rome/loss.py): `Rome/loss.py`
- [Training and evaluation](https://github.com/atifmallo/rome/blob/main/Rome/train_Rome.py): `Rome/train_Rome.py`

### Other components

- **Researcher**

## Connections

- Researcher → Training and evaluation: starts runs (evidence: `README.md`)
- Path configuration → Utterance loader: sets data paths (evidence: `Rome/utils.py`)
- Path configuration → Training and evaluation: sets output paths (evidence: `config.py`)
- CMU features → Utterance loader: provides features (evidence: `Rome/dataloader_cmumosi.py`)
- Training and evaluation → Data and model setup: requests loaders (evidence: `Rome/train_Rome.py`)
- Data and model setup → Utterance loader: builds dataset (evidence: `Rome/utils.py`)
- Data and model setup → RoME model: constructs model (evidence: `Rome/utils.py`)
- Training and evaluation → Masked losses: computes regression loss (evidence: `Rome/train_Rome.py`)
- Training and evaluation → RoME model: runs forward pass (evidence: `Rome/train_Rome.py`)
- RoME model → Modality experts: routes modalities (evidence: `Rome/model.py`)
- Modality experts → Expert attention: uses attention blocks (evidence: `Rome/model.py`)
- RoME model → Cross-modal interaction: interacts across modalities (evidence: `Rome/model.py`)
- RoME model → Reliability reweighting: scales by reliability (evidence: `README.md`)
- RoME model → Latent reconstruction: reconstructs latent features (evidence: `Rome/model.py`)
- RoME model → Residual correction: refines fused output (evidence: `Rome/model.py`)
- Training and evaluation → RoME model: saves checkpoints (evidence: `Rome/train_Rome.py`)
- Training and evaluation → Training and evaluation: evaluates conditions (evidence: `README.md`)