# hltchkust/moel architecture

> Architecture diagram of the GitHub repository hltchkust/moel, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hltchkust/moel
- Repository: https://github.com/hltchkust/moel

## Overview

This repository is a standalone PyTorch research codebase for empathetic dialogue generation, not a deployed service. The architecture is organized around one training/evaluation entrypoint, a small set of model variants, and a data pipeline built for the preprocessed EmpatheticDialogues dataset.

#### 1) Top-level purpose and runtime shape

The code trains and evaluates sequence-to-sequence dialogue models that generate a response from:

- a dialogue context
- optionally an emotion label
- a target response during supervised training

The repository supports two main model families:

- a standard Transformer baseline
- the MoEL variant, which mixes multiple “listener” experts to produce more empathetic responses

The primary runtime is:

- load preprocessed dataset files
- build vocabulary / embeddings / batches
- instantiate the selected model
- train with supervised objectives
- decode with beam search
- evaluate with BLEU-style metrics and save generated text

#### 2) Main subsystems

##### A. Entry and orchestration layer
Files:
- `main.py`
- `interact.py`
- `get_ed_data.py`

Roles:
- `main.py` is the central training/testing driver. It likely parses command-line options, selects the model, sets up data, runs optimization, and triggers evaluation.
- `interact.py` is the inference-facing script for interactive use: given a user context, it loads a trained model and returns generated responses.
- `get_ed_data.py` is the dataset preparation boundary. It exists to transform the EmpatheticDialogues source data into the `.npy` tensors used by the training code.

This is the main control boundary of the repository: everything else supports training or decoding.

##### B. Data and preprocessing layer
Files:
- `empathetic-dialogue/*.npy`
- `utils/data_reader.py`
- `utils/data_loader.py`

Dataset assets:
- `sys_dialog_texts.{train,dev,test}.npy`
- `sys_target_texts.{train,dev,test}.npy`
- `sys_emotion_texts.{train,dev,test}.npy`
- `sys_situation_texts.{train,dev,test}.npy`

These files form the preprocessed data boundary. The repo expects the dataset to already be stored in NumPy arrays rather than raw text.

Observed data roles:
- dialogue text: input context
- target text: response to generate
- emotion text/label: auxiliary emotion supervision
- situation text: contextual metadata

The reader/loader modules likely:
- read the `.npy` files
- build examples from parallel arrays
- convert text to token IDs
- batch and pad sequences
- feed the model with context/response/emotion fields

This is the main data flow from disk into training.

##### C. Model layer
Files:
- `model/common_layer.py`
- `model/transformer.py`
- `model/transformer_mulexpert.py`

This is the core learning subsystem.

###### `transformer.py`
Implements the baseline Transformer dialogue model used in the README as `--model trs`.

###### `transformer_mulexpert.py`
Implements the MoEL architecture used in the README as `--model experts`.

From the README, MoEL’s conceptual structure is:

- an emotion tracker that infers an emotion distribution from the dialogue
- multiple empathetic listeners / experts specialized for different emotions
- a shared listener for common response generation behavior
- a meta listener that fuses the listener outputs into the final response representation

So the model boundary is:

- shared Transformer-like components for encoding/decoding
- emotion-aware expert routing in the MoEL variant

###### `common_layer.py`
Shared neural building blocks used by both model variants, likely including:
- attention or transformer utility layers
- embedding/positional components
- projection and normalization helpers

This file is the common dependency layer beneath both model implementations.

##### D. Decoding / search layer
Files:
- `utils/beam_omt.py`
- `utils/beam_omt_experts.py`

These are beam-search decoding utilities.

Architecture role:
- convert model token probabilities into ranked response candidates during inference
- one beam search implementation for the baseline Transformer
- a separate one for the expert/MoEL model, reflecting different decoding state requirements

This is a critical boundary between probabilistic model output and final natural-language generation.

##### E. Evaluation layer
Files:
- `utils/metric.py`
- `utils/multi-bleu.perl`
- `generation_result.txt`

Roles:
- compute evaluation metrics for generated responses
- integrate BLEU scoring via the bundled Perl script
- store example or final generation outputs in `generation_result.txt`

This subsystem sits after decoding and before reporting or analysis.

##### F. Configuration and training utilities
Files:
- `utils/config.py`
- `requirements.txt`

Roles:
- define hyperparameters and runtime flags
- control model choice, optimization settings, embedding dimensions, beam size, GPU usage, multitask mode, label smoothing, scheduled optimization, and other experiment switches

The README shows this codebase is highly configuration-driven rather than organized around many different executables.

#### 3) Core data flow

The repository’s primary flow is:

1. **Preprocessed dataset on disk**
   - The empathetic dialogue corpus is stored as `.npy` arrays.

2. **Data loading and batching**
   - `data_reader.py` and `data_loader.py` assemble contexts, targets, and emotion labels into model-ready batches.

3. **Model construction**
   - `main.py` selects either:
     - Transformer baseline
     - MoEL expert model

4. **Encoding and generation**
   - The model consumes the dialogue context.
   - In the MoEL path, emotion-related information influences expert selection/combination.
   - The decoder produces token distributions for the target response.

5. **Decoding**
   - Beam search utilities turn token probabilities into a final response sequence.

6. **Evaluation and output**
   - Metrics are computed.
   - Generated examples/results are written out.

The supervised target response is both a training label and an evaluation reference.

#### 4) Architectural boundaries that matter

##### Dataset boundary
The code does not operate on raw conversational text at runtime. The key boundary is the preprocessed `.npy` corpus in `empathetic-dialogue/`. That means the training pipeline assumes a fixed offline preprocessing stage.

##### Model family boundary
The `--model` switch separates two architectural paths:
- `trs` for the standard Transformer
- `experts` for MoEL

This is not just a different hyperparameter setting; it changes the internal inference structure and decoding utilities.

##### Emotion-awareness boundary
Emotion is an auxiliary signal throughout the project. In the dataset it exists as a separate field, and in MoEL it becomes a functional routing signal for expert listeners. That makes emotion a first-class architectural input, not just metadata.

##### Decoding boundary
Beam search is not embedded directly in the model files. It is handled as a separate utility layer, which cleanly separates neural scoring from sequence search.

##### Evaluation boundary
Metrics are decoupled from model code and use a dedicated script plus the bundled BLEU Perl implementation.

#### 5) External technologies and dependencies

The architecture materially depends on:

- **PyTorch** for all neural modeling
- **NumPy** for storing and loading preprocessed dataset artifacts
- **Pretrained GloVe embeddings** for initialization when enabled
- **Perl BLEU script** for BLEU evaluation

There is no evidence of:
- a web server
- a database
- distributed training infrastructure
- a message queue
- cloud services

So the repo is best understood as a local research training/inference pipeline.

#### 6) What an architecture diagram should show

A clear diagram for this repository would include:

- **Input data**
  - EmpatheticDialogues preprocessed `.npy` files
  - optional pretrained GloVe vectors

- **Orchestration**
  - `main.py` as the training/evaluation controller
  - `interact.py` as interactive inference
  - `get_ed_data.py` as preprocessing

- **Data pipeline**
  - dataset reader
  - batcher/loader
  - tokenization/indexing

- **Model layer**
  - baseline Transformer
  - MoEL expert model
  - shared neural components

- **Decoding**
  - beam search for baseline
  - beam search for expert model

- **Evaluation/output**
  - BLEU/metrics
  - generated result file
  - saved checkpoints under the configured save path

The most important conceptual distinction is between the shared Transformer backbone and the MoEL-specific expert routing path, both of which consume the same preprocessed dialogue/emotion data but produce different generation behavior.

Main entry points: [`main.py`](https://github.com/hltchkust/moel/blob/HEAD/main.py) (main.py), [`model/transformer_mulexpert.py`](https://github.com/hltchkust/moel/blob/HEAD/model/transformer_mulexpert.py) (transformer_mulexpert.py), [`model/transformer.py`](https://github.com/hltchkust/moel/blob/HEAD/model/transformer.py) (transformer.py), [`interact.py`](https://github.com/hltchkust/moel/blob/HEAD/interact.py) (interact.py)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph "Orchestration"
  main_py["main.py<br/>train/eval driver<br/>[main.py]"]
  interact_py["interact.py<br/>interactive inference<br/>[interact.py]"]
  get_ed_data_py["get_ed_data.py<br/>preprocess script<br/>[get_ed_data.py]"]
end

subgraph "Data"
  ed_data[("ED corpus<br/>npy dataset")]
  data_reader_py["data_reader.py<br/>dataset reader<br/>[data_reader.py]"]
  data_loader_py["data_loader.py<br/>batch loader<br/>[data_loader.py]"]
end

subgraph "Models"
  common_layer_py["common_layer.py<br/>shared layers<br/>[common_layer.py]"]
  transformer_py["transformer.py<br/>baseline model<br/>[transformer.py]"]
  mulexpert_py["transformer_mulexpert.py<br/>MoEL model"]
end

subgraph "Decoding"
  beam_py["beam_omt.py<br/>beam search<br/>[beam_omt.py]"]
  beam_expert_py["beam_omt_experts.py<br/>beam search"]
end

subgraph "Evaluation"
  metric_py["metric.py<br/>metrics<br/>[metric.py]"]
  mbleu_perl["multi-bleu.perl<br/>BLEU script<br/>[multi-bleu.perl]"]
  gen_result["generation_result.txt<br/>output artifact"]
end

subgraph "Config"
  config_py["config.py<br/>runtime config<br/>[config.py]"]
  requirements_txt["requirements.txt<br/>deps<br/>[requirements.txt]"]
end

main_py -->|"reads settings"| config_py
main_py -->|"loads data"| data_reader_py
main_py -->|"batches"| data_loader_py
main_py -->|"selects baseline"| transformer_py
main_py -->|"selects experts"| mulexpert_py
data_reader_py -->|"reads arrays"| ed_data
data_loader_py -.->|"streams examples"| ed_data
get_ed_data_py -->|"builds corpus"| ed_data
transformer_py -->|"shares blocks"| common_layer_py
mulexpert_py -->|"shares blocks"| common_layer_py
mulexpert_py -.->|"extends"| transformer_py
transformer_py -->|"decodes with"| beam_py
mulexpert_py -->|"decodes with"| beam_expert_py
beam_py -->|"evaluated by"| metric_py
beam_expert_py -->|"evaluated by"| metric_py
metric_py -->|"calls BLEU"| mbleu_perl
main_py -->|"writes output"| gen_result
interact_py -->|"loads baseline"| transformer_py
interact_py -->|"loads experts"| mulexpert_py
interact_py -->|"decodes"| beam_py
requirements_txt -.->|"supports"| main_py

click main_py "https://github.com/hltchkust/moel/blob/master/main.py"
click interact_py "https://github.com/hltchkust/moel/blob/master/interact.py"
click get_ed_data_py "https://github.com/hltchkust/moel/blob/master/get_ed_data.py"
click config_py "https://github.com/hltchkust/moel/blob/master/utils/config.py"
click requirements_txt "https://github.com/hltchkust/moel/blob/master/requirements.txt"
click ed_data "https://github.com/hltchkust/moel/tree/master/empathetic-dialogue"
click data_reader_py "https://github.com/hltchkust/moel/blob/master/utils/data_reader.py"
click data_loader_py "https://github.com/hltchkust/moel/blob/master/utils/data_loader.py"
click common_layer_py "https://github.com/hltchkust/moel/blob/master/model/common_layer.py"
click transformer_py "https://github.com/hltchkust/moel/blob/master/model/transformer.py"
click mulexpert_py "https://github.com/hltchkust/moel/blob/master/model/transformer_mulexpert.py"
click beam_py "https://github.com/hltchkust/moel/blob/master/utils/beam_omt.py"
click beam_expert_py "https://github.com/hltchkust/moel/blob/master/utils/beam_omt_experts.py"
click metric_py "https://github.com/hltchkust/moel/blob/master/utils/metric.py"
click mbleu_perl "https://github.com/hltchkust/moel/blob/master/utils/multi-bleu.perl"
click gen_result "https://github.com/hltchkust/moel/blob/master/generation_result.txt"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class main_py,interact_py,get_ed_data_py toneBlue
class ed_data,data_reader_py,data_loader_py toneAmber
class common_layer_py,transformer_py,mulexpert_py toneMint
class beam_py,beam_expert_py toneRose
class metric_py,mbleu_perl,gen_result toneIndigo
class config_py,requirements_txt toneTeal
```

## Components

### Orchestration

Entry points that train, evaluate, or interact with the models.

- [main.py](https://github.com/hltchkust/moel/blob/HEAD/main.py): train/eval driver · Central training and testing entrypoint. · `main.py`
- [interact.py](https://github.com/hltchkust/moel/blob/HEAD/interact.py): interactive inference · Loads a trained model and generates responses for user input. · `interact.py`
- [get_ed_data.py](https://github.com/hltchkust/moel/blob/HEAD/get_ed_data.py): preprocess script · Builds the preprocessed EmpatheticDialogues dataset artifacts. · `get_ed_data.py`

### Data

Preprocessed EmpatheticDialogues assets and loading pipeline.

- [ED corpus](https://github.com/hltchkust/moel/tree/HEAD/empathetic-dialogue): npy dataset · Preprocessed EmpatheticDialogues splits and fields stored as NumPy arrays. · `empathetic-dialogue`
- [data_reader.py](https://github.com/hltchkust/moel/blob/HEAD/utils/data_reader.py): dataset reader · Reads dataset arrays and builds examples from dialogue, emotion, and target fields. · `utils/data_reader.py`
- [data_loader.py](https://github.com/hltchkust/moel/blob/HEAD/utils/data_loader.py): batch loader · Pads, batches, and feeds model-ready tensors to training and decoding. · `utils/data_loader.py`

### Models

Shared neural layers and the two generation model variants.

- [common_layer.py](https://github.com/hltchkust/moel/blob/HEAD/model/common_layer.py): shared layers · Common neural building blocks shared by both model variants. · `model/common_layer.py`
- [transformer.py](https://github.com/hltchkust/moel/blob/HEAD/model/transformer.py): baseline model · Standard Transformer dialogue generator. · `model/transformer.py`
- [transformer_mulexpert.py](https://github.com/hltchkust/moel/blob/HEAD/model/transformer_mulexpert.py): MoEL model · Emotion-aware expert mixture model for empathetic response generation. · `model/transformer_mulexpert.py`

### Decoding

Beam search utilities used to turn token scores into responses.

- [beam_omt.py](https://github.com/hltchkust/moel/blob/HEAD/utils/beam_omt.py): beam search · Beam search for the baseline Transformer decoder. · `utils/beam_omt.py`
- [beam_omt_experts.py](https://github.com/hltchkust/moel/blob/HEAD/utils/beam_omt_experts.py): beam search · Beam search variant for the expert/MoEL model. · `utils/beam_omt_experts.py`

### Evaluation

Metric computation and result artifacts.

- [metric.py](https://github.com/hltchkust/moel/blob/HEAD/utils/metric.py): metrics · Computes evaluation scores for generated responses. · `utils/metric.py`
- [multi-bleu.perl](https://github.com/hltchkust/moel/blob/HEAD/utils/multi-bleu.perl): BLEU script · Bundled BLEU scorer used during evaluation. · `utils/multi-bleu.perl`
- [generation_result.txt](https://github.com/hltchkust/moel/blob/HEAD/generation_result.txt): output artifact · Saved generated responses or examples from inference/evaluation. · `generation_result.txt`

### Config

Runtime options and experiment dependencies.

- [config.py](https://github.com/hltchkust/moel/blob/HEAD/utils/config.py): runtime config · Defines experiment flags and hyperparameters. · `utils/config.py`
- [requirements.txt](https://github.com/hltchkust/moel/blob/HEAD/requirements.txt): deps · Python package dependencies for the research codebase. · `requirements.txt`

## Connections

- main.py → config.py: reads settings · Training and evaluation are driven by runtime flags and hyperparameters.
- main.py → data_reader.py: loads data · Main orchestration pulls training/dev/test examples through the reader.
- main.py → data_loader.py: batches · Examples are padded and assembled into minibatches.
- main.py → transformer.py: selects baseline · One runtime path instantiates the standard Transformer.
- main.py → transformer_mulexpert.py: selects experts · Alternative runtime path instantiates the MoEL model.
- data_reader.py → ED corpus: reads arrays · The reader consumes preprocessed EmpatheticDialogues NumPy files.
- data_loader.py → ED corpus: streams examples · The loader ultimately serves the same preprocessed dataset boundary.
- get_ed_data.py → ED corpus: builds corpus · Preprocessing generates the dataset artifacts stored on disk.
- transformer.py → common_layer.py: shares blocks · Baseline model uses shared neural layers and utilities.
- transformer_mulexpert.py → common_layer.py: shares blocks · MoEL model reuses the common layer components.
- transformer_mulexpert.py → transformer.py: extends · MoEL is a specialized generation variant built on the same dialogue modeling backbone.
- transformer.py → beam_omt.py: decodes with · Baseline generation uses the standard beam search helper.
- transformer_mulexpert.py → beam_omt_experts.py: decodes with · Expert model uses a beam search variant tailored to its state.
- beam_omt.py → metric.py: evaluated by · Decoded outputs are scored by the metric layer.
- beam_omt_experts.py → metric.py: evaluated by · Expert-model outputs are scored by the metric layer.
- metric.py → multi-bleu.perl: calls BLEU · Evaluation incorporates the bundled BLEU implementation.
- main.py → generation_result.txt: writes output · Training/evaluation can persist generated responses to a result file.
- interact.py → transformer.py: loads baseline · Interactive mode can run the baseline model.
- interact.py → transformer_mulexpert.py: loads experts · Interactive mode can also run the MoEL model.
- interact.py → beam_omt.py: decodes · Interactive generation uses beam search for response creation.
- requirements.txt → main.py: supports · Declared dependencies enable the training and inference runtime.
