# green-wood/comer architecture

> Architecture diagram of the GitHub repository green-wood/comer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/green-wood/comer
- Repository: https://github.com/green-wood/comer

## Overview

CoMER recognizes handwritten mathematical-expression images: CROHME data is prepared as batches, encoded visually, decoded into token sequences, and searched to produce predictions. The repository also provides LaTeX-to-symLG conversion and symLG comparison/evaluation tools. Sampled imports and calls support the model’s internal links; the broader evaluation-tool relationships are documented at the capability level because their entry points were not sampled.

Main entry points: [`comer/lit_comer.py`](https://github.com/green-wood/comer/blob/master/comer/lit_comer.py) (Lightning task), [`comer/model/decoder.py`](https://github.com/green-wood/comer/blob/master/comer/model/decoder.py) (Expression decoder), [`comer/datamodule/datamodule.py`](https://github.com/green-wood/comer/blob/master/comer/datamodule/datamodule.py) (CROHME data module), [`comer/utils/generation_utils.py`](https://github.com/green-wood/comer/blob/master/comer/utils/generation_utils.py) (Bidirectional search)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data preparation"]
  node_data_module["CROHME data module<br/>[datamodule.py]"]
  node_dataset["Dataset transforms<br/>[dataset.py]"]
  node_vocabulary["Token vocabulary<br/>[vocab.py]"]
end

subgraph group_recognition["Expression recognition"]
  node_trainer["Lightning task<br/>[lit_comer.py]"]
  node_model["CoMER model<br/>[comer.py]"]
  node_encoder["Image encoder<br/>[encoder.py]"]
  node_position["Positional encoding<br/>[pos_enc.py]"]
  node_decoder["Expression decoder<br/>[decoder.py]"]
  node_transformer["Transformer decoder"]
  node_attention["Attention layers<br/>[attention.py]"]
  node_coverage["Coverage refinement<br/>[arm.py]"]
  node_generation["Bidirectional search"]
  node_beam["Beam scorer<br/>[beam_search.py]"]
  node_metrics["Loss and metrics<br/>[utils.py]"]
  node_prediction["Prediction archive<br/>[lit_comer.py]"]
end

subgraph group_conversion["Graph conversion"]
  node_converter["LaTeX converter<br/>[process_mml.py]"]
  node_tag_updater["Node tag updater<br/>[update_nodeTags.py]"]
end

subgraph group_evaluation["Graph evaluation"]
  node_graph_tools["Graph comparison tools"]
  node_graph_model["Label graph model<br/>[lg.py]"]
  node_small_graph["Small graph matching<br/>[smallGraph.py]"]
  node_graph_metrics["Evaluation metrics<br/>[SmGrConfMatrix.py]"]
  node_evaluation_results["Comparison results"]
end

node_user(("Researcher"))
node_crohme[("CROHME images")]
node_latex["LaTeX expressions"]
node_symgraphs["symLG files"]

node_user -->|"configures"| node_data_module
node_crohme -->|"loads"| node_data_module
node_data_module -->|"builds batches"| node_dataset
node_data_module -->|"uses tokens"| node_vocabulary
node_user -->|"trains model"| node_trainer
node_data_module -->|"provides batches"| node_trainer
node_trainer -->|"calls"| node_model
node_trainer -->|"computes loss"| node_metrics
node_trainer -->|"starts search"| node_generation
node_model -->|"encodes images"| node_encoder
node_model -->|"decodes features"| node_decoder
node_encoder -->|"uses encoding"| node_position
node_decoder -->|"uses encoding"| node_position
node_decoder -->|"uses tokens"| node_vocabulary
node_decoder -->|"delegates decoding"| node_transformer
node_transformer -->|"uses attention"| node_attention
node_transformer -->|"refines attention"| node_coverage
node_decoder -->|"inherits search"| node_generation
node_generation -->|"scores beams"| node_beam
node_generation -->|"uses tokens"| node_vocabulary
node_generation -->|"uses hypotheses"| node_metrics
node_trainer -->|"writes predictions"| node_prediction
node_prediction -->|"contains expressions"| node_latex
node_latex -->|"converts to symLG"| node_converter
node_converter -->|"produces"| node_symgraphs
node_user -->|"evaluates graphs"| node_graph_tools
node_symgraphs -->|"compares folders"| node_graph_tools
node_graph_tools -->|"uses graph model"| node_graph_model
node_graph_model -->|"matches subgraphs"| node_small_graph
node_graph_model -.->|"evaluates correctness"| node_graph_metrics
node_graph_tools -->|"reports metrics"| node_evaluation_results

click node_data_module "https://github.com/green-wood/comer/blob/master/comer/datamodule/datamodule.py"
click node_dataset "https://github.com/green-wood/comer/blob/master/comer/datamodule/dataset.py"
click node_vocabulary "https://github.com/green-wood/comer/blob/master/comer/datamodule/vocab.py"
click node_trainer "https://github.com/green-wood/comer/blob/master/comer/lit_comer.py"
click node_model "https://github.com/green-wood/comer/blob/master/comer/model/comer.py"
click node_encoder "https://github.com/green-wood/comer/blob/master/comer/model/encoder.py"
click node_position "https://github.com/green-wood/comer/blob/master/comer/model/pos_enc.py"
click node_decoder "https://github.com/green-wood/comer/blob/master/comer/model/decoder.py"
click node_transformer "https://github.com/green-wood/comer/blob/master/comer/model/transformer/transformer_decoder.py"
click node_attention "https://github.com/green-wood/comer/blob/master/comer/model/transformer/attention.py"
click node_coverage "https://github.com/green-wood/comer/blob/master/comer/model/transformer/arm.py"
click node_generation "https://github.com/green-wood/comer/blob/master/comer/utils/generation_utils.py"
click node_beam "https://github.com/green-wood/comer/blob/master/comer/utils/beam_search.py"
click node_metrics "https://github.com/green-wood/comer/blob/master/comer/utils/utils.py"
click node_prediction "https://github.com/green-wood/comer/blob/master/comer/lit_comer.py"
click node_converter "https://github.com/green-wood/comer/blob/master/convert2symLG/process_mml.py"
click node_tag_updater "https://github.com/green-wood/comer/blob/master/convert2symLG/update_nodeTags.py"
click node_graph_tools "https://github.com/green-wood/comer/tree/master/lgeval/src"
click node_graph_model "https://github.com/green-wood/comer/blob/master/lgeval/src/lg.py"
click node_small_graph "https://github.com/green-wood/comer/blob/master/lgeval/src/smallGraph.py"
click node_graph_metrics "https://github.com/green-wood/comer/blob/master/lgeval/src/SmGrConfMatrix.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_data_module,node_dataset,node_vocabulary toneBlue
class node_trainer,node_model,node_encoder,node_position,node_decoder,node_transformer,node_attention,node_coverage,node_generation,node_beam,node_metrics,node_prediction,node_crohme toneAmber
class node_converter,node_tag_updater toneMint
class node_graph_tools,node_graph_model,node_small_graph,node_graph_metrics,node_evaluation_results toneRose
class node_user,node_latex,node_symgraphs toneIndigo
```

## Components

### Data preparation

- [CROHME data module](https://github.com/green-wood/comer/blob/master/comer/datamodule/datamodule.py): `comer/datamodule/datamodule.py`
- [Dataset transforms](https://github.com/green-wood/comer/blob/master/comer/datamodule/dataset.py): `comer/datamodule/dataset.py`
- [Token vocabulary](https://github.com/green-wood/comer/blob/master/comer/datamodule/vocab.py): `comer/datamodule/vocab.py`

### Expression recognition

- [Lightning task](https://github.com/green-wood/comer/blob/master/comer/lit_comer.py): `comer/lit_comer.py`
- [CoMER model](https://github.com/green-wood/comer/blob/master/comer/model/comer.py): `comer/model/comer.py`
- [Image encoder](https://github.com/green-wood/comer/blob/master/comer/model/encoder.py): `comer/model/encoder.py`
- [Positional encoding](https://github.com/green-wood/comer/blob/master/comer/model/pos_enc.py): `comer/model/pos_enc.py`
- [Expression decoder](https://github.com/green-wood/comer/blob/master/comer/model/decoder.py): `comer/model/decoder.py`
- [Transformer decoder](https://github.com/green-wood/comer/blob/master/comer/model/transformer/transformer_decoder.py): `comer/model/transformer/transformer_decoder.py`
- [Attention layers](https://github.com/green-wood/comer/blob/master/comer/model/transformer/attention.py): `comer/model/transformer/attention.py`
- [Coverage refinement](https://github.com/green-wood/comer/blob/master/comer/model/transformer/arm.py): `comer/model/transformer/arm.py`
- [Bidirectional search](https://github.com/green-wood/comer/blob/master/comer/utils/generation_utils.py): `comer/utils/generation_utils.py`
- [Beam scorer](https://github.com/green-wood/comer/blob/master/comer/utils/beam_search.py): `comer/utils/beam_search.py`
- [Loss and metrics](https://github.com/green-wood/comer/blob/master/comer/utils/utils.py): `comer/utils/utils.py`
- [Prediction archive](https://github.com/green-wood/comer/blob/master/comer/lit_comer.py): `comer/lit_comer.py`

### Graph conversion

- [LaTeX converter](https://github.com/green-wood/comer/blob/master/convert2symLG/process_mml.py): `convert2symLG/process_mml.py`
- [Node tag updater](https://github.com/green-wood/comer/blob/master/convert2symLG/update_nodeTags.py): `convert2symLG/update_nodeTags.py`

### Graph evaluation

- [Graph comparison tools](https://github.com/green-wood/comer/tree/master/lgeval/src): `lgeval/src`
- [Label graph model](https://github.com/green-wood/comer/blob/master/lgeval/src/lg.py): `lgeval/src/lg.py`
- [Small graph matching](https://github.com/green-wood/comer/blob/master/lgeval/src/smallGraph.py): `lgeval/src/smallGraph.py`
- [Evaluation metrics](https://github.com/green-wood/comer/blob/master/lgeval/src/SmGrConfMatrix.py): `lgeval/src/SmGrConfMatrix.py`
- **Comparison results**

### Other components

- **Researcher**
- **CROHME images**
- **LaTeX expressions**
- **symLG files**

## Connections

- Researcher → CROHME data module: configures (evidence: `README.md`)
- CROHME images → CROHME data module: loads (evidence: `comer/datamodule/datamodule.py`)
- CROHME data module → Dataset transforms: builds batches (evidence: `comer/datamodule/datamodule.py`)
- CROHME data module → Token vocabulary: uses tokens (evidence: `comer/datamodule/datamodule.py`)
- Researcher → Lightning task: trains model (evidence: `README.md`)
- CROHME data module → Lightning task: provides batches
- Lightning task → CoMER model: calls (evidence: `comer/lit_comer.py`)
- Lightning task → Loss and metrics: computes loss (evidence: `comer/lit_comer.py`)
- Lightning task → Bidirectional search: starts search (evidence: `comer/lit_comer.py`)
- CoMER model → Image encoder: encodes images (evidence: `comer/model/comer.py`)
- CoMER model → Expression decoder: decodes features (evidence: `comer/model/comer.py`)
- Image encoder → Positional encoding: uses encoding (evidence: `comer/model/encoder.py`)
- Expression decoder → Positional encoding: uses encoding (evidence: `comer/model/decoder.py`)
- Expression decoder → Token vocabulary: uses tokens (evidence: `comer/model/decoder.py`)
- Expression decoder → Transformer decoder: delegates decoding (evidence: `comer/model/decoder.py`)
- Transformer decoder → Attention layers: uses attention (evidence: `comer/model/transformer/transformer_decoder.py`)
- Transformer decoder → Coverage refinement: refines attention (evidence: `comer/model/transformer/transformer_decoder.py`)
- Expression decoder → Bidirectional search: inherits search (evidence: `comer/model/decoder.py`)
- Bidirectional search → Beam scorer: scores beams (evidence: `comer/utils/generation_utils.py`)
- Bidirectional search → Token vocabulary: uses tokens (evidence: `comer/utils/generation_utils.py`)
- Bidirectional search → Loss and metrics: uses hypotheses (evidence: `comer/utils/generation_utils.py`)
- Lightning task → Prediction archive: writes predictions (evidence: `comer/lit_comer.py`)
- Prediction archive → LaTeX expressions: contains expressions (evidence: `comer/lit_comer.py`)
- LaTeX expressions → LaTeX converter: converts to symLG (evidence: `README.md`)
- LaTeX converter → symLG files: produces (evidence: `README.md`)
- Researcher → Graph comparison tools: evaluates graphs (evidence: `README.md`)
- symLG files → Graph comparison tools: compares folders (evidence: `README.md`)
- Graph comparison tools → Label graph model: uses graph model (evidence: `lgeval/src/lg.py`)
- Label graph model → Small graph matching: matches subgraphs (evidence: `lgeval/src/lg.py`)
- Label graph model → Evaluation metrics: evaluates correctness
- Graph comparison tools → Comparison results: reports metrics (evidence: `README.md`)
