# malichot/pymotifs architecture

> Architecture diagram of the GitHub repository malichot/pymotifs, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/malichot/pymotifs
- Repository: https://github.com/malichot/pymotifs

## Overview

PyMotifs is a Python library for modeling and analyzing humanities texts. The main workflow takes a corpus or pre-tokenized documents, annotates text as linguistic tokens or motifs, builds n-grams, and derives frequency or TF-IDF features for plotting or PCA. Separate capabilities include co-occurrence analysis, contextual return to source text, and projection/clustering. Edges follow the sampled pipeline calls and SOURCE INDEX references; some subsystem internals were not sampled, so uncertain connections are omitted.

Main entry points: [`src/motifs/pipeline.py`](https://github.com/malichot/pymotifs/blob/main/src/motifs/pipeline.py) (Analysis pipeline), [`src/motifs/config.py`](https://github.com/malichot/pymotifs/blob/main/src/motifs/config.py) (Logging and data config), [`src/motifs/tokenizer.py`](https://github.com/malichot/pymotifs/blob/main/src/motifs/tokenizer.py) (Linguistic tokenizer), [`src/motifs/features.py`](https://github.com/malichot/pymotifs/blob/main/src/motifs/features.py) (N-grams and features)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Text ingestion"]
  node_tokenizer["Linguistic tokenizer<br/>[tokenizer.py]"]
  node_motifdata["Motif patterns<br/>[fr_motifs.json]"]
  node_dataloader["Token loader<br/>[dataloader.py]"]
end

subgraph group_analysis["Feature analysis"]
  node_pipeline["Analysis pipeline<br/>[pipeline.py]"]
  node_features["N-grams and features<br/>[features.py]"]
  node_metrics["Co-occurrence metrics<br/>[metrics.py]"]
  node_utils["Context and filtering<br/>[utils.py]"]
end

subgraph group_explore["Exploration"]
  node_pca["PCA transforms<br/>[pca.py]"]
  node_cluster["Projection and clustering<br/>[cluster.py]"]
  node_plots["Analysis plots<br/>[plots.py]"]
end

subgraph group_support["Shared runtime"]
  node_config["Logging and data config<br/>[config.py]"]
  node_constants["Analysis defaults<br/>[constants.py]"]
end

node_user(("Analyst"))
node_corpus["Text corpus"]

node_user -->|"runs analysis"| node_pipeline
node_corpus -->|"provides text"| node_tokenizer
node_user -.->|"loads tokens"| node_dataloader
node_user -->|"clusters features"| node_cluster
node_tokenizer -->|"loads patterns"| node_motifdata
node_pipeline -->|"creates tokens"| node_tokenizer
node_pipeline -->|"loads tokens"| node_dataloader
node_pipeline -->|"builds features"| node_features
node_pipeline -->|"filters tokens"| node_utils
node_pipeline -->|"transforms features"| node_pca
node_pipeline -->|"plots features"| node_plots
node_metrics -->|"builds co-occurrence"| node_features
node_pipeline -->|"logs activity"| node_config
node_tokenizer -->|"uses data path"| node_config
node_features -.->|"logs activity"| node_config
node_pca -->|"logs activity"| node_config
node_plots -.->|"logs activity"| node_config
node_cluster -->|"uses defaults"| node_constants

click node_tokenizer "https://github.com/malichot/pymotifs/blob/main/src/motifs/tokenizer.py"
click node_motifdata "https://github.com/malichot/pymotifs/blob/main/src/motifs/data/fr_motifs.json"
click node_dataloader "https://github.com/malichot/pymotifs/blob/main/src/motifs/dataloader.py"
click node_pipeline "https://github.com/malichot/pymotifs/blob/main/src/motifs/pipeline.py"
click node_features "https://github.com/malichot/pymotifs/blob/main/src/motifs/features.py"
click node_metrics "https://github.com/malichot/pymotifs/blob/main/src/motifs/metrics.py"
click node_utils "https://github.com/malichot/pymotifs/blob/main/src/motifs/utils.py"
click node_pca "https://github.com/malichot/pymotifs/blob/main/src/motifs/pca.py"
click node_cluster "https://github.com/malichot/pymotifs/blob/main/src/motifs/cluster.py"
click node_plots "https://github.com/malichot/pymotifs/blob/main/src/motifs/plots.py"
click node_config "https://github.com/malichot/pymotifs/blob/main/src/motifs/config.py"
click node_constants "https://github.com/malichot/pymotifs/blob/main/src/motifs/constants.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_tokenizer,node_motifdata,node_dataloader toneBlue
class node_pipeline,node_features,node_metrics,node_utils toneAmber
class node_pca,node_cluster,node_plots toneMint
class node_config,node_constants toneRose
class node_user,node_corpus toneIndigo
```

## Components

### Text ingestion

- [Linguistic tokenizer](https://github.com/malichot/pymotifs/blob/main/src/motifs/tokenizer.py): `src/motifs/tokenizer.py`
- [Motif patterns](https://github.com/malichot/pymotifs/blob/main/src/motifs/data/fr_motifs.json): `src/motifs/data/fr_motifs.json`
- [Token loader](https://github.com/malichot/pymotifs/blob/main/src/motifs/dataloader.py): `src/motifs/dataloader.py`

### Feature analysis

- [Analysis pipeline](https://github.com/malichot/pymotifs/blob/main/src/motifs/pipeline.py): `src/motifs/pipeline.py`
- [N-grams and features](https://github.com/malichot/pymotifs/blob/main/src/motifs/features.py): `src/motifs/features.py`
- [Co-occurrence metrics](https://github.com/malichot/pymotifs/blob/main/src/motifs/metrics.py): `src/motifs/metrics.py`
- [Context and filtering](https://github.com/malichot/pymotifs/blob/main/src/motifs/utils.py): `src/motifs/utils.py`

### Exploration

- [PCA transforms](https://github.com/malichot/pymotifs/blob/main/src/motifs/pca.py): `src/motifs/pca.py`
- [Projection and clustering](https://github.com/malichot/pymotifs/blob/main/src/motifs/cluster.py): `src/motifs/cluster.py`
- [Analysis plots](https://github.com/malichot/pymotifs/blob/main/src/motifs/plots.py): `src/motifs/plots.py`

### Shared runtime

- [Logging and data config](https://github.com/malichot/pymotifs/blob/main/src/motifs/config.py): `src/motifs/config.py`
- [Analysis defaults](https://github.com/malichot/pymotifs/blob/main/src/motifs/constants.py): `src/motifs/constants.py`

### Other components

- **Analyst**
- **Text corpus**

## Connections

- Analyst → Analysis pipeline: runs analysis (evidence: `src/motifs/pipeline.py`)
- Text corpus → Linguistic tokenizer: provides text (evidence: `src/motifs/tokenizer.py`)
- Analyst → Token loader: loads tokens
- Analyst → Projection and clustering: clusters features (evidence: `src/motifs/cluster.py`)
- Linguistic tokenizer → Motif patterns: loads patterns (evidence: `src/motifs/tokenizer.py`)
- Analysis pipeline → Linguistic tokenizer: creates tokens (evidence: `src/motifs/pipeline.py`)
- Analysis pipeline → Token loader: loads tokens (evidence: `src/motifs/pipeline.py`)
- Analysis pipeline → N-grams and features: builds features (evidence: `src/motifs/pipeline.py`)
- Analysis pipeline → Context and filtering: filters tokens (evidence: `src/motifs/pipeline.py`)
- Analysis pipeline → PCA transforms: transforms features (evidence: `src/motifs/pipeline.py`)
- Analysis pipeline → Analysis plots: plots features (evidence: `src/motifs/pipeline.py`)
- Co-occurrence metrics → N-grams and features: builds co-occurrence (evidence: `src/motifs/metrics.py`)
- Analysis pipeline → Logging and data config: logs activity (evidence: `src/motifs/pipeline.py`)
- Linguistic tokenizer → Logging and data config: uses data path (evidence: `src/motifs/tokenizer.py`)
- N-grams and features → Logging and data config: logs activity (evidence: `src/motifs/features.py`)
- PCA transforms → Logging and data config: logs activity (evidence: `src/motifs/pca.py`)
- Analysis plots → Logging and data config: logs activity (evidence: `src/motifs/plots.py`)
- Projection and clustering → Analysis defaults: uses defaults (evidence: `src/motifs/cluster.py`)
