How malichot/pymotifs works
PyMotifs is a Python library for modeling and analyzing humanities texts. The main workflow takes a corpus or pre-tokenized documents, annotates text as linguistic tokens or motifs, builds n-grams, and derives frequency or TF-IDF features for plotting or PCA. Separate capabilities include co-occurrence analysis, contextual return to source text, and projection/clustering. Edges follow the sampled pipeline calls and SOURCE INDEX references; some subsystem internals were not sampled, so uncertain connections are omitted.
Main entry points: pipeline.py Analysis pipeline · config.py Logging and data config · tokenizer.py Linguistic tokenizer · features.py N-grams and features
Components 14
Text ingestion
- Linguistic tokenizer
src/motifs/tokenizer.py - Motif patterns
src/motifs/data/fr_motifs.json - Token loader
src/motifs/dataloader.py
Feature analysis
- Analysis pipeline
src/motifs/pipeline.py - N-grams and features
src/motifs/features.py - Co-occurrence metrics
src/motifs/metrics.py - Context and filtering
src/motifs/utils.py
Exploration
- PCA transforms
src/motifs/pca.py - Projection and clustering
src/motifs/cluster.py - Analysis plots
src/motifs/plots.py
Shared runtime
- Logging and data config
src/motifs/config.py - Analysis defaults
src/motifs/constants.py
Other components
- Analyst
- Text corpus
Connections 18
- Analyst → Analysis pipeline: runs analysis
src/motifs/pipeline.py - Text corpus → Linguistic tokenizer: provides text
src/motifs/tokenizer.py - Analyst → Token loader: loads tokens
- Analyst → Projection and clustering: clusters features
src/motifs/cluster.py - Linguistic tokenizer → Motif patterns: loads patterns
src/motifs/tokenizer.py - Analysis pipeline → Linguistic tokenizer: creates tokens
src/motifs/pipeline.py - Analysis pipeline → Token loader: loads tokens
src/motifs/pipeline.py - Analysis pipeline → N-grams and features: builds features
src/motifs/pipeline.py - Analysis pipeline → Context and filtering: filters tokens
src/motifs/pipeline.py - Analysis pipeline → PCA transforms: transforms features
src/motifs/pipeline.py - Analysis pipeline → Analysis plots: plots features
src/motifs/pipeline.py - Co-occurrence metrics → N-grams and features: builds co-occurrence
src/motifs/metrics.py - Analysis pipeline → Logging and data config: logs activity
src/motifs/pipeline.py - Linguistic tokenizer → Logging and data config: uses data path
src/motifs/tokenizer.py - N-grams and features → Logging and data config: logs activity
src/motifs/features.py - PCA transforms → Logging and data config: logs activity
src/motifs/pca.py - Analysis plots → Logging and data config: logs activity
src/motifs/plots.py - Projection and clustering → Analysis defaults: uses defaults
src/motifs/cluster.py