# anamul-hoque-emtiaj/crismer architecture

> Architecture diagram of the GitHub repository anamul-hoque-emtiaj/crismer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/anamul-hoque-emtiaj/crismer
- Repository: https://github.com/anamul-hoque-emtiaj/crismer

## Overview

##### Project type & purpose (from README + tree)
This is a **Python-based ML research/tooling project** for **genome-wide CRISPR-Cas9 off-target prediction and optimization**, centered on a **transformer model** (“transformer-based interpretable deep learning approach”). It looks like a **CLI/scriptable library + notebooks** project rather than a web service.

The repository contains:
- A core package-like folder: `crismer-opti/` with Python modules, models, utils, and scripts
- Pretrained artifacts: `crismer-opti/models/*.pth`, `*.pkl` and top-level `models/IDT.pth`
- Datasets for training/benchmarking: `datasets/*.csv`
- Notebooks for parameter exploration and test scenarios: `*.ipynb`
- A shell script referencing an external off-target search tool: `crismer-opti/script/casoffinder_genome.sh`
- Example inputs: `crismer-opti/example/*`
- A test genome FASTA: `crismer-opti/data/test_genome.fa`

So your “system” is primarily a **local ML pipeline**: input sequences → candidate generation / enumeration (possibly via Cas-OFFinder) → feature/encoding → transformer inference → postprocessing/interpretability → optimization/ranking outputs.

##### File structure → infer architectural components
Key files/folders and what they likely represent architecturally:

##### Core application / orchestration
- `crismer-opti/CRISMER.py`
- Likely the main entrypoint: runs prediction/optimization pipeline end-to-end (read sgRNAs, generate candidates, call model, produce outputs).

##### Domain logic / modules
- `crismer-opti/crismer_modules.py`
- Likely contains model wrappers, training/inference helpers, sequence encoding, scoring, interpretability components.
- `crismer-opti/utils.py`
- Likely general utilities: IO, preprocessing, metrics, helpers.

##### ML model artifacts
- `crismer-opti/models/`
- `change_site_circleseq_model.pth` (transformer or similar)
- `bin_weights.pkl`, `minmax_scaler.pkl` (calibration, scaling, binning)
- Top-level `models/IDT.pth`
- Another model/checkpoint (maybe a baseline or alternative architecture).

##### External tool integration
- `crismer-opti/script/casoffinder_genome.sh`
- Implies a dependency on **Cas-OFFinder** (or similar) for genome-wide enumeration of potential off-target sites.

##### Data assets
- `datasets/*.csv`
- Benchmark/training datasets (GUIDE-seq, CIRCLE-seq, SITE-seq, etc.)
- `crismer-opti/data/test_genome.fa`
- Small genome for testing local runs.

##### Outputs

- **Level 1 (high-level):** major blocks and data flow
- **Level 2 (drill-down):** expand the “CRISMER pipeline” block into submodules and artifacts

This yields maximum accuracy and detail without pretending this is a deployed service.

##### Entrypoints / execution modes (left side)
Two entry paths:
##### Notebook workflow
- `test-scenario-1.ipynb`, `test-scenario-2.ipynb`, `crismer-opti-params.ipynb`
##### Script/CLI workflow
- `crismer-opti/CRISMER.py` (primary)

Both feed into the same underlying library modules.

##### Inputs (explicit, because this is a scientific pipeline)
- **sgRNA inputs**
- `crismer-opti/example/example_sgRNAs.txt`
- `crismer-opti/example/example.csv`
- **Reference genome**
- `crismer-opti/data/test_genome.fa` (and “user-provided genome FASTA” as a generalized node)

##### Candidate off-target enumeration (external integration)
A distinct block:
- **Cas-OFFinder (external binary/tool)** invoked via:
- `crismer-opti/script/casoffinder_genome.sh`
Data flow:
- Genome + sgRNA → Cas-OFFinder → list of candidate sites (potential off-targets)

##### Preprocessing / feature engineering
“sequence encoding + feature scaling” as a stage implemented by:
- `crismer_modules.py`
- `utils.py`
Artifacts it loads:
- `crismer-opti/models/minmax_scaler.pkl`
- Possibly `bin_weights.pkl`

Data flow:
- Candidate sites → encoded tensors/features → normalized/scaled features → model-ready inputs

##### ML inference core (transformer model)
A core block:
- **Transformer Off-target Prediction Model**
- Loads: `crismer-opti/models/change_site_circleseq_model.pth`
- Possibly alternative model: `models/IDT.pth` (show as “optional/alternate checkpoint”)
Outputs:
- Prediction scores / probabilities / risk ranking per candidate site

##### Interpretability + optimization / ranking
Because the README emphasizes “interpretable” and “optimization”, make this its own block (even if implementation is in `crismer_modules.py`):
- Postprocessing: thresholds, bins, calibration (ties to `bin_weights.pkl`)
- Interpretability outputs (attribution, attention summaries, motif importance, etc.—label generically if not sure)
- Optimization: suggest improved guides / rank guides by off-target risk

##### Outputs (right side)
- Ranked off-target list (CSV/TSV)
- Ranked sgRNA recommendations
- Plots/reports (especially in notebooks)

That notebooks might output figures inline.

##### Datasets (training/benchmark, not necessarily runtime)
`datasets/*.csv` in a “Data Sources (offline)” section:
- `guideseq.csv`, `circleseq_all.csv`, etc.
Them to:
- “Model development/training” (even if training scripts aren’t visible here, it’s still architecturally true that these datasets relate to model creation)
And connect “Model development” to the generated artifacts:
- `.pth`, `.pkl`

##### Architectural patterns / principles to highlight
- **Pipeline architecture** (sequential stages)
- **Separation of concerns**
- Orchestration (`CRISMER.py`)
- Domain logic (`crismer_modules.py`)
- Utilities (`utils.py`)
- External enumeration tool (`casoffinder_genome.sh`)
- Model artifacts (`models/*.pth`, `*.pkl`)
- **Two-phase lifecycle**
- Offline: dataset → training → saved checkpoints/scalers
- Online/local runtime: inputs → enumeration → preprocessing → inference → ranking

##### “Don’t overthink it” detail maximization checklist (specific to this repo)
- `CRISMER.py` (main orchestrator)
- `crismer_modules.py` (core logic)
- `utils.py` (helpers)
- `script/casoffinder_genome.sh` (integration layer)
- `models/change_site_circleseq_model.pth`
- `models/minmax_scaler.pkl`
- `models/bin_weights.pkl`
- top-level `models/IDT.pth` (alternate model)
- `datasets/*.csv` (offline data)
- `example/*` and `data/test_genome.fa` (sample inputs)
- `*.ipynb` notebooks (interactive entrypoints)

## Diagram (Mermaid)

```mermaid
flowchart TD

%% =========================
%% Entry / Execution Modes
%% =========================
subgraph "Execution Modes (User Driven)"
direction TB
U["Researcher / User"]:::external

subgraph "Notebook Workflow (Interactive)"
direction TB
NBParams["crismer-opti-params.ipynb"]:::notebook
NB1["test-scenario-1.ipynb"]:::notebook
NB2["test-scenario-2.ipynb"]:::notebook
end

subgraph "Script / CLI Workflow"
direction TB
CRISMER["CRISMER.py (Orchestrator)"]:::entry
end

U -->|"runs"| NBParams
U -->|"runs"| NB1
U -->|"runs"| NB2
U -->|"runs"| CRISMER
end

%% =========================
%% Runtime / Online Pipeline
%% =========================
subgraph "Runtime: Inference & Optimization (Local Pipeline)"
direction TB

subgraph "Inputs"
direction TB
InCSV["Example sgRNA input (CSV)"]:::data
InTXT["Example sgRNA input (TXT)"]:::data
GenomeTest["Test genome FASTA"]:::data
GenomeUser["User-provided genome FASTA"]:::data
end

subgraph "Candidate Off-target Enumeration (Optional)"
direction TB
CasWrap["casoffinder_genome.sh (wrapper)"]:::script
CasBin["Cas-OFFinder (external binary)"]:::external
Candidates["Candidate off-target sites (list)"]:::data
end

subgraph "Core Library Modules"
direction TB
Mods["crismer_modules.py"]:::module
Utils["utils.py"]:::module
end

subgraph "Preprocessing & Feature Engineering"
direction TB
Encode["Sequence encoding / tensorization"]:::process
Scale["Feature scaling / normalization"]:::process
Scaler[("minmax_scaler.pkl")]:::artifact
end

subgraph "Transformer Inference"
direction TB
Model["Transformer off-target prediction"]:::process
PTHPrimary[("change_site_circleseq_model.pth")]:::artifact
PTHAlt[("IDT.pth (alternate)")]:::artifact
Preds["Per-site prediction scores"]:::data
end

subgraph "Postprocess, Interpretability & Optimization"
direction TB
Calib["Calibration / binning"]:::process
BinW[("bin_weights.pkl")]:::artifact
Interp["Interpretability summaries (generic)"]:::process
Optimize["Guide ranking / optimization"]:::process
end

subgraph "Outputs"
direction TB
OutOff["Ranked off-target list (CSV/TSV)"]:::output
OutGuides["Recommended sgRNAs"]:::output
OutPlots["Plots / reports (esp. notebooks)"]:::output
end

%% Entrypoints -> Inputs/Pipeline
CRISMER -->|"reads"| InCSV
CRISMER -->|"reads"| InTXT
CRISMER -->|"reads"| GenomeTest
CRISMER -->|"reads"| GenomeUser

NBParams -->|"loads_inputs"| InCSV
NB1 -->|"loads_inputs"| InTXT
NB2 -->|"loads_inputs"| InCSV
NB1 -->|"loads_genome"| GenomeTest
NB2 -->|"loads_genome"| GenomeTest

%% Optional enumeration via Cas-OFFinder
InCSV -.->|"sgRNAs"| CasWrap
InTXT -.->|"sgRNAs"| CasWrap
GenomeTest -.->|"genome"| CasWrap
GenomeUser -.->|"genome"| CasWrap
CasWrap -.->|"invokes"| CasBin
CasBin -.->|"returns_candidates"| Candidates

%% Core modules orchestrate stages
CRISMER -->|"calls"| Mods
NBParams -->|"calls"| Mods
NB1 -->|"calls"| Mods
NB2 -->|"calls"| Mods
Mods -->|"uses"| Utils

%% Preprocessing (from candidates)
Candidates -->|"to_encode"| Encode
Encode -->|"features"| Scale
Scale -->|"loads"| Scaler
Scale -->|"model_inputs"| Model

%% Inference artifacts
Model -->|"loads"| PTHPrimary
Model -.->|"optional_load"| PTHAlt
Model -->|"outputs"| Preds

%% Postprocess/optimization
Preds -->|"to_postprocess"| Calib
Calib -->|"loads"| BinW
Calib -->|"feeds"| Interp
Calib -->|"feeds"| Optimize
Interp -->|"explains"| OutPlots
Optimize -->|"writes"| OutOff
Optimize -->|"writes"| OutGuides
Optimize -->|"reports"| OutPlots

end

%% =========================
%% Offline / Training & Benchmark
%% =========================
subgraph "Offline: Model Development (Datasets -> Artifacts)"
direction TB

subgraph "Training / Benchmark Datasets"
direction TB
DSAll["datasets/all_off_target.csv"]:::data
DSChange["datasets/changeseq_siteseq.csv"]:::data
DSCircle["datasets/circleseq_all.csv"]:::data
DSGuide["datasets/guideseq.csv"]:::data
DSSurro["datasets/surroseq.csv"]:::data
DSTT["datasets/ttiss.csv"]:::data
end

Train["Training/validation pipeline (implicit)"]:::process
Artifacts["Saved model & preprocessing artifacts"]:::artifactGroup

DSAll -->|"train/validate"| Train
DSChange -->|"train/validate"| Train
DSCircle -->|"train/validate"| Train
DSGuide -->|"train/validate"| Train
DSSurro -->|"train/validate"| Train
DSTT -->|"train/validate"| Train

Train -->|"produces"| Artifacts
Artifacts -->|"checkpoint"| PTHPrimary
Artifacts -->|"scaler"| Scaler
Artifacts -->|"binning"| BinW
Artifacts -.->|"alt_checkpoint"| PTHAlt

end

%% =========================
%% Click events (mapped paths)
%% =========================
click CRISMER "https://github.com/anamul-hoque-emtiaj/crismer/blob/main/crismer-opti/CRISMER.py"
click Mods "https://github.com/anamul-hoque-emtiaj/crismer/blob/main/crismer-opti/crismer_modules.py"
click Utils "https://github.com/anamul-hoque-emtiaj/crismer/blob/main/crismer-opti/utils.py"

click NBParams "https://github.com/anamul-hoque-emtiaj/crismer/blob/main/crismer-opti/crismer-opti-params.ipynb"
click NB1 "https://github.com/anamul-hoque-emtiaj/crismer/blob/main/test-scenario-1.ipynb"
click NB2 "https://github.com/anamul-hoque-emtiaj/crismer/blob/main/test-scenario-2.ipynb"

click InCSV "https://github.com/anamul-hoque-emtiaj/crismer/blob/main/crismer-opti/example/example.csv"
click InTXT "https://github.com/anamul-hoque-emtiaj/crismer/blob/main/crismer-opti/example/example_sgRNAs.txt"
click GenomeTest "https://github.com/anamul-hoque-emtiaj/crismer/blob/main/crismer-opti/data/test_genome.fa"

click CasWrap "https://github.com/anamul-hoque-emtiaj/crismer/blob/main/crismer-opti/script/casoffinder_genome.sh"

click PTHPrimary "https://github.com/anamul-hoque-emtiaj/crismer/blob/main/crismer-opti/models/change_site_circleseq_model.pth"
click Scaler "https://github.com/anamul-hoque-emtiaj/crismer/blob/main/crismer-opti/models/minmax_scaler.pkl"
click BinW "https://github.com/anamul-hoque-emtiaj/crismer/blob/main/crismer-opti/models/bin_weights.pkl"
click PTHAlt "https://github.com/anamul-hoque-emtiaj/crismer/blob/main/models/IDT.pth"

click DSAll "https://github.com/anamul-hoque-emtiaj/crismer/blob/main/datasets/all_off_target.csv"
click DSChange "https://github.com/anamul-hoque-emtiaj/crismer/blob/main/datasets/changeseq_siteseq.csv"
click DSCircle "https://github.com/anamul-hoque-emtiaj/crismer/blob/main/datasets/circleseq_all.csv"
click DSGuide "https://github.com/anamul-hoque-emtiaj/crismer/blob/main/datasets/guideseq.csv"
click DSSurro "https://github.com/anamul-hoque-emtiaj/crismer/blob/main/datasets/surroseq.csv"
click DSTT "https://github.com/anamul-hoque-emtiaj/crismer/blob/main/datasets/ttiss.csv"

%% =========================
%% Styles
%% =========================
classDef external fill:#111827,stroke:#374151,color:#F9FAFB,stroke-width:1.5px
classDef notebook fill:#EEF2FF,stroke:#4338CA,color:#111827,stroke-width:1.5px
classDef entry fill:#DBEAFE,stroke:#1D4ED8,color:#111827,stroke-width:1.5px
classDef module fill:#E0F2FE,stroke:#0284C7,color:#0F172A,stroke-width:1.5px
classDef script fill:#FFE4E6,stroke:#E11D48,color:#111827,stroke-width:1.5px

classDef process fill:#ECFDF5,stroke:#10B981,color:#064E3B,stroke-width:1.5px
classDef data fill:#FFF7ED,stroke:#FB923C,color:#7C2D12,stroke-width:1.5px
classDef artifact fill:#F5F3FF,stroke:#7C3AED,color:#2E1065,stroke-width:1.5px
classDef artifactGroup fill:#F5F3FF,stroke:#6D28D9,color:#2E1065,stroke-width:2px,stroke-dasharray: 4 3
classDef output fill:#FEFCE8,stroke:#CA8A04,color:#713F12,stroke-width:1.5px
```
