# dakshankarthic/ffdm architecture

> Architecture diagram of the GitHub repository dakshankarthic/ffdm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dakshankarthic/ffdm
- Repository: https://github.com/dakshankarthic/ffdm

## Overview

This repository is a research codebase for detecting manipulated images and localizing edited regions, alongside baseline benchmark runners and an explicitly untrained global/local prototype. The main workflow validates and loads image/mask data, trains detector variants with segmentation and consistency objectives, then performs inference and benchmark scoring. The graph includes documented and indexed relationships; some experiment scripts and model internals are unsampled, so those links are omitted rather than inferred.

Main entry points: [`controlled_localization/src/controlled_localization/eval_benchmark_comparison.py`](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/eval_benchmark_comparison.py) (Benchmark Evaluation), [`controlled_localization/src/controlled_localization/train.py`](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/train.py) (Core Training), [`controlled_localization/src/controlled_localization/models/super_frame_v2_detector.py`](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/models/super_frame_v2_detector.py) (SuperFRAME v2), [`controlled_localization/src/controlled_localization/models/forensic_encoder.py`](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/models/forensic_encoder.py) (Forensic Encoder)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data and Tiling"]
  node_research_inputs["Research Loader<br/>[research_loader.py]"]
  node_data_schema["Manifest Schema<br/>[schema.py]"]
  node_corrected_data["Corrected Dataset"]
  node_tile_ops["Tile Operations<br/>[tiles.py]"]
end

subgraph group_models["Detection Models"]
  node_seg_loss["Segmentation Loss<br/>[segmentation.py]"]
  node_consistency_loss["Consistency Loss<br/>[consistency.py]"]
  node_detector["Base Detector<br/>[detector.py]"]
  node_forensic_encoder["Forensic Encoder"]
  node_bayar_stem["Bayar SRM Stem<br/>[bayar_srm_stem.py]"]
  node_frame_detector["Frame Detector<br/>[frame_detector.py]"]
  node_superframe["SuperFRAME"]
  node_superframe_v2["SuperFRAME v2"]
  node_dino_encoder["DINO Encoder<br/>[dino_encoder.py]"]
  node_cross_decoder["Cross-Attention Decoder"]
end

subgraph group_workflow["Training and Evaluation"]
  node_training["Core Training<br/>[train.py]"]
  node_pilot_training["Matched Pilot"]
  node_kaggle_training["Dual-T4 Training"]
  node_inference["Inference<br/>[infer.py]"]
  node_policy["Decision Policy<br/>[policy.py]"]
  node_evaluation["Benchmark Evaluation"]
  node_metrics["Metrics<br/>[metrics.py]"]
end

subgraph group_research["Baseline and Prototype"]
  node_baseline_eval["Baseline Test Evaluation"]
  node_baseline_adapter["VAAS Adapter"]
  node_prototype["Global-Local Prototype<br/>[model.py]"]
  node_prototype_train["Prototype Training<br/>[train.py]"]
  node_prototype_eval["Prototype Evaluation<br/>[evaluate_pilot.py]"]
end

node_vaas{{"VAAS Model Service"}}

node_research_inputs -->|"validates records"| node_data_schema
node_training -->|"loads batches"| node_research_inputs
node_training -->|"trains"| node_detector
node_training -->|"computes loss"| node_seg_loss
node_training -->|"computes consistency"| node_consistency_loss
node_training -->|"uses policy"| node_policy
node_training -->|"runs validation"| node_evaluation
node_pilot_training -->|"loads records"| node_corrected_data
node_pilot_training -->|"uses tiling"| node_tile_ops
node_pilot_training -->|"trains"| node_superframe_v2
node_pilot_training -->|"computes loss"| node_seg_loss
node_kaggle_training -->|"trains"| node_superframe_v2
node_kaggle_training -->|"uses tiling"| node_tile_ops
node_kaggle_training -->|"computes loss"| node_seg_loss
node_detector -->|"uses encoder"| node_forensic_encoder
node_forensic_encoder -->|"uses stem"| node_bayar_stem
node_frame_detector -->|"uses encoder"| node_forensic_encoder
node_superframe -->|"uses encoder"| node_dino_encoder
node_superframe -->|"uses encoder"| node_forensic_encoder
node_superframe_v2 -->|"uses encoder"| node_dino_encoder
node_superframe_v2 -->|"uses encoder"| node_forensic_encoder
node_superframe_v2 -->|"uses decoder"| node_cross_decoder
node_inference -->|"tiles images"| node_tile_ops
node_inference -->|"runs model"| node_detector
node_inference -->|"applies policy"| node_policy
node_evaluation -->|"evaluates predictions"| node_inference
node_evaluation -->|"computes metrics"| node_metrics
node_evaluation -->|"uses policy"| node_policy
node_evaluation -->|"evaluates variant"| node_frame_detector
node_evaluation -->|"evaluates variant"| node_superframe
node_evaluation -->|"evaluates variant"| node_superframe_v2
node_baseline_eval -->|"loads and predicts"| node_baseline_adapter
node_baseline_adapter -->|"calls pipeline"| node_vaas
node_prototype_train -->|"trains prototype"| node_prototype
node_prototype_eval -->|"evaluates prototype"| node_prototype

click node_research_inputs "https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/data/research_loader.py"
click node_data_schema "https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/data/schema.py"
click node_corrected_data "https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/data/corrected_dataset.py"
click node_tile_ops "https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/data/tiles.py"
click node_training "https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/train.py"
click node_pilot_training "https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/train_matched_pilot.py"
click node_kaggle_training "https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/train_kaggle_dual_t4.py"
click node_seg_loss "https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/losses/segmentation.py"
click node_consistency_loss "https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/losses/consistency.py"
click node_detector "https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/models/detector.py"
click node_forensic_encoder "https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/models/forensic_encoder.py"
click node_bayar_stem "https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/models/bayar_srm_stem.py"
click node_frame_detector "https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/models/frame_detector.py"
click node_superframe "https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/models/super_frame_detector.py"
click node_superframe_v2 "https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/models/super_frame_v2_detector.py"
click node_dino_encoder "https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/models/dino_encoder.py"
click node_cross_decoder "https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/models/cross_attention_decoder.py"
click node_inference "https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/infer.py"
click node_policy "https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/policy.py"
click node_evaluation "https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/eval_benchmark_comparison.py"
click node_metrics "https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/metrics.py"
click node_baseline_eval "https://github.com/dakshankarthic/ffdm/blob/main/baseline_work/code/evaluate_test_sets.py"
click node_baseline_adapter "https://github.com/dakshankarthic/ffdm/blob/main/baseline_work/code/baseline_adapter.py"
click node_prototype "https://github.com/dakshankarthic/ffdm/blob/main/echoseam_research/model.py"
click node_prototype_train "https://github.com/dakshankarthic/ffdm/blob/main/echoseam_research/train.py"
click node_prototype_eval "https://github.com/dakshankarthic/ffdm/blob/main/echoseam_research/evaluate_pilot.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_research_inputs,node_data_schema,node_corrected_data,node_tile_ops toneBlue
class node_seg_loss,node_consistency_loss,node_detector,node_forensic_encoder,node_bayar_stem,node_frame_detector,node_superframe,node_superframe_v2,node_dino_encoder,node_cross_decoder toneAmber
class node_training,node_pilot_training,node_kaggle_training,node_inference,node_policy,node_evaluation,node_metrics toneMint
class node_baseline_eval,node_baseline_adapter,node_prototype,node_prototype_train,node_prototype_eval toneRose
class node_vaas toneIndigo
```

## Components

### Data and Tiling

- [Research Loader](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/data/research_loader.py): `controlled_localization/src/controlled_localization/data/research_loader.py`
- [Manifest Schema](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/data/schema.py): `controlled_localization/src/controlled_localization/data/schema.py`
- [Corrected Dataset](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/data/corrected_dataset.py): `controlled_localization/src/controlled_localization/data/corrected_dataset.py`
- [Tile Operations](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/data/tiles.py): `controlled_localization/src/controlled_localization/data/tiles.py`

### Detection Models

- [Segmentation Loss](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/losses/segmentation.py): `controlled_localization/src/controlled_localization/losses/segmentation.py`
- [Consistency Loss](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/losses/consistency.py): `controlled_localization/src/controlled_localization/losses/consistency.py`
- [Base Detector](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/models/detector.py): `controlled_localization/src/controlled_localization/models/detector.py`
- [Forensic Encoder](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/models/forensic_encoder.py): `controlled_localization/src/controlled_localization/models/forensic_encoder.py`
- [Bayar SRM Stem](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/models/bayar_srm_stem.py): `controlled_localization/src/controlled_localization/models/bayar_srm_stem.py`
- [Frame Detector](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/models/frame_detector.py): `controlled_localization/src/controlled_localization/models/frame_detector.py`
- [SuperFRAME](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/models/super_frame_detector.py): `controlled_localization/src/controlled_localization/models/super_frame_detector.py`
- [SuperFRAME v2](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/models/super_frame_v2_detector.py): `controlled_localization/src/controlled_localization/models/super_frame_v2_detector.py`
- [DINO Encoder](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/models/dino_encoder.py): `controlled_localization/src/controlled_localization/models/dino_encoder.py`
- [Cross-Attention Decoder](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/models/cross_attention_decoder.py): `controlled_localization/src/controlled_localization/models/cross_attention_decoder.py`

### Training and Evaluation

- [Core Training](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/train.py): `controlled_localization/src/controlled_localization/train.py`
- [Matched Pilot](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/train_matched_pilot.py): `controlled_localization/src/controlled_localization/train_matched_pilot.py`
- [Dual-T4 Training](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/train_kaggle_dual_t4.py): `controlled_localization/src/controlled_localization/train_kaggle_dual_t4.py`
- [Inference](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/infer.py): `controlled_localization/src/controlled_localization/infer.py`
- [Decision Policy](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/policy.py): `controlled_localization/src/controlled_localization/policy.py`
- [Benchmark Evaluation](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/eval_benchmark_comparison.py): `controlled_localization/src/controlled_localization/eval_benchmark_comparison.py`
- [Metrics](https://github.com/dakshankarthic/ffdm/blob/main/controlled_localization/src/controlled_localization/metrics.py): `controlled_localization/src/controlled_localization/metrics.py`

### Baseline and Prototype

- [Baseline Test Evaluation](https://github.com/dakshankarthic/ffdm/blob/main/baseline_work/code/evaluate_test_sets.py): `baseline_work/code/evaluate_test_sets.py`
- [VAAS Adapter](https://github.com/dakshankarthic/ffdm/blob/main/baseline_work/code/baseline_adapter.py): `baseline_work/code/baseline_adapter.py`
- [Global-Local Prototype](https://github.com/dakshankarthic/ffdm/blob/main/echoseam_research/model.py): `echoseam_research/model.py`
- [Prototype Training](https://github.com/dakshankarthic/ffdm/blob/main/echoseam_research/train.py): `echoseam_research/train.py`
- [Prototype Evaluation](https://github.com/dakshankarthic/ffdm/blob/main/echoseam_research/evaluate_pilot.py): `echoseam_research/evaluate_pilot.py`

### Other components

- **VAAS Model Service**

## Connections

- Research Loader → Manifest Schema: validates records (evidence: `controlled_localization/src/controlled_localization/data/research_loader.py`)
- Core Training → Research Loader: loads batches (evidence: `controlled_localization/src/controlled_localization/train.py`)
- Core Training → Base Detector: trains (evidence: `controlled_localization/src/controlled_localization/train.py`)
- Core Training → Segmentation Loss: computes loss (evidence: `controlled_localization/src/controlled_localization/train.py`)
- Core Training → Consistency Loss: computes consistency (evidence: `controlled_localization/src/controlled_localization/train.py`)
- Core Training → Decision Policy: uses policy (evidence: `controlled_localization/src/controlled_localization/train.py`)
- Core Training → Benchmark Evaluation: runs validation (evidence: `controlled_localization/src/controlled_localization/train.py`)
- Matched Pilot → Corrected Dataset: loads records (evidence: `controlled_localization/src/controlled_localization/train_matched_pilot.py`)
- Matched Pilot → Tile Operations: uses tiling (evidence: `controlled_localization/src/controlled_localization/train_matched_pilot.py`)
- Matched Pilot → SuperFRAME v2: trains (evidence: `controlled_localization/src/controlled_localization/train_matched_pilot.py`)
- Matched Pilot → Segmentation Loss: computes loss (evidence: `controlled_localization/src/controlled_localization/train_matched_pilot.py`)
- Dual-T4 Training → SuperFRAME v2: trains (evidence: `controlled_localization/src/controlled_localization/train_kaggle_dual_t4.py`)
- Dual-T4 Training → Tile Operations: uses tiling (evidence: `controlled_localization/src/controlled_localization/train_kaggle_dual_t4.py`)
- Dual-T4 Training → Segmentation Loss: computes loss (evidence: `controlled_localization/src/controlled_localization/train_kaggle_dual_t4.py`)
- Base Detector → Forensic Encoder: uses encoder (evidence: `controlled_localization/src/controlled_localization/models/detector.py`)
- Forensic Encoder → Bayar SRM Stem: uses stem (evidence: `controlled_localization/src/controlled_localization/models/forensic_encoder.py`)
- Frame Detector → Forensic Encoder: uses encoder (evidence: `controlled_localization/src/controlled_localization/models/frame_detector.py`)
- SuperFRAME → DINO Encoder: uses encoder (evidence: `controlled_localization/src/controlled_localization/models/super_frame_detector.py`)
- SuperFRAME → Forensic Encoder: uses encoder (evidence: `controlled_localization/src/controlled_localization/models/super_frame_detector.py`)
- SuperFRAME v2 → DINO Encoder: uses encoder (evidence: `controlled_localization/src/controlled_localization/models/super_frame_v2_detector.py`)
- SuperFRAME v2 → Forensic Encoder: uses encoder (evidence: `controlled_localization/src/controlled_localization/models/super_frame_v2_detector.py`)
- SuperFRAME v2 → Cross-Attention Decoder: uses decoder (evidence: `controlled_localization/src/controlled_localization/models/super_frame_v2_detector.py`)
- Inference → Tile Operations: tiles images (evidence: `controlled_localization/src/controlled_localization/infer.py`)
- Inference → Base Detector: runs model (evidence: `controlled_localization/src/controlled_localization/infer.py`)
- Inference → Decision Policy: applies policy (evidence: `controlled_localization/src/controlled_localization/infer.py`)
- Benchmark Evaluation → Inference: evaluates predictions (evidence: `controlled_localization/src/controlled_localization/eval_benchmark_comparison.py`)
- Benchmark Evaluation → Metrics: computes metrics (evidence: `controlled_localization/src/controlled_localization/eval_benchmark_comparison.py`)
- Benchmark Evaluation → Decision Policy: uses policy (evidence: `controlled_localization/src/controlled_localization/eval_benchmark_comparison.py`)
- Benchmark Evaluation → Frame Detector: evaluates variant (evidence: `controlled_localization/src/controlled_localization/eval_benchmark_comparison.py`)
- Benchmark Evaluation → SuperFRAME: evaluates variant (evidence: `controlled_localization/src/controlled_localization/eval_benchmark_comparison.py`)
- Benchmark Evaluation → SuperFRAME v2: evaluates variant (evidence: `controlled_localization/src/controlled_localization/eval_benchmark_comparison.py`)
- Baseline Test Evaluation → VAAS Adapter: loads and predicts (evidence: `baseline_work/code/evaluate_test_sets.py`)
- VAAS Adapter → VAAS Model Service: calls pipeline (evidence: `baseline_work/code/baseline_adapter.py`)
- Prototype Training → Global-Local Prototype: trains prototype (evidence: `echoseam_research/train.py`)
- Prototype Evaluation → Global-Local Prototype: evaluates prototype (evidence: `echoseam_research/evaluate_pilot.py`)
