# t-mclaurin/individual-penguin-identification architecture

> Architecture diagram of the GitHub repository t-mclaurin/individual-penguin-identification, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/t-mclaurin/individual-penguin-identification
- Repository: https://github.com/t-mclaurin/individual-penguin-identification

## Overview

This research project builds an open-set penguin re-identification workflow: prepare individual images, train an embedding model, evaluate it, and use embeddings for visualization or population estimation. The README documents the broader workflow; sampled source confirms image cropping, tag occlusion, splitting, model construction, and embedding/population routines. Some training and evaluation wiring is documented but not fully visible in the excerpts, so links between those stages follow the README rather than inferred helper calls.

Main entry points: [`Code/train_gridsearch.py`](https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/train_gridsearch.py) (Grid-search training), [`Code/crop.py`](https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/crop.py) (Detection crops), [`Code/utils/create_tf_data_csv.py`](https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/utils/create_tf_data_csv.py) (Dataset splitting), [`Code/pop_est_test.py`](https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/pop_est_test.py) (Known threshold test)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_preparation["Image preparation"]
  node_crop["Detection crops<br/>[crop.py]"]
  node_occluder["Tag occlusion<br/>[tag_occluder.py]"]
  node_normalise["Histogram normalization"]
  node_split["Dataset splitting"]
end

subgraph group_learning["Model learning"]
  node_trainer["Grid-search training"]
  node_model["Embedding model<br/>[model.py]"]
  node_augmentation["Image augmentation<br/>[augmentation.py]"]
  node_mining["Triplet mining<br/>[mining.py]"]
  node_evaluation["Recognition evaluation"]
end

subgraph group_analysis["Embedding analysis"]
  node_plainviz["Embedding map"]
  node_colourviz["Split-coloured map"]
  node_medoidviz["Medoid and poles"]
end

subgraph group_population["Population estimation"]
  node_knownest["Known threshold test<br/>[pop_est_test.py]"]
  node_unknownest["Unlabelled estimate"]
end

node_researcher(("Researcher"))
node_photos["Source photos"]
node_detector["MegaDetector"]
node_detections["Detection JSON"]
node_splitcsv["Split metadata"]
node_weights["Trained weights"]
node_estimateplot["Population results"]

node_researcher -->|"provides"| node_photos
node_researcher -->|"runs detection"| node_detector
node_photos -->|"scans"| node_detector
node_detector -->|"writes results"| node_detections
node_researcher -->|"configures"| node_crop
node_photos -->|"reads images"| node_crop
node_detections -->|"provides boxes"| node_crop
node_crop -->|"supplies images"| node_occluder
node_occluder -.->|"copies images"| node_normalise
node_researcher -->|"runs split"| node_split
node_split -->|"reads dataset"| node_photos
node_split -->|"writes labels"| node_splitcsv
node_researcher -->|"starts search"| node_trainer
node_trainer -->|"uses splits"| node_splitcsv
node_trainer -->|"builds model"| node_model
node_trainer -->|"saves weights"| node_weights
node_trainer -->|"computes metrics"| node_evaluation
node_trainer -.->|"uses augmentation"| node_augmentation
node_trainer -.->|"uses mining"| node_mining
node_researcher -->|"runs visualization"| node_plainviz
node_weights -->|"loads weights"| node_plainviz
node_researcher -->|"runs visualization"| node_colourviz
node_weights -->|"loads weights"| node_colourviz
node_splitcsv -->|"labels images"| node_colourviz
node_researcher -->|"runs visualization"| node_medoidviz
node_weights -->|"loads weights"| node_medoidviz
node_splitcsv -->|"labels images"| node_medoidviz
node_researcher -->|"tests thresholds"| node_knownest
node_weights -->|"provides model"| node_knownest
node_researcher -->|"estimates population"| node_unknownest
node_weights -->|"loads weights"| node_unknownest
node_unknownest -->|"plots counts"| node_estimateplot
node_knownest -->|"plots errors"| node_estimateplot

click node_crop "https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/crop.py"
click node_occluder "https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/tag_occluder.py"
click node_normalise "https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/database_preprocess.py"
click node_split "https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/utils/create_tf_data_csv.py"
click node_trainer "https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/train_gridsearch.py"
click node_model "https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/model.py"
click node_augmentation "https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/utils/augmentation.py"
click node_mining "https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/utils/mining.py"
click node_evaluation "https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/train_gridsearch.py"
click node_plainviz "https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/updated_visualise_embeddings.py"
click node_colourviz "https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/colour_visualise_embeddings.py"
click node_medoidviz "https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/medoid_poles_vis_embeddings.py"
click node_knownest "https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/pop_est_test.py"
click node_unknownest "https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/unlabelled_pop_est.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_crop,node_occluder,node_normalise,node_split toneBlue
class node_trainer,node_model,node_augmentation,node_mining,node_evaluation toneAmber
class node_plainviz,node_colourviz,node_medoidviz toneMint
class node_knownest,node_unknownest toneRose
class node_researcher,node_photos,node_detector,node_detections,node_splitcsv,node_weights,node_estimateplot toneIndigo
```

## Components

### Image preparation

- [Detection crops](https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/crop.py): `Code/crop.py`
- [Tag occlusion](https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/tag_occluder.py): `Code/tag_occluder.py`
- [Histogram normalization](https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/database_preprocess.py): `Code/database_preprocess.py`
- [Dataset splitting](https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/utils/create_tf_data_csv.py): `Code/utils/create_tf_data_csv.py`

### Model learning

- [Grid-search training](https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/train_gridsearch.py): `Code/train_gridsearch.py`
- [Embedding model](https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/model.py): `Code/model.py`
- [Image augmentation](https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/utils/augmentation.py): `Code/utils/augmentation.py`
- [Triplet mining](https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/utils/mining.py): `Code/utils/mining.py`
- [Recognition evaluation](https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/train_gridsearch.py): `Code/train_gridsearch.py`

### Embedding analysis

- [Embedding map](https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/updated_visualise_embeddings.py): `Code/updated_visualise_embeddings.py`
- [Split-coloured map](https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/colour_visualise_embeddings.py): `Code/colour_visualise_embeddings.py`
- [Medoid and poles](https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/medoid_poles_vis_embeddings.py): `Code/medoid_poles_vis_embeddings.py`

### Population estimation

- [Known threshold test](https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/pop_est_test.py): `Code/pop_est_test.py`
- [Unlabelled estimate](https://github.com/t-mclaurin/individual-penguin-identification/blob/main/Code/unlabelled_pop_est.py): `Code/unlabelled_pop_est.py`

### Other components

- **Researcher**
- **Source photos**
- **MegaDetector**
- **Detection JSON**
- **Split metadata**
- **Trained weights**
- **Population results**

## Connections

- Researcher → Source photos: provides
- Researcher → MegaDetector: runs detection
- Source photos → MegaDetector: scans
- MegaDetector → Detection JSON: writes results
- Researcher → Detection crops: configures
- Source photos → Detection crops: reads images
- Detection JSON → Detection crops: provides boxes
- Detection crops → Tag occlusion: supplies images
- Tag occlusion → Histogram normalization: copies images
- Researcher → Dataset splitting: runs split
- Dataset splitting → Source photos: reads dataset
- Dataset splitting → Split metadata: writes labels
- Researcher → Grid-search training: starts search
- Grid-search training → Split metadata: uses splits
- Grid-search training → Embedding model: builds model
- Grid-search training → Trained weights: saves weights
- Grid-search training → Recognition evaluation: computes metrics
- Grid-search training → Image augmentation: uses augmentation
- Grid-search training → Triplet mining: uses mining
- Researcher → Embedding map: runs visualization
- Trained weights → Embedding map: loads weights
- Researcher → Split-coloured map: runs visualization
- Trained weights → Split-coloured map: loads weights
- Split metadata → Split-coloured map: labels images
- Researcher → Medoid and poles: runs visualization
- Trained weights → Medoid and poles: loads weights
- Split metadata → Medoid and poles: labels images
- Researcher → Known threshold test: tests thresholds
- Trained weights → Known threshold test: provides model
- Researcher → Unlabelled estimate: estimates population
- Trained weights → Unlabelled estimate: loads weights
- Unlabelled estimate → Population results: plots counts
- Known threshold test → Population results: plots errors