# dhruv16s/chesssense architecture

> Architecture diagram of the GitHub repository dhruv16s/chesssense, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dhruv16s/chesssense
- Repository: https://github.com/dhruv16s/chesssense

## Overview

1. **What this repository is**
   ChessSense is a local Python computer-vision pipeline, not a web service. Its job is to take a chessboard image captured from the screen, detect the board and pieces, convert the position to FEN, and query a bundled Stockfish engine for a best move.

2. **Top-level orchestration**
   `app.py` is the entry point and likely coordinates the whole flow end-to-end: acquire image/frame, detect the board, infer piece locations, build FEN, and request an engine move recommendation. That makes it the main system boundary for the application runtime.

3. **Input domain and assumptions**
   The project is tuned for a specific chess.com visual style: Neo pieces on the green board configuration. This matters architecturally because the recognition pipeline depends on reference templates and calibrated thresholds rather than being generic across arbitrary chess UIs.

4. **Board detection subsystem**
   `detect_board.py` appears to locate the chessboard within a larger screenshot using contouring and other image-processing steps. `board_lines.py` then detects or reconstructs the board’s internal grid so the board can be split into 64 cells. This is the geometric foundation for everything downstream.

5. **Piece recognition subsystem**
   `recognize_pieces_positions.py` is the core classifier/matcher for identifying which piece occupies each square. It likely compares board-cell crops against template images rather than using a learned model. The `pieces/` and `resized_pieces/` directories provide the reference images used for matching.

6. **Template and asset storage**
   `board_templates/` holds board reference images, while `pieces/black_pieces` and `pieces/white_pieces` contain piece templates under different background conditions. `resized_pieces/` is a processed copy sized for matching. These folders are data assets, not executable logic, and they form the recognition model’s “training/reference set.”

7. **Color-threshold calibration**
   The `thresholds/` directory exists to separate white and black pieces using color statistics. `thresholds/define_thresholds.py` and the plots in that folder indicate an offline calibration step used to derive decision boundaries for piece-color classification. This is important because the board/piece colors are part of the recognition signal.

8. **Image preprocessing utilities**
   `image_augmentation.py` and `image_resizing.py` prepare the template data into usable forms for matching. This is an offline preprocessing layer that supports robustness by creating consistent template sizes and possibly more varied samples.

9. **Position encoding**
   `fen_notation.py` converts the recognized piece layout into a FEN string. This is a key boundary: it transforms vision output into a chess-engine-friendly representation, decoupling image analysis from chess logic.

10. **Move suggestion engine**
   `get_best_move.py` uses the generated FEN to query `stockfish.exe`, which is bundled in the repo. This is a separate external computation boundary: the computer vision system hands off board state to a deterministic engine process and receives a best move back.

11. **Data flow through the system**
   The runtime flow is: screen/image input → board detection → board segmentation into squares → template/threshold-based piece recognition → FEN generation → Stockfish evaluation → recommended move. Intermediate outputs may be saved for inspection, especially in the board-detection stage.

12. **Operational footprint**
   `requirements.txt` defines the Python environment, while the bundled `stockfish.exe` makes the engine dependency local and self-contained. Overall, the repository is a single-machine computer-vision pipeline with offline-prepared assets and a lightweight runtime orchestrator.

Main entry points: [`recognize_pieces_positions.py`](https://github.com/dhruv16s/chesssense/blob/main/recognize_pieces_positions.py) (Recognize pieces), [`detect_board.py`](https://github.com/dhruv16s/chesssense/blob/main/detect_board.py) (Detect board), [`image_augmentation.py`](https://github.com/dhruv16s/chesssense/blob/main/image_augmentation.py) (Augment images), [`image_resizing.py`](https://github.com/dhruv16s/chesssense/blob/main/image_resizing.py) (Resize images)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Runtime pipeline"]
  node_app["App<br/>entrypoint<br/>[app.py]"]
  node_detect_board["Detect board<br/>vision stage<br/>[detect_board.py]"]
  node_board_lines["Board grid<br/>geometry stage<br/>[board_lines.py]"]
  node_recognize_pieces["Recognize pieces<br/>classification stage"]
  node_fen_notation["FEN encode<br/>translation stage<br/>[fen_notation.py]"]
  node_get_best_move["Best move<br/>engine bridge<br/>[get_best_move.py]"]
  node_stockfish["Stockfish<br/>engine binary<br/>[stockfish.exe]"]
end

subgraph group_assets["Reference assets"]
  node_board_templates["Board templates<br/>asset set"]
  node_pieces_black["Black pieces<br/>template set"]
  node_pieces_white["White pieces<br/>template set"]
  node_resized_black["Resized black<br/>derived templates"]
  node_resized_white["Resized white<br/>derived templates"]
  node_augment["Augment images<br/>preprocess script"]
  node_resize["Resize images<br/>preprocess script<br/>[image_resizing.py]"]
end

subgraph group_calibration["Calibration"]
  node_thresholds["Thresholds<br/>calibration data"]
  node_define_thresholds["Define thresholds<br/>calibration script"]
end

node_app -->|"capture frame"| node_detect_board
node_detect_board -->|"use templates"| node_board_templates
node_detect_board -->|"board region"| node_board_lines
node_board_lines -->|"64 squares"| node_recognize_pieces
node_recognize_pieces -->|"match black"| node_pieces_black
node_recognize_pieces -->|"match white"| node_pieces_white
node_recognize_pieces -->|"aligned refs"| node_resized_black
node_recognize_pieces -->|"aligned refs"| node_resized_white
node_recognize_pieces -->|"color cues"| node_thresholds
node_recognize_pieces -->|"piece map"| node_fen_notation
node_fen_notation -->|"fen"| node_get_best_move
node_get_best_move -->|"query"| node_stockfish
node_augment -->|"generate"| node_pieces_black
node_augment -->|"generate"| node_pieces_white
node_resize -->|"build"| node_resized_black
node_resize -->|"build"| node_resized_white
node_define_thresholds -->|"calibrate"| node_thresholds

click node_app "https://github.com/dhruv16s/chesssense/blob/main/app.py"
click node_detect_board "https://github.com/dhruv16s/chesssense/blob/main/detect_board.py"
click node_board_lines "https://github.com/dhruv16s/chesssense/blob/main/board_lines.py"
click node_recognize_pieces "https://github.com/dhruv16s/chesssense/blob/main/recognize_pieces_positions.py"
click node_fen_notation "https://github.com/dhruv16s/chesssense/blob/main/fen_notation.py"
click node_get_best_move "https://github.com/dhruv16s/chesssense/blob/main/get_best_move.py"
click node_stockfish "https://github.com/dhruv16s/chesssense/blob/main/stockfish.exe"
click node_board_templates "https://github.com/dhruv16s/chesssense/tree/main/board_templates"
click node_pieces_black "https://github.com/dhruv16s/chesssense/tree/main/pieces/black_pieces"
click node_pieces_white "https://github.com/dhruv16s/chesssense/tree/main/pieces/white_pieces"
click node_resized_black "https://github.com/dhruv16s/chesssense/tree/main/resized_pieces/black_pieces"
click node_resized_white "https://github.com/dhruv16s/chesssense/tree/main/resized_pieces/white_pieces"
click node_thresholds "https://github.com/dhruv16s/chesssense/tree/main/thresholds"
click node_define_thresholds "https://github.com/dhruv16s/chesssense/blob/main/thresholds/define_thresholds.py"
click node_augment "https://github.com/dhruv16s/chesssense/blob/main/image_augmentation.py"
click node_resize "https://github.com/dhruv16s/chesssense/blob/main/image_resizing.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_detect_board,node_board_lines,node_recognize_pieces,node_fen_notation,node_get_best_move,node_stockfish toneBlue
class node_board_templates,node_pieces_black,node_pieces_white,node_resized_black,node_resized_white,node_augment,node_resize toneAmber
class node_thresholds,node_define_thresholds toneMint
```

## Components

### Runtime pipeline

Live image-to-move processing path

- [App](https://github.com/dhruv16s/chesssense/blob/main/app.py): entrypoint · Main orchestrator for capture, vision, FEN, and engine query · `app.py`
- [Detect board](https://github.com/dhruv16s/chesssense/blob/main/detect_board.py): vision stage · Finds the chessboard in a screenshot · `detect_board.py`
- [Board grid](https://github.com/dhruv16s/chesssense/blob/main/board_lines.py): geometry stage · Reconstructs board lines and 64-square layout · `board_lines.py`
- [Recognize pieces](https://github.com/dhruv16s/chesssense/blob/main/recognize_pieces_positions.py): classification stage · Matches square crops to piece templates and colors · `recognize_pieces_positions.py`
- [FEN encode](https://github.com/dhruv16s/chesssense/blob/main/fen_notation.py): translation stage · Converts recognized board state into FEN · `fen_notation.py`
- [Best move](https://github.com/dhruv16s/chesssense/blob/main/get_best_move.py): engine bridge · Sends FEN to Stockfish and returns a recommendation · `get_best_move.py`
- [Stockfish](https://github.com/dhruv16s/chesssense/blob/main/stockfish.exe): engine binary · Bundled chess engine used for evaluation · `stockfish.exe`

### Reference assets

Template images and generated match data

- [Board templates](https://github.com/dhruv16s/chesssense/tree/main/board_templates): asset set · Reference board images for board detection · `board_templates`
- [Black pieces](https://github.com/dhruv16s/chesssense/tree/main/pieces/black_pieces): template set · Black piece reference images · `pieces/black_pieces`
- [White pieces](https://github.com/dhruv16s/chesssense/tree/main/pieces/white_pieces): template set · White piece reference images · `pieces/white_pieces`
- [Resized black](https://github.com/dhruv16s/chesssense/tree/main/resized_pieces/black_pieces): derived templates · Processed black piece templates for matching · `resized_pieces/black_pieces`
- [Resized white](https://github.com/dhruv16s/chesssense/tree/main/resized_pieces/white_pieces): derived templates · Processed white piece templates for matching · `resized_pieces/white_pieces`
- [Augment images](https://github.com/dhruv16s/chesssense/blob/main/image_augmentation.py): preprocess script · Creates varied training/reference images · `image_augmentation.py`
- [Resize images](https://github.com/dhruv16s/chesssense/blob/main/image_resizing.py): preprocess script · Normalizes template dimensions for matching · `image_resizing.py`

### Calibration

Offline threshold tuning and analysis

- [Thresholds](https://github.com/dhruv16s/chesssense/tree/main/thresholds): calibration data · Stored calibration artifacts for color-based separation · `thresholds`
- [Define thresholds](https://github.com/dhruv16s/chesssense/blob/main/thresholds/define_thresholds.py): calibration script · Offline tool for deriving piece-color thresholds · `thresholds/define_thresholds.py`

## Connections

- App → Detect board: capture frame · Start vision pipeline on an input image
- Detect board → Board templates: use templates · Matches board appearance against board references
- Detect board → Board grid: board region · Pass detected board for grid reconstruction
- Board grid → Recognize pieces: 64 squares · Produce square crops for piece identification
- Recognize pieces → Black pieces: match black · Compare crops against black piece templates
- Recognize pieces → White pieces: match white · Compare crops against white piece templates
- Recognize pieces → Resized black: aligned refs · Use resized black templates during matching
- Recognize pieces → Resized white: aligned refs · Use resized white templates during matching
- Recognize pieces → Thresholds: color cues · Apply calibrated thresholds to separate piece colors
- Recognize pieces → FEN encode: piece map · Send recognized board layout for encoding
- FEN encode → Best move: fen · Convert board state into engine input
- Best move → Stockfish: query · Invoke bundled engine process
- Augment images → Black pieces: generate · Preprocess source templates into augmented references
- Augment images → White pieces: generate · Preprocess source templates into augmented references
- Resize images → Resized black: build · Create matching-ready black templates
- Resize images → Resized white: build · Create matching-ready white templates
- Define thresholds → Thresholds: calibrate · Derive and store threshold values
