# cmwaseemyousef/pid-pipe-removal architecture

> Architecture diagram of the GitHub repository cmwaseemyousef/pid-pipe-removal, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cmwaseemyousef/pid-pipe-removal
- Repository: https://github.com/cmwaseemyousef/pid-pipe-removal

## Overview

##### File structure
- Top-level scripts each implement one stage of the pipeline:
- preprocess.py – initial image cleanup (e.g., resizing, denoising)
- generate_mask.py – computes a binary pipe mask via grayscale differencing & thresholding
- detect_symbols.py – (optional) identifies and protects critical symbols
- remove_pipes.py / remove_pipes_final.py – applies inpainting to remove pipes
- generate_pipe_free_output.py – orchestrates the full pipeline end-to-end
- train_unet.py – an abandoned U-Net–based approach
- Two data folders:
- data/ – holds input.jpg, nearest_expected_output.jpg, final_output.jpg
- results/ – holds intermediate previews such as mask_preview.png

##### README for additional insights
- Confirms a linear, five-step pipeline: load images → grayscale diff → mask generation (threshold + dilation) → inpainting → save outputs.
- No external services or databases—entirely local processing in Python.

a. Main components:
1. Input Module
- Reads original P&ID (image or PDF)
- Script: convert_pdf.py (if starting from PDF) or direct image load
2. Preprocessing Module
- Normalizes image (resize, denoise)
- Script: preprocess.py
3. Mask Generation Module
- Computes `cv2.absdiff` between original and reference
- Applies threshold + morphological dilation
- Script: generate_mask.py
4. Symbol Detection Module (optional)
- Identifies and protects valves, sensors, labels
- Script: detect_symbols.py
5. Inpainting Module
- Uses `cv2.inpaint` with the binary mask to remove pipes
- Scripts: remove_pipes.py & remove_pipes_final.py
6. Orchestrator / Runner
- Ties all stages together end-to-end
- Script: generate_pipe_free_output.py
7. Output Module
- Writes final cleaned P&ID to data/final_output.jpg
- Saves results/mask_preview.png

b. Relationships and interactions:
- Data flows strictly left→right through the six processing stages.
- Preprocessing outputs feed into mask generation; mask feeds into inpainting; final is written to disk.
- The orchestrator module imports and invokes each stage in sequence.

c. Architectural patterns / design principles:
- Linear batch pipeline: single-pass ETL style
- Single responsibility: each script handles one logical step
- Modular design: easy to swap out mask generation (e.g., U-Net) or symbol detection

d. Key technologies & libraries:
- Python 3.x
- OpenCV (cv2.absdiff, threshold, dilation, inpaint)
- NumPy (array ops)
- Google Colab (dev environment, optional)

## Diagram (Mermaid)

```mermaid
flowchart TD
    Orchestrator["Orchestrator / Runner\ngenerate_pipe_free_output.py"]:::proc

    Orchestrator --> Input
    Input["Input Module\nconvert_pdf.py\n(Load image/PDF)"]:::io
    Input --> Preprocess

    Preprocess["Preprocessing Module\npreprocess.py\n(resize, denoise)"]:::proc
    Preprocess --> MaskGen

    MaskGen["Mask Generation Module\ngenerate_mask.py\n(diff + threshold + dilation)"]:::proc
    MaskGen --> Symbol
    MaskGen -.-> Deprecated

    Symbol["Symbol Detection Module\ndetect_symbols.py\n(optional protection)"]:::optional
    Symbol --> Inpaint1

    Inpaint1["Inpainting Module\nremove_pipes.py"]:::proc
    Inpaint1 --> Inpaint2

    Inpaint2["Inpainting Module\nremove_pipes_final.py"]:::proc
    Inpaint2 --> Out1
    Inpaint2 --> Out2

    subgraph "Output Module"
        direction TB
        Out1["data/\n(final_output.jpg)"]:::io
        Out2["results/\n(mask_preview.png)"]:::io
    end

    Deprecated["U-Net Training Branch\ntrain_unet.py"]:::optional

    %% Click Events
    click Input "https://github.com/cmwaseemyousef/pid-pipe-removal/blob/main/convert_pdf.py"
    click Preprocess "https://github.com/cmwaseemyousef/pid-pipe-removal/blob/main/preprocess.py"
    click MaskGen "https://github.com/cmwaseemyousef/pid-pipe-removal/blob/main/generate_mask.py"
    click Symbol "https://github.com/cmwaseemyousef/pid-pipe-removal/blob/main/detect_symbols.py"
    click Inpaint1 "https://github.com/cmwaseemyousef/pid-pipe-removal/blob/main/remove_pipes.py"
    click Inpaint2 "https://github.com/cmwaseemyousef/pid-pipe-removal/blob/main/remove_pipes_final.py"
    click Orchestrator "https://github.com/cmwaseemyousef/pid-pipe-removal/blob/main/generate_pipe_free_output.py"
    click Out1 "https://github.com/cmwaseemyousef/pid-pipe-removal/tree/main/data/"
    click Out2 "https://github.com/cmwaseemyousef/pid-pipe-removal/tree/main/results/"
    click Deprecated "https://github.com/cmwaseemyousef/pid-pipe-removal/blob/main/train_unet.py"

    %% Styles
    classDef proc fill:#BBE1FA,stroke:#1E90FF;
    classDef io fill:#BFFCC6,stroke:#2E8B57;
    classDef optional fill:#FFE1A8,stroke:#FF8C00,stroke-dasharray: 5 5;
```
