# udaygirish/einstein-vision architecture

> Architecture diagram of the GitHub repository udaygirish/einstein-vision, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/udaygirish/einstein-vision
- Repository: https://github.com/udaygirish/einstein-vision

## Overview

##### Project Type & Purpose
- This is a computer vision research/implementation project (RBE 595 Project) that brings together different CV techniques to analyze and interpret visual scenes.
- The project involves both simulation (via Blender) and real image processing modules. Its purpose is to process rendered images (or real images) to detect objects, lanes, human poses, traffic indicators, etc.

##### File Structure
- Top-Level Directories:
- blender: Contains scripts (e.g., blender_json_generator.py, final_render.py, new_renderer.py) and .blend files for simulation. This folder represents the simulation/rendering component.
- config: Holds configuration files for Blender and OpenLane settings, providing parameters and tuning options.
- data: Likely serves as a repository for input data or intermediate files (like flow_output.flo).
- helpers and utilities: These directories include general-purpose helper functions such as OpenCV calibration, logging, frame extraction, and 3D utilities.
- lib: Contains the core library modules implementing almost every CV algorithm or subtask (e.g., lane detection, pose estimation, object detection with YOLO, optical flow, depth estimation).
- models: Hosted pre-trained model scripts (like RAFT for optical flow) and a YOLO weight file (yolov5s.pt) crucial for model inference.
- The existence of multiple main entry scripts (main.py, main2.py, and a “main copy.py”) suggests different launching points or experiments within the pipeline.

##### Insights from the README
- The README specifies a long To-Do list marking the status of various CV tasks such as lane detection, object detection (in 2D/3D), human pose estimation, and more.
- There’s an object structure defined in the README for representing detected objects. This data structure (with fields like 3d_world_coords, bbox_2d, class_name, etc.) is a central element of the data flow.

##### Components & Interactions
a. Main Components:
- Simulation/Rendering Engine: Represented by the blender directory that performs scene rendering and simulation.
- Configuration Manager: Represented by the config directory, feeding runtime parameters to various modules.
- Data Management: The data directory for input files, temporary outputs, and results (including results.json).
- Core Algorithms & CV Modules: The lib directory which contains dedicated modules for object detection (using YOLO variants), lane detection, human pose estimation, optical flow, depth estimation, and more.
- Support Utilities/Helpers: The helpers and utilities directories which provide supplementary functions (OpenCV wrappers, logger, 3D utilities, etc.).
- Model Management: The models directory and additional weight files (like yolov5s.pt) to support CV inference.
- Entry Points: The main.py (and variants) scripts that orchestrate the interactions among these modules.

b. Relationships and Interactions:
- Data Flow: The pipeline starts with simulation outputs from Blender -> configuration parameters are applied -> images and data files are fed into the main process.
- Processing: The main script directs input data to various CV modules contained in lib, which might interact with helper utilities for tasks like calibration, logging, or frame extraction.
- Model Inference: Specific modules call upon pre-trained models (YOLO, RAFT, etc.) from the models directory.
- Result Aggregation: Processed outputs are packaged in a defined object structure (as per README), and results are stored (e.g., in results.json or visualized outputs).
- External Dependencies: Integration with external software (e.g., Blender and possibly OpenCV/PyTorch) is noted.

##### Architectural Patterns & Design Principles
- Modularity: The project is split into clear modules (simulation, configuration, CV algorithms, helpers) which encourages separation of concerns.
- Pipeline Architecture: A sequential data flow where rendered scenes are processed through different detection and estimation modules.
- Reusability: Helper and utility functions, along with configurable parameters, allow for reusability across different detection tasks.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Simulation Layer
    subgraph "Simulation Layer"
        SIM_TITLE["Simulation/Rendering Engine"]:::simulation
        BLENDER_JSON["blender_json_generator.py"]:::file
        FINAL_RENDER["final_render.py"]:::file
        NEW_RENDERER["new_renderer.py"]:::file
        PERSON_RENDERED["person_rendered.py"]:::file
        TRAFFIC_SIGN["traffic_sign_render.py"]:::file
    end

    %% Configuration Layer
    subgraph "Configuration Layer"
        CONF["Configuration Manager"]:::config
    end

    %% Entry Points
    subgraph "Entry Points"
        MAIN["main.py"]:::entry
        MAIN2["main2.py"]:::entry
        MAINCOPY["main copy.py"]:::entry
    end

    %% Processing Layer
    subgraph "Processing Layer"
        CV_TITLE["CV Algorithms"]:::processing
        YOLO3D["yolov3d_infer.py"]:::file
        YOLO8["yolov8_det.py"]:::file
        LANE["clrernet_lane_detect.py"]:::file
        OPTICAL["optical_flow.py"]:::file
        POSE["pose_2d.py"]:::file
        HELP["Helper Functions"]:::helper
        UTIL["Additional Utilities"]:::helper
        MODEL["Model Management"]:::model
    end

    %% Data & Output Layer
    subgraph "Data & Output Layer"
        DATA["Data Management"]:::data
        VIS["Visualization/Diagrams"]:::viz
    end

    %% Relationships
    %% Simulation outputs
    SIM_TITLE --> DATA

    %% Configuration flows to entry points and processing
    CONF --> MAIN
    CONF --> MAIN2
    CONF --> MAINCOPY
    CONF --> CV_TITLE

    %% Entry Points trigger Processing modules
    MAIN --> CV_TITLE
    MAIN2 --> CV_TITLE
    MAINCOPY --> CV_TITLE

    %% Processing dependencies
    HELP -.-> CV_TITLE
    UTIL -.-> CV_TITLE
    MODEL --> CV_TITLE

    %% Processing outputs to Data Management
    CV_TITLE --> DATA

    %% Data flows to Visualization
    DATA --> VIS

    %% File module associations in Processing Layer (vertical grouping)
    CV_TITLE --- YOLO3D
    CV_TITLE --- YOLO8
    CV_TITLE --- LANE
    CV_TITLE --- OPTICAL
    CV_TITLE --- POSE

    %% File listings in Simulation Layer
    SIM_TITLE --- BLENDER_JSON
    SIM_TITLE --- FINAL_RENDER
    SIM_TITLE --- NEW_RENDERER
    SIM_TITLE --- PERSON_RENDERED
    SIM_TITLE --- TRAFFIC_SIGN

    %% Click Events for Simulation Layer
    click SIM_TITLE "https://github.com/udaygirish/einstein-vision/tree/main/blender/"
    click BLENDER_JSON "https://github.com/udaygirish/einstein-vision/blob/main/blender/blender_json_generator.py"
    click FINAL_RENDER "https://github.com/udaygirish/einstein-vision/blob/main/blender/final_render.py"
    click NEW_RENDERER "https://github.com/udaygirish/einstein-vision/blob/main/blender/new_renderer.py"
    click PERSON_RENDERED "https://github.com/udaygirish/einstein-vision/blob/main/blender/person_rendered.py"
    click TRAFFIC_SIGN "https://github.com/udaygirish/einstein-vision/blob/main/blender/traffic_sign_render.py"

    %% Click Events for Configuration Manager
    click CONF "https://github.com/udaygirish/einstein-vision/tree/main/config/"

    %% Click Events for Entry Points
    click MAIN "https://github.com/udaygirish/einstein-vision/blob/main/main.py"
    click MAIN2 "https://github.com/udaygirish/einstein-vision/blob/main/main2.py"
    click MAINCOPY "https://github.com/udaygirish/einstein-vision/blob/main/main copy.py"

    %% Click Events for Processing Layer
    click CV_TITLE "https://github.com/udaygirish/einstein-vision/tree/main/lib/"
    click YOLO3D "https://github.com/udaygirish/einstein-vision/blob/main/lib/yolov3d_infer.py"
    click YOLO8 "https://github.com/udaygirish/einstein-vision/blob/main/lib/yolov8_det.py"
    click LANE "https://github.com/udaygirish/einstein-vision/blob/main/lib/clrernet_lane_detect.py"
    click OPTICAL "https://github.com/udaygirish/einstein-vision/blob/main/lib/optical_flow.py"
    click POSE "https://github.com/udaygirish/einstein-vision/blob/main/lib/pose_2d.py"
    click HELP "https://github.com/udaygirish/einstein-vision/tree/main/helpers/"
    click UTIL "https://github.com/udaygirish/einstein-vision/tree/main/utilities/"
    click MODEL "https://github.com/udaygirish/einstein-vision/tree/main/models/"

    %% Click Event for Data & Output Layer
    click DATA "https://github.com/udaygirish/einstein-vision/tree/main/data/"
    click VIS "https://github.com/udaygirish/einstein-vision/tree/main/images/"

    %% Styles
    classDef simulation fill:#FFCCCB,stroke:#333,stroke-width:1px;
    classDef config fill:#ADD8E6,stroke:#333,stroke-width:1px;
    classDef entry fill:#90EE90,stroke:#333,stroke-width:1px;
    classDef processing fill:#FFFACD,stroke:#333,stroke-width:1px;
    classDef helper fill:#D3D3D3,stroke:#333,stroke-width:1px;
    classDef model fill:#E6E6FA,stroke:#333,stroke-width:1px;
    classDef data fill:#FFDAB9,stroke:#333,stroke-width:1px;
    classDef viz fill:#FFE4E1,stroke:#333,stroke-width:1px;
    classDef file fill:#D1EEEE,stroke:#333,stroke-width:1px;
```
