# frogai/frogpilot architecture

> Architecture diagram of the GitHub repository frogai/frogpilot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/frogai/frogpilot
- Repository: https://github.com/frogai/frogpilot

## Overview

##### Main Components
a. Hardware & Sensor Interfaces:
- Modules under system/camerad, system/sensord, and system/hardware provide low-level sensor data (e.g., camera streams, radar, and CAN data via opendbc and Panda).
- These components interface with the vehicle’s sensors and actuators.
b. Messaging & Serialization Layer:
- “cereal” and “msgq” (and their related repositories) are used to serialize and exchange messages across system modules.
c. Perception and Model Processing:
- Directories like selfdrive/modeld, selfdrive/classic_modeld, and selfdrive/tinygrad_modeld house neural network models and perception pipelines that process sensor input.
d. Planning & Control:
- The selfdrive/controls directory contains lateral and longitudinal planning algorithms (including modules for MPC, PID control, and various vehicle-specific control strategies).
- Additional processes (e.g., navd for navigation and locationd for localization) complement the planning stage.
e. Vehicle Integration & Car Modules:
- selfdrive/car and its various subfolders (e.g., for Honda, Toyota, GM, etc.) implement the vehicle-specific integration, including CAN communication, fingerprinting, and ECU interactions.
f. Driver Monitoring & UI:
- The selfdrive/ui and selfdrive/frogpilot directories provide the onroad and offroad displays, custom themes, alerts, and developer UIs.
- FrogPilot-specific enhancements (custom themes, toggles for “Experimental Mode,” additional notifications, etc.) are integrated here.
g. Device Management, Logging & System Services:
- Under system/manager, system/loggerd, system/logcatd, and system/updated you have modules for process management, logging, diagnostics, and device updates.
h. External Tools & Third-Party Libraries:
- The directories such as third_party (e.g., with acados, libyuv, and others) and tools contain deployment scripts, diagnostic utilities, and external dependencies.

##### Out Relationships and Data Flow
- From messaging, feed data into the perception layer (modeld, classic_modeld, etc.) where neural models process it.
- The processed sensor/vision data then flows into the planning and control modules (found in selfdrive/controls and related lateral/longitudinal modules) that compute driving commands.
- Control commands are dispatched back through vehicle-specific car modules for actuation.
- A parallel path handles driver monitoring and user interaction – sensor data plus custom UI signals from selfdrive/ui and selfdrive/frogpilot feed back into the system for dynamic adjustments.
- System services and management (system/manager, loggerd, etc.) continuously supervise the data flows and manage updates, logging, and diagnostics.
- Finally, the tools and third-party integrations provide auxiliary capabilities such as diagnostics, testing, and data visualization.

##### Architectural Patterns & Design Principles
- Layering – the system is best viewed as multiple layers: from hardware interfaces to middleware messaging, perception/planning/control logic, and finally, user–device interactions.
- Event-Driven Communication – data flows continuously via a messaging system (using cap’n proto–style serialization, typical in cereal) to loosely couple components.
- Extensibility – the design allows for additional car integrations and custom UI/themes (the FrogPilot modifications), so indicate extensibility points.

## Diagram (Mermaid)

```mermaid
graph TD
    %% Hardware & Sensor Interfaces
    subgraph "Hardware & Sensor Interfaces"
      h1("Camerad"):::hardware
      h2("Sensord"):::hardware
      h3("Hardware"):::hardware
      h4("Opendbc"):::hardware
      h5("Panda"):::hardware
    end

    %% Messaging & Serialization Layer
    subgraph "Messaging & Serialization Layer"
      m1("Cereal"):::messaging
      m2("MSGQ"):::messaging
    end

    %% Perception and Model Processing
    subgraph "Perception and Model Processing"
      p1("Modeld"):::perception
      p2("Classic Modeld"):::perception
      p3("Tinygrad Modeld"):::perception
    end

    %% Planning & Control
    subgraph "Planning & Control"
      pc1("Controls"):::planning
      pc2("Navd"):::planning
    end

    %% Vehicle Integration & Car Modules
    subgraph "Vehicle Integration & Car Modules"
      car("Car Integration"):::car
    end

    %% Driver Monitoring & UI
    subgraph "Driver Monitoring & UI"
      ui1("Driver UI"):::ui
      ui2("FrogPilot Enhancements"):::ui
    end

    %% Device Management, Logging & System Services
    subgraph "Device Management, Logging & System Services"
      dm1("Manager"):::management
      dm2("Logger"):::management
      dm3("Logcat"):::management
      dm4("Updater"):::management
    end

    %% External Tools & Third-Party Libraries
    subgraph "External Tools & Third-Party Libraries"
      tp1("Third Party Libraries"):::thirdParty
      tp2("Tools"):::thirdParty
    end

    %% Main Data Flow
    h1 -->|"data"| m1
    m1 -->|"serialized_data"| p1
    p1 -->|"perception_output"| pc1
    pc1 -->|"control_commands"| car

    %% Parallel and Feedback Flows
    m2 -->|"msgq_stream"| ui1
    ui1 -->|"user_feedback"| pc1
    ui2 -->|"custom_alerts"| pc1
    tp1 -->|"MPC_solvers"| pc1

    %% Device Management Monitoring
    m1 -- "log" --> dm1
    p1 -- "log" --> dm1
    pc1 -- "log" --> dm1

    %% Click Events for Hardware & Sensor Interfaces
    click h1 "https://github.com/frogai/frogpilot/tree/FrogPilot/system/camerad"
    click h2 "https://github.com/frogai/frogpilot/tree/FrogPilot/system/sensord"
    click h3 "https://github.com/frogai/frogpilot/tree/FrogPilot/system/hardware"
    click h4 "https://github.com/frogai/frogpilot/tree/FrogPilot/opendbc"
    click h5 "https://github.com/frogai/frogpilot/tree/FrogPilot/panda"

    %% Click Events for Messaging & Serialization Layer
    click m1 "https://github.com/frogai/frogpilot/tree/FrogPilot/cereal"
    click m2 "https://github.com/frogai/frogpilot/tree/FrogPilot/msgq_repo"

    %% Click Events for Perception and Model Processing
    click p1 "https://github.com/frogai/frogpilot/tree/FrogPilot/selfdrive/modeld"
    click p2 "https://github.com/frogai/frogpilot/tree/FrogPilot/selfdrive/classic_modeld"
    click p3 "https://github.com/frogai/frogpilot/tree/FrogPilot/selfdrive/tinygrad_modeld"

    %% Click Events for Planning & Control
    click pc1 "https://github.com/frogai/frogpilot/tree/FrogPilot/selfdrive/controls"
    click pc2 "https://github.com/frogai/frogpilot/tree/FrogPilot/selfdrive/navd"

    %% Click Events for Vehicle Integration & Car Modules
    click car "https://github.com/frogai/frogpilot/tree/FrogPilot/selfdrive/car"

    %% Click Events for Driver Monitoring & UI
    click ui1 "https://github.com/frogai/frogpilot/tree/FrogPilot/selfdrive/ui"
    click ui2 "https://github.com/frogai/frogpilot/tree/FrogPilot/selfdrive/frogpilot"

    %% Click Events for Device Management, Logging & System Services
    click dm1 "https://github.com/frogai/frogpilot/tree/FrogPilot/system/manager"
    click dm2 "https://github.com/frogai/frogpilot/tree/FrogPilot/system/loggerd"
    click dm3 "https://github.com/frogai/frogpilot/tree/FrogPilot/system/logcatd"
    click dm4 "https://github.com/frogai/frogpilot/tree/FrogPilot/system/updated"

    %% Click Events for External Tools & Third-Party Libraries
    click tp1 "https://github.com/frogai/frogpilot/tree/FrogPilot/third_party"
    click tp2 "https://github.com/frogai/frogpilot/tree/FrogPilot/tools"

    %% Styles
    classDef hardware fill:#f9e79f,stroke:#d35400,stroke-width:2px;
    classDef messaging fill:#aed6f1,stroke:#2471a3,stroke-width:2px;
    classDef perception fill:#d5f5e3,stroke:#27ae60,stroke-width:2px;
    classDef planning fill:#f9e79f,stroke:#d35400,stroke-width:2px;
    classDef car fill:#f5b7b1,stroke:#c0392b,stroke-width:2px;
    classDef ui fill:#fadbd8,stroke:#e74c3c,stroke-width:2px;
    classDef management fill:#d6eaf8,stroke:#3498db,stroke-width:2px;
    classDef thirdParty fill:#fdebd0,stroke:#e67e22,stroke-width:2px;
```
