jimdawdy-hub/local-llm-launcher-gui

1. What this repo is

- A local-first desktop-style web app for managing LLM models on the user’s own machine.

- The architecture is split into a Python backend that does all system/model/launch work and a React frontend that presents the control panel UI.

- The app is intended to run entirely on 127.0.0.1 and serve its own built frontend assets.

2. Top-level runtime shape

- The Python package in src/local_llm_launcher is the core runtime.

- The frontend in frontend/ is a separate Vite/React app whose production build is embedded into src/local_llm_launcher/static/.

- The backend appears to be the long-lived orchestrator: it discovers hardware, inspects installed engines, manages downloads, launches model servers, and tracks running state.

3. Backend entrypoints and serving boundary

- __main__.py and app.py are the startup path for the package.

- api.py is the HTTP boundary between frontend and backend.

- static/ contains the compiled frontend bundle, so the Python process likely serves both the API and the UI from one local process.

- This makes the backend the single source of truth for state; the frontend is mostly a client rendering backend-derived status.

4. Frontend structure

- The frontend is organized by views: Dashboard, Models, Launch, Servers, and Settings.

- api.js centralizes browser-to-backend calls.

- App.jsx and main.jsx wire the SPA shell, while components.jsx and theme.css provide shared UI building blocks and styling.

- The UI is operational rather than generic: each page maps to a backend subsystem like discovery, model selection, launch configuration, or active server control.

5. Hardware and environment discovery

- hardware.py and discovery.py detect the local machine’s capabilities and installed tooling.

- This includes GPU/VRAM or RAM availability, plus whether supported engines are present.

- The results feed directly into the “Fit / Tight / Won’t fit” recommendations and the dashboard health summary.

- This is a critical boundary: the app is not guessing from static model metadata alone; it evaluates against the user’s current machine state.

6. Model catalog and fit/advice logic

- catalog.py holds model/engine metadata and likely the flag/preset definitions used to configure launches.

- advisor.py performs the core decision logic: it compares a model’s memory needs with available hardware and emits the green/yellow/red verdicts plus plain-English reasoning.

- data/flags_vllm.json, data/flags_llamacpp.json, and data/failures.json are static knowledge bases used to keep those recommendations consistent.

- failures.py likely maps launch errors into human-readable diagnoses using those failure definitions.

7. Downloads and model storage

- downloads.py handles fetching models, most likely from Hugging Face.

- The Models page is therefore both a catalog browser and a local asset manager: it can inspect what is already present and trigger new downloads.

- The download subsystem sits between the frontend and the filesystem/network, with the backend keeping track of local model locations and availability.

8. Launch engine subsystem

- The engines/ package abstracts the different ways to run a model server.

- base.py defines the common engine contract; _args.py likely normalizes command-line construction.

- llamacpp.py, vllm_native.py, and vllm_docker.py are concrete launch paths for the supported runtimes.

- The key architectural point is that the rest of the app does not launch models directly; it asks a selected engine implementation to build and run the correct server process.

9. Server lifecycle and running-state tracking

- registry.py appears to track active servers, their ports, model assignments, and status.

- The Servers page in the UI is likely backed by this registry so it can show logs, addresses, and current process state.

- This registry is the coordination point between launch requests, process lifetime, and UI status display.

10. Configuration and user-specific environment

- config.py manages user settings such as Hugging Face tokens, extra model directories, and executable paths like llama-server.

- These settings matter because they affect model access, discovery success, and which engines can be launched.

- The Settings page is therefore not just preferences; it configures the backend’s ability to find and run local resources.

11. Open WebUI integration

- openwebui.py provides a secondary integration path: launching Open WebUI and connecting it to the running model server.

- This is a separate external app, but it is treated as part of the local workflow and surfaced from the dashboard.

- Architecturally, it is an optional companion process started by the backend rather than a core subsystem.

12. Data flow across the system

- Frontend actions call the backend API.

- The backend consults discovery/hardware state, model catalog data, and config.

- Advisor logic produces launch suitability and recommendations.

- If the user proceeds, an engine implementation starts a local model server.

- The registry records the live server, and the UI reads that state back for logs, connection info, and troubleshooting.

13. Testing and reliability boundaries

- The tests/ suite covers advisor logic, API behavior, catalog/discovery, hardware, engines, Open WebUI integration, and registry behavior.

- That test structure mirrors the core architecture: separate verification for system inspection, planning, launching, and runtime management.

- For an architecture diagram, this repo should be drawn as a local orchestrator + UI + engine adapters system, not as a monolithic chat app.

14. Implementation stack that materially affects architecture

- Backend: Python 3.10+ package layout under src/.

- Frontend: React + Vite, bundled into static assets served by the Python app.

- Model serving backends: vLLM and llama.cpp, with an optional Docker-based vLLM path.

- External dependency surfaces: local GPUs/CPU/RAM, Hugging Face model sources, and optionally Open WebUI.