How danhdoan/intel-gaudi-lab works
Main Components
• Docker/Container Environment: Represented by the Docker-related files (compose.yml, example.env, Makefile targets like “up”, “connect”, “down”). This component handles the development environment setup (especially for the habana_frameworks service).
• Application Entry Point: The “apps” directory (especially apps/app.py) likely contains the main application logic that gets executed.
• Experimentals: The “experimentals” directory contains scripts (e.g., inference_stable_diffusion.py, stable_diffusion_2_1_app.py, run.sh) which seem to be used for testing new inference methods, likely integrating with Intel Gaudi.
• Libraries (libs): This directory is key as it aggregates reusable functionality:
• CLI Parsing (libs/cli/cli_parser.py) — command-line interface for user interactions.
• Configurations (libs/configs/configs.py) — for managing environment and application configuration.
• Utilities (libs/utils), including logging, time, and averaging functionalities, which play supporting roles throughout the codebase.
• Public Assets: The “public” folder stores static assets (index.html, script.js, styles.css) that may serve as the front end or dashboard for interacting with the AI experiments.
• Setup and Dependency Management: Files such as requirements.txt, setup.py, pyproject.toml indicate Python packaging and dependency resolution, which are important for both development and deployment.
Relationships and Interactions
• Experimentals as Extensions: The experimentals directory, which includes scripts for running stable diffusion experiments, interacts with both the core application and the underlying libraries. It might either run as separate processes or be triggered via the CLI.
• Public Frontend: If the front end (in public) is involved (perhaps for visualization or dashboards), show a connection between the frontend and the application backend.
• Dependency/Configuration Flow: Include how Makefile and scripts (like scripts/source_env.sh) manage configuration and prepare the development environment.
Architectural Patterns & Design Principles
• Modular Design: The clear separation into apps, libs, experimentals, and public directories suggests a modular architecture. Each module has well-defined roles.
• Containerization: The use of Docker (compose.yml, example.env, and Makefile commands) emphasizes container-based deployment and isolation.
• Extensibility: The structure in “libs” to handle CLI, configuration management, and utilities suggests that the codebase was designed for reusability and ease of extension for various experiments.
Elements
• Use clear labels for each component with names (e.g., “Docker Environment”, “Application Core (apps/app.py)”, “CLI & Config Utils”, “Experiment Scripts”, “Static Frontend”).
• Deployment configuration details such as the Docker container layer if useful, and annotate infrastructure concerns (like the Makefile and environment setup via scripts).
Final Detailed Outline
• Top Layer: Deployment & Environment Management
• Docker Container (with compose.yml & Makefile commands)
• Virtual Environment Management (scripts/source_env.sh, requirements.txt, pyproject.toml)
• Mid Layer: Application Core
• Main Application (apps/app.py)
• Libraries (libs) divided into:
• CLI (libs/cli)
• Config Managers (libs/configs)
• Utility Modules (libs/utils, logger_utils, time_utils, averager)
• Side/Extension Layer: Experiment Modules
• Experiment Scripts (experimentals folder)
• Connections (integration with the main app or libraries)
• Frontend Interface (Public Assets)
• Static Web Assets (index.html, script.js, styles.css)
• Interaction with the application if applicable (e.g., for visualization).
This approach will provide a comprehensive, accurate visual representation of the project's architecture tailored to its purpose as a platform for AI experiments with Intel Gaudi.