# idealab-isu/slice-100k architecture

> Architecture diagram of the GitHub repository idealab-isu/slice-100k, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/idealab-isu/slice-100k
- Repository: https://github.com/idealab-isu/slice-100k

## Overview

##### Main Components
- Dataset & Data Sources: Represent the raw inputs such as G-code files, CAD models, and LVIS category definitions (evident from the LVIS_categories_final.txt in Category_Generation).
- Category Generation Module: (Directory: Category_Generation) – Processes LVIS data and potentially assigns or organizes category labels.
- Slicing Module: (Directory: slicing) – Uses configuration files (INI files and llm.yml) and scripts (e.g., slice_binary_gcode.py) to "slice" the G-code files. This likely transforms or segments the G-code into meaningful parts.
- Blender Rendering Module: (Directory: blender_rendering) – Leverages Blender scripts (blender_script.py, custom_main.py) and configuration (blender.yml) to generate STL renderings. It even includes a SLURM batch file for scheduling on clusters.
- G-code Rendering Module: (Directory: gcode_rendering) – Contains scripts (gcode_render.py) to produce visualizations or alternative renderings specifically from G-code.
- Translation & Experimentation Module: (Directory: translation_experiments) – Contains submodules:
- gcode_preprocessing: Applies transformations like chunking, contour flipping, and various preprocessing steps.
- Model Evaluation: (model_eval) – Scripts for evaluating generated data or models (e.g., eval_model.py).
- Model Training: (model_training) – Implements training strategies (e.g., sft.py).

##### Relationships and Data Flow
- The Slicing Module takes raw G-code files as input and produces sliced segments.
- The Blender Rendering Module uses 3D models or STL files (perhaps generated from sliced data) to produce visual renderings.
- The G-code Rendering Module may take the same or processed G-code files to produce an alternate representation.
- The Category Generation module appears to specialize in mapping or assigning LVIS categories to the dataset.
- Translation experiments work on the processed G-code data, indicating a loop that further enhances and analyzes the outputs with custom preprocessing, evaluation, and training routines.
- These modules ultimately contribute to assembling the final multimodal dataset described in the README (Slice-100K).

##### Architectural Patterns and Technologies
- The architecture is modular and loosely coupled—each module performs a distinct part of the data preparation and processing pipeline.
- The use of configuration files (INI, YAML) indicates a design that supports parameterization and possibly different processing scenarios.
- There are clear batch processing/automation aspects (e.g., SLURM batch file, Blender YAML) which suggest the solution may be deployed at scale.
- Each module may be developed and run independently but eventually integrates to create the unified dataset.

##### Detailed Elements
- Component block includes its primary files or configuration details where relevant (for instance, mention that the Blender Rendering block uses blender.yml and the execution of blender_script.py and custom_main.py).
- That the Translation Experiments contains multiple submodules (gcode preprocessing, model evaluation, and model training) to emphasize its experimental and iterative role.
- Including external integration points such as the dataset explorer and access link provided in the README, which may represent how the final dataset is exposed to users.

##### Final Advice
- Aim to capture every key architectural element by breaking down the project into as many logical components as possible without oversimplifying.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Sources
    DS["Data Sources"]:::database
    RG["Raw G-code Files"]:::database
    CAD["CAD Models"]:::database
    LV["LVIS Category Definitions"]:::database

    DS --> RG
    DS --> CAD
    DS --> LV

    %% Processing Modules
    CG["Category Generation Module"]:::module
    SLIC["Slicing Module"]:::module
    BR["Blender Rendering Module"]:::module
    GR["G-code Rendering Module"]:::module

    %% Translation Experiments Subgraph
    subgraph "Translation Experiments"
        GP["G-code Preprocessing Submodule"]:::module
        ME["Model Evaluation Submodule"]:::module
        MT["Model Training Submodule"]:::module
        GP -->|"PreprocessedG-code"| ME
        GP -->|"PreprocessedG-code"| MT
    end

    %% Data Flow Connections
    RG -->|"G-code"| SLIC
    LV -->|"CategoryMapping"| CG

    SLIC -->|"SlicedG-code"| GR
    SLIC -->|"SegmentedData"| BR
    SLIC -->|"ProcessedG-code"| GP

    BR -->|"3DSTLRenderings"| FD["Final Multimodal Dataset"]:::final
    GR -->|"RenderedVisuals"| FD
    ME -->|"EvaluationResults"| FD
    MT -->|"TrainedModels"| FD

    FD -->|"ExposedViaDatasetExplorer"| EX["Dataset Explorer"]:::external

    %% Click Events for Component Mapping
    click CG "https://github.com/idealab-isu/slice-100k/tree/main/Category_Generation"
    click SLIC "https://github.com/idealab-isu/slice-100k/tree/main/slicing"
    click BR "https://github.com/idealab-isu/slice-100k/tree/main/blender_rendering"
    click GR "https://github.com/idealab-isu/slice-100k/tree/main/gcode_rendering"
    click GP "https://github.com/idealab-isu/slice-100k/tree/main/translation_experiments/gcode_preprocessing"
    click ME "https://github.com/idealab-isu/slice-100k/tree/main/translation_experiments/model_eval"
    click MT "https://github.com/idealab-isu/slice-100k/tree/main/translation_experiments/model_training"

    %% Styles
    classDef module fill:#bbdefb,stroke:#0d47a1,stroke-width:2px;
    classDef database fill:#c8e6c9,stroke:#1b5e20,stroke-width:2px;
    classDef external fill:#fff9c4,stroke:#f9a825,stroke-width:2px;
    classDef final fill:#ffcdd2,stroke:#c62828,stroke-width:2px;
```
