# vnktadithya/rca-lora-scaling-study architecture

> Architecture diagram of the GitHub repository vnktadithya/rca-lora-scaling-study, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vnktadithya/rca-lora-scaling-study
- Repository: https://github.com/vnktadithya/rca-lora-scaling-study

## Overview

This repository is a controlled Mistral-7B LoRA scaling study for structured root-cause analysis. Its main workflow collects and filters GitHub issue/PR examples, builds train/evaluation datasets, trains two adapters, generates predictions, and scores them across format, semantic-similarity, and LLM-judge tiers before comparing results. The README documents these capabilities; sampled source confirms key collection, training, serving, and judge calls. Evaluation scripts not sampled are represented by their exact tree paths, with only documented relationships shown.

Main entry points: [`Data_Collection/generate_dataset.py`](https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Data_Collection/generate_dataset.py) (Issue pipeline), [`Model_Training/train_ft20.py`](https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Model_Training/train_ft20.py) (FT-20 trainer), [`Model_Training/train_ft500.py`](https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Model_Training/train_ft500.py) (FT-500 trainer), [`logs`](https://github.com/vnktadithya/rca-lora-scaling-study/tree/main/logs) (Model predictions)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_collection["Data collection"]
  node_dataset_pipeline["Issue pipeline"]
  node_trace_filter["Traceback filter"]
  node_dedup["Deduplication<br/>[dedup.py]"]
  node_pipeline_state[("Processed ledger<br/>[pipeline_state.py]")]
  node_synthetic_data["Synthetic examples<br/>[LLM_Generator.py]"]
  node_dataset_writer["JSONL writer<br/>[add_to_jsonl.py]"]
  node_splitter["Train/eval split"]
  node_datasets["Train/eval datasets"]
end

subgraph group_training["Model training"]
  node_train20["FT-20 trainer<br/>[train_ft20.py]"]
  node_train500["FT-500 trainer<br/>[train_ft500.py]"]
  node_formatting["Prompt formatter<br/>[data_formatting.py]"]
  node_tokenizer["Input tokenizer<br/>[input_tokenizer.py]"]
  node_checkpoints["LoRA adapters"]
end

subgraph group_serving["Inference"]
  node_inference["Batched generation"]
  node_outputs["Model predictions"]
end

subgraph group_evaluation["Evaluation"]
  node_validity["Tier 1 validity"]
  node_similarity["Tier 2 similarity"]
  node_judge_runner["Tier 3 judge"]
  node_judge_client["Judge API client<br/>[LLM_judge.py]"]
  node_comparison["Metric comparison"]
  node_tier3_scores["Judge scores"]
  node_plots["Benchmark plots"]
end

node_researcher(("Study operator"))
node_github["GitHub issues/PRs"]
node_gemini_generate["Gemini generator"]
node_base_model["Mistral base model"]
node_judge["Gemini judge"]

node_researcher -->|"starts collection"| node_dataset_pipeline
node_dataset_pipeline -->|"searches issues"| node_github
node_github -->|"returns issue data"| node_dataset_pipeline
node_dataset_pipeline -->|"checks tracebacks"| node_trace_filter
node_dataset_pipeline -->|"checks duplicates"| node_dedup
node_dataset_pipeline -->|"reads/writes ledger"| node_pipeline_state
node_dataset_pipeline -->|"requests examples"| node_synthetic_data
node_synthetic_data -->|"generates RCA"| node_gemini_generate
node_dataset_pipeline -->|"assigns split"| node_splitter
node_dataset_pipeline -->|"appends accepted data"| node_dataset_writer
node_dataset_writer -->|"writes JSONL"| node_datasets
node_datasets -->|"supplies FT-20 data"| node_train20
node_datasets -->|"supplies full data"| node_train500
node_train20 -->|"formats prompts"| node_formatting
node_train500 -->|"formats prompts"| node_formatting
node_train20 -->|"tokenizes examples"| node_tokenizer
node_train500 -->|"tokenizes examples"| node_tokenizer
node_train20 -->|"fine-tunes"| node_base_model
node_train500 -->|"fine-tunes"| node_base_model
node_train20 -->|"saves adapter"| node_checkpoints
node_train500 -->|"saves adapter"| node_checkpoints
node_datasets -->|"provides eval prompts"| node_inference
node_inference -->|"generates base output"| node_base_model
node_inference -->|"loads adapters"| node_checkpoints
node_inference -->|"writes predictions"| node_outputs
node_outputs -->|"scores format"| node_validity
node_outputs -->|"scores semantics"| node_similarity
node_outputs -->|"supplies predictions"| node_judge_runner
node_judge_runner -->|"evaluates examples"| node_judge_client
node_judge_client -->|"requests scoring"| node_judge
node_judge_runner -->|"writes scores"| node_tier3_scores
node_outputs -->|"provides metrics input"| node_comparison
node_tier3_scores -->|"provides judge scores"| node_comparison
node_comparison -->|"computes Tier 1"| node_validity
node_comparison -->|"computes Tier 2"| node_similarity
node_comparison -->|"writes chart"| node_plots

click node_dataset_pipeline "https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Data_Collection/generate_dataset.py"
click node_trace_filter "https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Data_Collection/check_error_logs.py"
click node_dedup "https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Data_Collection/dedup.py"
click node_pipeline_state "https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Data_Collection/pipeline_state.py"
click node_synthetic_data "https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Data_Collection/LLM_Generator.py"
click node_dataset_writer "https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Data_Collection/add_to_jsonl.py"
click node_splitter "https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Data_Collection/train_test_split.py"
click node_datasets "https://github.com/vnktadithya/rca-lora-scaling-study/tree/main/Data"
click node_train20 "https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Model_Training/train_ft20.py"
click node_train500 "https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Model_Training/train_ft500.py"
click node_formatting "https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Model_Training/data_formatting.py"
click node_tokenizer "https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Model_Training/input_tokenizer.py"
click node_checkpoints "https://github.com/vnktadithya/rca-lora-scaling-study/tree/main/Checkpoints"
click node_inference "https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Model_Evaluation/output_generation.py"
click node_outputs "https://github.com/vnktadithya/rca-lora-scaling-study/tree/main/logs"
click node_validity "https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Model_Evaluation/validity_evaluation.py"
click node_similarity "https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Model_Evaluation/similarity_evaluation.py"
click node_judge_runner "https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Model_Evaluation/LLM_judge_evaluation.py"
click node_judge_client "https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Model_Evaluation/LLM_judge.py"
click node_comparison "https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Model_Evaluation/performance_comparision.py"
click node_tier3_scores "https://github.com/vnktadithya/rca-lora-scaling-study/tree/main/logs"
click node_plots "https://github.com/vnktadithya/rca-lora-scaling-study/tree/main/plots"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dataset_pipeline,node_trace_filter,node_dedup,node_pipeline_state,node_synthetic_data,node_dataset_writer,node_splitter,node_datasets toneBlue
class node_train20,node_train500,node_formatting,node_tokenizer,node_checkpoints toneAmber
class node_inference,node_outputs toneMint
class node_validity,node_similarity,node_judge_runner,node_judge_client,node_comparison,node_tier3_scores,node_plots toneRose
class node_researcher,node_github,node_gemini_generate,node_base_model,node_judge toneIndigo
```

## Components

### Data collection

- [Issue pipeline](https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Data_Collection/generate_dataset.py): `Data_Collection/generate_dataset.py`
- [Traceback filter](https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Data_Collection/check_error_logs.py): `Data_Collection/check_error_logs.py`
- [Deduplication](https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Data_Collection/dedup.py): `Data_Collection/dedup.py`
- [Processed ledger](https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Data_Collection/pipeline_state.py): `Data_Collection/pipeline_state.py`
- [Synthetic examples](https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Data_Collection/LLM_Generator.py): `Data_Collection/LLM_Generator.py`
- [JSONL writer](https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Data_Collection/add_to_jsonl.py): `Data_Collection/add_to_jsonl.py`
- [Train/eval split](https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Data_Collection/train_test_split.py): `Data_Collection/train_test_split.py`
- [Train/eval datasets](https://github.com/vnktadithya/rca-lora-scaling-study/tree/main/Data): `Data`

### Model training

- [FT-20 trainer](https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Model_Training/train_ft20.py): `Model_Training/train_ft20.py`
- [FT-500 trainer](https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Model_Training/train_ft500.py): `Model_Training/train_ft500.py`
- [Prompt formatter](https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Model_Training/data_formatting.py): `Model_Training/data_formatting.py`
- [Input tokenizer](https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Model_Training/input_tokenizer.py): `Model_Training/input_tokenizer.py`
- [LoRA adapters](https://github.com/vnktadithya/rca-lora-scaling-study/tree/main/Checkpoints): `Checkpoints`

### Inference

- [Batched generation](https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Model_Evaluation/output_generation.py): `Model_Evaluation/output_generation.py`
- [Model predictions](https://github.com/vnktadithya/rca-lora-scaling-study/tree/main/logs): `logs`

### Evaluation

- [Tier 1 validity](https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Model_Evaluation/validity_evaluation.py): `Model_Evaluation/validity_evaluation.py`
- [Tier 2 similarity](https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Model_Evaluation/similarity_evaluation.py): `Model_Evaluation/similarity_evaluation.py`
- [Tier 3 judge](https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Model_Evaluation/LLM_judge_evaluation.py): `Model_Evaluation/LLM_judge_evaluation.py`
- [Judge API client](https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Model_Evaluation/LLM_judge.py): `Model_Evaluation/LLM_judge.py`
- [Metric comparison](https://github.com/vnktadithya/rca-lora-scaling-study/blob/main/Model_Evaluation/performance_comparision.py): `Model_Evaluation/performance_comparision.py`
- [Judge scores](https://github.com/vnktadithya/rca-lora-scaling-study/tree/main/logs): `logs`
- [Benchmark plots](https://github.com/vnktadithya/rca-lora-scaling-study/tree/main/plots): `plots`

### Other components

- **Study operator**
- **GitHub issues/PRs**
- **Gemini generator**
- **Mistral base model**
- **Gemini judge**

## Connections

- Study operator → Issue pipeline: starts collection
- Issue pipeline → GitHub issues/PRs: searches issues
- GitHub issues/PRs → Issue pipeline: returns issue data
- Issue pipeline → Traceback filter: checks tracebacks
- Issue pipeline → Deduplication: checks duplicates
- Issue pipeline → Processed ledger: reads/writes ledger
- Issue pipeline → Synthetic examples: requests examples
- Synthetic examples → Gemini generator: generates RCA
- Issue pipeline → Train/eval split: assigns split
- Issue pipeline → JSONL writer: appends accepted data
- JSONL writer → Train/eval datasets: writes JSONL
- Train/eval datasets → FT-20 trainer: supplies FT-20 data
- Train/eval datasets → FT-500 trainer: supplies full data
- FT-20 trainer → Prompt formatter: formats prompts
- FT-500 trainer → Prompt formatter: formats prompts
- FT-20 trainer → Input tokenizer: tokenizes examples
- FT-500 trainer → Input tokenizer: tokenizes examples
- FT-20 trainer → Mistral base model: fine-tunes
- FT-500 trainer → Mistral base model: fine-tunes
- FT-20 trainer → LoRA adapters: saves adapter
- FT-500 trainer → LoRA adapters: saves adapter
- Train/eval datasets → Batched generation: provides eval prompts
- Batched generation → Mistral base model: generates base output
- Batched generation → LoRA adapters: loads adapters
- Batched generation → Model predictions: writes predictions
- Model predictions → Tier 1 validity: scores format
- Model predictions → Tier 2 similarity: scores semantics
- Model predictions → Tier 3 judge: supplies predictions
- Tier 3 judge → Judge API client: evaluates examples
- Judge API client → Gemini judge: requests scoring
- Tier 3 judge → Judge scores: writes scores
- Model predictions → Metric comparison: provides metrics input
- Judge scores → Metric comparison: provides judge scores
- Metric comparison → Tier 1 validity: computes Tier 1
- Metric comparison → Tier 2 similarity: computes Tier 2
- Metric comparison → Benchmark plots: writes chart