# sufe-aiflm-lab/fin-r1 architecture

> Architecture diagram of the GitHub repository sufe-aiflm-lab/fin-r1, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sufe-aiflm-lab/fin-r1
- Repository: https://github.com/sufe-aiflm-lab/fin-r1

## Overview

##### Project Type and Purpose
- Project Type: A research/product release for a specialized financial reasoning LLM.
- Purpose: Leverage high-quality financial data and dual-stage training (supervised fine-tuning and reinforcement learning using GRPO) to enable robust financial decision-making and analysis in banking, securities, and other financial domains.

##### File Structure
- Configuration or code: Although there isn’t a rich codebase (the repository seems to be documentation-heavy), the technical details provided in the README serve as a blueprint for the data pipeline, training phases, and model serving instructions.

a. Main Components to Include:
- Data Construction & Distillation Module: This includes ingesting multiple diverse financial datasets (FinCorpus, FinQA, ConvFinQA, etc.) and processing them using a two-stage “答案+推理” (Answer + Reasoning) quality scoring mechanism.
- Data Filtering & Labeling: Separation of data into “good” (for SFT) and “bad” (for RL) based on rule-based matching and model scores.
- Reinforcement Learning (RL) Module: Using GRPO (Group Relative Policy Optimization), double-reward mechanisms (format and accuracy rewards), and model-based verification (via Qwen2.5-Max) to further refine the model.
- Evaluation & Benchmarking: Components showing how performance across different datasets (FinQA, ConvFinQA, etc.) is measured and compared.
- Deployment/Inference Service: Dedicated serving layer using vllm to load model weights and provide an API endpoint (as illustrated by the provided command-line and Python usage).

b. Relationships and Data Flows:
- Data flows into a “Data Distillation & Filtering” stage where quality gating is applied using two scoring rounds.
- From here, a bifurcation: one stream goes to SFT training, and the other (lower-quality or variant data) is used for RL training.
- The training modules are sequentially placed: First SFT to inject reasoning ability, then RL optimization to further enhance performance.
- Finally, the refined model is evaluated (comparing scores on benchmarks) and eventually deployed as a service that feeds user requests via API.

c. Architectural Patterns and Design Principles:
- Two-phase training pipeline (SFT then RL) is a critical architectural pattern.
- Emphasis on quality control via dual-stage data filtering.
- The “answer+reasoning” scheme highlights the importance of intermediate representations (chain-of-thought).
- The modular design allows a clear separation between data preprocessing, training/transformation, and deployment/inference.

d. Technologies, Frameworks, and Libraries:
- Training Algorithms: Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO) for RL.
- Verification Mechanism: Qwen2.5-Max used in the model-based verifier.
- Deployment: vllm for serving the model with API endpoints.

- Clear Labels: Label every stage or component clearly (e.g., “Data Distillation Module”, “SFT Training”, “RL Optimization (GRPO)”, “Model-Based Verifier”, “Inference Server”).
- Annotations: Include brief text annotations indicating key processes such as “Dual-Stage Quality Scoring” in the data filtering module and “Double Reward Mechanism” in the RL training module.
- Integration Points: Clearly denote external integration points such as model download via HuggingFace and the serving layer command (vllm serve).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Ingestion
    DS["Financial Datasets<br/>(FinCorpus, FinQA, ConvFinQA, etc.)"]:::dataset

    %% Data Processing Pipeline
    DCDM["Data Construction & Distillation Module<br/>(Dual-Stage Quality Scoring)"]:::module
    DFL["Data Filtering & Labeling<br/>(Good for SFT, Variant for RL)"]:::module

    %% Training Pipeline
    SFT["Supervised Fine-Tuning (SFT) Module"]:::module
    RL["Reinforcement Learning (RL) Module<br/>(DoubleRewardMechanism)"]:::module

    %% Post-Training & Deployment
    Eval["Evaluation & Benchmarking<br/>(Performance Metrics)"]:::module
    Deploy["Deployment/Inference Service<br/>(vllm API Serve)"]:::module

    %% Data Flow Connections
    DS -->|"ingests"| DCDM
    DCDM -->|"processes"| DFL
    DFL -->|"routesToSFT"| SFT
    DFL -->|"routesToRL"| RL
    SFT -->|"mergesResults"| Eval
    RL -->|"mergesResults"| Eval
    Eval -->|"deployedVia"| Deploy

    %% Click Events for Component Mapping
    click DCDM "https://github.com/sufe-aiflm-lab/fin-r1/blob/main/Technical_report.pdf"
    click DFL "https://github.com/sufe-aiflm-lab/fin-r1/blob/main/Technical_report.pdf"
    click SFT "https://github.com/sufe-aiflm-lab/fin-r1/blob/main/README.md"
    click RL "https://github.com/sufe-aiflm-lab/fin-r1/blob/main/README.md"
    click Eval "https://github.com/sufe-aiflm-lab/fin-r1/blob/main/Technical_report.pdf"
    click Deploy "https://github.com/sufe-aiflm-lab/fin-r1/tree/main/Images/README"

    %% Styles
    classDef dataset fill:#FFD700,stroke:#DAA520,stroke-width:2px;
    classDef module fill:#ADD8E6,stroke:#4682B4,stroke-width:2px;
    
    class DS dataset;
    class DCDM,DFL,SFT,RL,Eval,Deploy module;
```
