# gvariable/chataffairs architecture

> Architecture diagram of the GitHub repository gvariable/chataffairs, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gvariable/chataffairs
- Repository: https://github.com/gvariable/chataffairs

## Overview

##### Project Type and Purpose
- The project is a multi-module full-stack application aimed at enhancing government services, especially targeting elderly users. Its main functions include interactive consultation using an LLM (ChatAffair+), data collection via web crawling, model training, and audio cloning for customized voice output. The project therefore integrates components for data ingestion, model training, back-end services, and a front-end user interface.

##### Analyzing the File Structure
- Key directories include:
- ac – Contains the audio cloning module (e.g., README, scripts specific to audio cloning).
- backend – Includes API and deployment scripts (Dockerfile, run.sh), suggesting a server module that handles requests/logic.
- crawler – Contains web scraping scripts (crawler.py, parse.py) and demo files to gather administrative process data.
- dataset – Holds the collected JSON/Labeled data files, HTML meta information, and training summaries.
- frontend – Built with Vue (see App.vue, components, pages, manifest.json) to offer an interactive client experience.
- train – Encompasses model training scripts, configuration files (args.yaml), and source code for training the ChatAffair+ model.

a. Main Components:
- Data Ingestion & Collection:
- Crawler Module: Responsible for scraping administrative documents and metadata from government sources.
- Dataset: Acts as the data repository/store for all collected data.
- Model Training:
- Train Module: Takes data from "dataset" to train the ChatAffair+ LLM model along with handling tasks such as preprocessing.
- Audio Processing:
- AC Module: Implements audio cloning and voice customization functionalities.
- Back-end Services:
- Backend Module: Provides core API endpoints to integrate the trained model, audio processing, and data management, likely containerized (see Dockerfile).
- Front-end Services:
- Frontend Module: A Vue-based application that serves as the UI, enabling interactive consultations and access to the services.

b. Relationships and Data Flow:
- The Crawler gathers government process data and stores it in the dataset.
- The Train module consumes this dataset to refine and train the ChatAffair+ model.
- The Backend uses the trained model to provide interactive consultation services – it may interface with the Train module (or its outputs) as a service.
- The AC module is used by the Backend to provide customized voice responses (text-to-speech and speech-to-text integration), which tie into the audio cloning functions.
- The Frontend communicates with the Backend via API calls to provide a seamless user interface, handling user input and displaying consultation results.

c. Architectural Patterns & Design Principles:
- Separation of concerns: Each major functionality (data collection, training, audio cloning, API services, UI) is encapsulated in its own module.
- Full-stack design with clearly separated client (frontend) and server (backend) layers.
- Micro-modular or microservices aspects, where modules like crawler, train, ac, and backend can potentially operate as independent services.
- Containerization for deployment (see backend/Dockerfile), which helps to standardize the environment across components.

d. Relevant Technologies/Frameworks/Libraries:
- Frontend: Vue.js (App.vue, pages, components) along with supporting assets (CSS, libs like html2json.js, htmlparser.js).
- Backend: Python scripts (utils_api.py) along with Docker and shell scripts for deployment.
- Training: Python-based training scripts (train/src/train.py) using configuration files like args.yaml and dp_zero2.json.
- Audio Cloning (AC): Specialized algorithms for text-to-speech and speech-to-text, and external integration possibly inspired by projects like RVC.
- Data Crawling: Python scripts dedicated to web scraping (crawler.py, parse.py).

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Presentation Layer"
        FE["Frontend Module (Vue.js)"]:::frontend
    end

    subgraph "Application Layer"
        BE["Backend Module (API, Dockerized)"]:::backend
        AC["Audio Cloning (AC Module)"]:::audio
    end

    subgraph "Processing Layer"
        TR["Train Module (LLM Training)"]:::train
    end

    subgraph "Data Layer"
        DS["Dataset Repository (Data Storage)"]:::data
        CR["Crawler Module (Data Ingestion)"]:::data
    end

    %% Arrows indicating relationships and data flow
    FE -->|"API calls"| BE
    TR -->|"provides model"| BE
    DS -->|"used for training"| TR
    CR -->|"sends data"| DS
    BE -->|"calls audio clone"| AC

    %% Click Events
    click CR "https://github.com/gvariable/chataffairs/tree/master/crawler/"
    click DS "https://github.com/gvariable/chataffairs/tree/master/dataset/"
    click TR "https://github.com/gvariable/chataffairs/tree/master/train/"
    click AC "https://github.com/gvariable/chataffairs/tree/master/ac/"
    click BE "https://github.com/gvariable/chataffairs/tree/master/backend/"
    click FE "https://github.com/gvariable/chataffairs/tree/master/frontend/"

    %% Styles
    classDef frontend fill:#90CAF9,stroke:#0D47A1,stroke-width:2px;
    classDef backend fill:#A5D6A7,stroke:#1B5E20,stroke-width:2px;
    classDef train fill:#B3E5FC,stroke:#01579B,stroke-width:2px;
    classDef data fill:#FFE082,stroke:#FF6F00,stroke-width:2px;
    classDef audio fill:#FFAB91,stroke:#BF360C,stroke-width:2px;
```
