# krt3xp/teste2projeto architecture

> Architecture diagram of the GitHub repository krt3xp/teste2projeto, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/krt3xp/teste2projeto
- Repository: https://github.com/krt3xp/teste2projeto

## Overview

##### Project type and purpose
- This is a standalone machine-learning-driven text-classification pipeline (COMLProcessing) that reads pre-labeled news articles, trains multiple classification pipelines, evaluates them, picks the best model, and then applies that model to unlabeled news—storing all results in MongoDB.

##### File structure
- Top-level scripts and modules:
- main.py: orchestration entry point
- prompt.py: (likely handles user prompts or dynamic parameter injection)
- requirements.txt: Python dependencies
- data/ directory: data ingestion & transformation
- database.py: opens SSH tunnel & connects to remote DB
- preprocessing.py: text cleaning and normalization
- models/ directory: feature engineering & modeling
- vectorizers.py: TF-IDF or custom vectorizers
- pipelines.py: definitions of sklearn or similar classification pipelines
- evaluation/ directory: model scoring & selection
- evaluation.py: cross-validation, metric computation, best-model selector
- Other:
- caracteristicas_extraidas.csv: sample or cached features
- README.md: setup & workflow description

##### README for additional insights
- Workflow:
1. Read credentials from config.yaml
2. Via SSH + bind to MongoDB
3. Pull “accepted” (manually labeled) news → run feature pipelines → store all intermediate & final scores in MongoDB collection `classification_pipelines`
4. Pick best-performing pipeline → classify unlabeled news → write predictions back to MongoDB

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Orchestration Layer"
        Config["Configuration Loader\n(config.yaml)"]:::orchestrator
        Prompt["Prompt Module\nprompt.py"]:::orchestrator
        Orchestrator["Orchestrator\nmain.py"]:::orchestrator
    end

    subgraph "Data Layer"
        DBConnector["DB Connector\nSSH Tunnel + MongoDB Client"]:::data
        DataIngest["Data Ingestion\naccepted & unaccepted news"]:::data
        Preprocessing["Preprocessing Module\ndata/preprocessing.py"]:::data
    end

    subgraph "Modeling Layer"
        Vectorizers["Feature Engineering\nmodels/vectorizers.py"]:::model
        Pipelines["Model Training & Pipelines\nmodels/pipelines.py"]:::model
        Evaluation["Evaluation & Selection\nevaluation/evaluation.py"]:::model
    end

    MongoDB["MongoDB Storage\n(classification_pipelines,\naccepted_news, unaccepted_news)"]:::storage

    subgraph "Project Files"
        CSV["Sample Features CSV\ncaracteristicas_extraidas.csv"]:::external
        Req["Dependencies\nrequirements.txt"]:::external
        Readme["Overview & Workflow\nREADME.md"]:::external
    end

    Config --> Orchestrator
    Prompt --> Orchestrator
    Orchestrator -->|Reads config & prompts| DBConnector

    Orchestrator -->|Read accepted news| DBConnector
    DBConnector -->|Fetch accepted_news| DataIngest
    DataIngest --> Preprocessing
    Preprocessing --> Vectorizers
    Vectorizers --> Pipelines
    Pipelines --> Evaluation
    Evaluation -->|Store metrics & models| MongoDB

    Evaluation -->|Best-model selected| Orchestrator

    Orchestrator -->|Read unaccepted news| DBConnector
    DBConnector -->|Fetch unaccepted_news| DataIngest
    DataIngest --> Preprocessing
    Preprocessing --> Vectorizers
    Vectorizers -->|Apply best model| Pipelines
    Pipelines -->|Predictions| MongoDB

    click DBConnector "https://github.com/krt3xp/teste2projeto/blob/main/data/database.py"
    click Preprocessing "https://github.com/krt3xp/teste2projeto/blob/main/data/preprocessing.py"
    click Vectorizers "https://github.com/krt3xp/teste2projeto/blob/main/models/vectorizers.py"
    click Pipelines "https://github.com/krt3xp/teste2projeto/blob/main/models/pipelines.py"
    click Evaluation "https://github.com/krt3xp/teste2projeto/blob/main/evaluation/evaluation.py"
    click Orchestrator "https://github.com/krt3xp/teste2projeto/blob/main/main.py"
    click Prompt "https://github.com/krt3xp/teste2projeto/blob/main/prompt.py"
    click CSV "https://github.com/krt3xp/teste2projeto/blob/main/caracteristicas_extraidas.csv"
    click Req "https://github.com/krt3xp/teste2projeto/blob/main/requirements.txt"
    click Readme "https://github.com/krt3xp/teste2projeto/blob/main/README.md"

    classDef orchestrator fill:#D6EFFF,stroke:#333,stroke-width:1px
    classDef data fill:#E5F5E0,stroke:#333,stroke-width:1px
    classDef model fill:#FFE5B4,stroke:#333,stroke-width:1px
    classDef storage fill:#E6E0F8,stroke:#333,stroke-width:1px
    classDef external fill:#F2F2F2,stroke:#333,stroke-width:1px
```
