# cpeggy/net_learnin architecture

> Architecture diagram of the GitHub repository cpeggy/net_learnin, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cpeggy/net_learnin
- Repository: https://github.com/cpeggy/net_learnin

## Overview

##### Project type and purpose
- This is a “net_learning” monorepo documenting weekly Python-based exercises in a network-assisted learning system. Each week’s folder (week2rec…week9rec) contains standalone scripts that demonstrate:
- AI agent orchestration (Hugging Face pipelines, Gemini API)
- Data-agent patterns (authorization, data ingestion)
- Web scraping/Playwright demos
- CSV/JSON export
- Public opinion analysis with Google Custom Search
- The final deliverable (“輿情ana”) is a public-opinion-analysis web app (a Flask “app.py” at https://github.com/cpeggy/poa).

##### File structure
- Top-level folders:
- week2rec…week9rec: batch scripts + inputs (CSV, test files, interview MDs)
- 輿情ana: final-project folder with README + Flask code
- Key patterns:
- No monolithic service—each week is a self-contained pipeline.
- Final project follows MVC via Flask (app.py serves routes, invokes AI modules, renders templates).
- External config: Python venv, requirements installed via pip.

##### Insights from README
- Emphasis on a multi-agent AI architecture (assistant, surfer, critic)
- Data-agent patterns for authorized company data
- Progressive integration: from CLI scripts to UI to web service
- Final project: a sentiment/opinion analyzer that ties together scraping, AI, and a web UI.

a. Main components
1. Data Sources
– Company data (authorized dataset)
– Google Custom Search API
– Central Weather Agency website (via Playwright)
– Local CSV/JSON files
2. Data Ingestion Layer
– Python modules in week3rec, week9rec using requests/playwright
3. AI Processing Layer
– Hugging Face pipelines (torch/transformers)
– Gemini API multi-agent orchestration
4. Storage Layer
– Local CSV/JSON outputs
5. Application Layer
– Batch CLI scripts (weeks 2–9)
– Web UI (Flask app in 輿情ana)
6. External Services
– Hugging Face Hub
– Gemini API endpoint
– Google Custom Search
b. Relationships & data flow
Data Sources → Ingestion Layer → AI Processing → Storage
Application Layer → invokes Ingestion & AI → returns to UI or CLI → writes to Storage
Web UI → Application Layer → AI Processing → Response → UI
– Show API calls (Google/Gemini) as outbound from AI Processing
c. Architectural patterns
– Modular pipeline: clearly separated ingestion, processing, storage
– MVC in Flask: Route handlers (Controller) → AI modules (Model) → Templates/JSON (View)
– Agent orchestration: multi-agent calls with termination logic
d. Core technologies
– Python 3.x, venv
– requests, playwright, google-api-python-client
– torch, transformers (Hugging Face)
– Gemini API client
– Flask (web framework)
– CSV/JSON file I/O

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Sources
    CompanyData["Company Data (Authorized)"]:::external
    GoogleAPI["Google Custom Search API"]:::external
    CWB["Central Weather Agency Site"]:::external
    LocalFiles["Local CSV/JSON Files"]:::external

    %% Ingestion Layer
    DataAgent["Data-Agent Ingestion"]:::processing
    WebScraping["Playwright Scraping"]:::processing

    %% AI Processing Layer
    HF["Hugging Face Pipelines"]:::processing
    Gemini["Gemini API Multi-Agent"]:::processing

    %% Storage Layer
    CSVJSON["CSV/JSON Storage"]:::storage

    %% Batch Processing Subgraph
    subgraph "Batch Processing"
        week2rec_test_aiage3["week2rec/test_aiage3.py"]:::cli
        week2rec_test_aimodel["week2rec/test_aimodel.py"]:::cli
        week3rec_proj_dataagent["week3rec/proj_dataagent.py"]:::cli
        week3rec_test_dataagent["week3rec/test_dataagent.py"]:::cli
        week4rec_proj_dataagentUI["week4rec/proj_dataagentUI.py"]:::cli
        week5rec_proj_anasaying["week5rec/proj_anasaying.py"]:::cli
        week7rec_proj_dataper["week7rec/proj_dataper.py"]:::cli
        week8rec_proj_csvstream["week8rec/record/proj_csvstream.py"]:::cli
        week8rec_proj_mdstream["week8rec/record/proj_mdstream.py"]:::cli
        week8rec_proj_perfeedback["week8rec/record/proj_perfeedbacktest.py"]:::cli
        week9rec_pra_playoutput["week9rec/pra_playoutput.py"]:::cli
    end

    %% Web Application Subgraph
    subgraph "Web Application"
        FlaskApp["Flask app.py"]:::ui
        Templates["Templates / Views"]:::ui
    end

    %% Relationships
    CompanyData -->|"HTTP/API"| DataAgent
    LocalFiles -->|"File Read"| DataAgent
    CWB -->|"Playwright"| WebScraping
    DataAgent -->|"data feed"| HF
    WebScraping -->|"scraped data"| HF
    GoogleAPI -->|"search API"| Gemini
    HF -->|"inference"| CSVJSON
    Gemini -->|"agent orchestration"| CSVJSON

    %% Batch invokes
    week2rec_test_aiage3 -->|invokes| HF
    week2rec_test_aimodel -->|invokes| HF
    week3rec_proj_dataagent -->|invokes| DataAgent
    week4rec_proj_dataagentUI -->|invokes| DataAgent
    week5rec_proj_anasaying -->|invokes| Gemini
    week7rec_proj_dataper -->|invokes| Gemini
    week8rec_proj_csvstream -->|writes| CSVJSON
    week8rec_proj_mdstream -->|writes| CSVJSON
    week8rec_proj_perfeedback -->|writes| CSVJSON
    week9rec_pra_playoutput -->|invokes| WebScraping

    %% Web App flow
    FlaskApp -->|calls| Gemini
    FlaskApp -->|calls| HF
    Gemini -->|response| Templates
    HF -->|response| Templates
    Templates -->|render UI| FlaskApp
    FlaskApp -->|writes| CSVJSON

    %% Click Events
    click week2rec_test_aiage3 "https://github.com/cpeggy/net_learnin/blob/main/week2rec/test_aiage3.py"
    click week2rec_test_aimodel "https://github.com/cpeggy/net_learnin/blob/main/week2rec/test_aimodel.py"
    click week3rec_proj_dataagent "https://github.com/cpeggy/net_learnin/blob/main/week3rec/proj_dataagent.py"
    click week3rec_test_dataagent "https://github.com/cpeggy/net_learnin/blob/main/week3rec/test_dataagent.py"
    click week4rec_proj_dataagentUI "https://github.com/cpeggy/net_learnin/blob/main/week4rec/proj_dataagentUI.py"
    click week5rec_proj_anasaying "https://github.com/cpeggy/net_learnin/blob/main/week5rec/proj_anasaying.py"
    click week7rec_proj_dataper "https://github.com/cpeggy/net_learnin/blob/main/week7rec/proj_dataper.py"
    click week8rec_proj_csvstream "https://github.com/cpeggy/net_learnin/blob/main/week8rec/record/proj_csvstream.py"
    click week8rec_proj_mdstream "https://github.com/cpeggy/net_learnin/blob/main/week8rec/record/proj_mdstream.py"
    click week8rec_proj_perfeedback "https://github.com/cpeggy/net_learnin/blob/main/week8rec/record/proj_perfeedbacktest.py"
    click week9rec_pra_playoutput "https://github.com/cpeggy/net_learnin/blob/main/week9rec/pra_playoutput.py"

    %% Styles
    classDef external fill:#B0E0E6,stroke:#333,stroke-width:1px
    classDef processing fill:#98FB98,stroke:#333,stroke-width:1px
    classDef storage fill:#FFFACD,stroke:#333,stroke-width:1px
    classDef ui fill:#FFA500,stroke:#333,stroke-width:1px
    classDef cli fill:#DDA0DD,stroke:#333,stroke-width:1px
```
