How cpeggy/net_learnin works
Project type and purpose
• This is a “net_learning” monorepo documenting weekly Python-based exercises in a network-assisted learning system. Each week’s folder (week2rec…week9rec) contains standalone scripts that demonstrate:
• AI agent orchestration (Hugging Face pipelines, Gemini API)
• Data-agent patterns (authorization, data ingestion)
• Web scraping/Playwright demos
• CSV/JSON export
• Public opinion analysis with Google Custom Search
• The final deliverable (“輿情ana”) is a public-opinion-analysis web app (a Flask “app.py” at https://github.com/cpeggy/poa).
File structure
• Top-level folders:
• week2rec…week9rec: batch scripts + inputs (CSV, test files, interview MDs)
• 輿情ana: final-project folder with README + Flask code
• Key patterns:
• No monolithic service—each week is a self-contained pipeline.
• Final project follows MVC via Flask (app.py serves routes, invokes AI modules, renders templates).
• External config: Python venv, requirements installed via pip.
Insights from README
• Emphasis on a multi-agent AI architecture (assistant, surfer, critic)
• Data-agent patterns for authorized company data
• Progressive integration: from CLI scripts to UI to web service
• Final project: a sentiment/opinion analyzer that ties together scraping, AI, and a web UI.
a. Main components
1. Data Sources
– Company data (authorized dataset)
– Google Custom Search API
– Central Weather Agency website (via Playwright)
– Local CSV/JSON files
2. Data Ingestion Layer
– Python modules in week3rec, week9rec using requests/playwright
3. AI Processing Layer
– Hugging Face pipelines (torch/transformers)
– Gemini API multi-agent orchestration
4. Storage Layer
– Local CSV/JSON outputs
5. Application Layer
– Batch CLI scripts (weeks 2–9)
– Web UI (Flask app in 輿情ana)
6. External Services
– Hugging Face Hub
– Gemini API endpoint
– Google Custom Search
b. Relationships & data flow
Data Sources → Ingestion Layer → AI Processing → Storage
Application Layer → invokes Ingestion & AI → returns to UI or CLI → writes to Storage
Web UI → Application Layer → AI Processing → Response → UI
– Show API calls (Google/Gemini) as outbound from AI Processing
c. Architectural patterns
– Modular pipeline: clearly separated ingestion, processing, storage
– MVC in Flask: Route handlers (Controller) → AI modules (Model) → Templates/JSON (View)
– Agent orchestration: multi-agent calls with termination logic
d. Core technologies
– Python 3.x, venv
– requests, playwright, google-api-python-client
– torch, transformers (Hugging Face)
– Gemini API client
– Flask (web framework)
– CSV/JSON file I/O