How martinkimoe/api works
Project Type and Purpose
• This is a Python-based, monolithic backend API (Zeeguu-API) that exposes REST endpoints to track and model language learning.
• It is deployed as the backend for zeeguu.org and integrates with mobile/web clients, teacher dashboards, and batch tooling for feed crawling, indexing, and migrations.
High-Level File Structure Analysis
• zeeguu/api: HTTP routing layer (Flask-style endpoints served via WSGI/Apache2).
• zeeguu/core: Domain/business logic modules (account management, content retrieval, recommendation, NLP pipeline, emailer, scheduling, semantic search, etc.).
• zeeguu/core/model & zeeguu/core/sql: ORM + raw SQL models & queries (MySQL).
• tools: CLI utilities, data migrations, feed crawlers, batch scripts, report generators.
• config & docker files: Docker/Docker-compose setup, environment/config templates.
Main Components
A. API Layer
– “zeeguu/api/app.py” & “cross_domain_app.py” (entry points)
– Endpoint modules under zeeguu/api/endpoints (accounts, articles, sessions, teacher_dashboard, etc.)
– Utilities & middleware (authentication, CORS, JSON serialization)
B. Business-Logic Layer
– Core modules under zeeguu/core (content_retriever, content_cleaning, recommender, behavioral_modeling, nlp_pipeline, emailer, etc.)
– ML models & artifacts under zeeguu/core/ml_models
C. Persistence Layer
– MySQL relational database (zeeguu core/model)
– ElasticSearch index for semantic & text search (zeeguu/core/elastic)
D. External Services & Libraries
– Third-party NLP: Stanza, spaCy (zeeguu/core/tokenization, nlp_pipeline)
– Readability server & newspaper3k for parsing articles
– External translation services (if configured)
– Cron/batch runner invoking “tools/*” scripts
E. Infrastructure & DevOps
– Docker containers (API, MySQL, ElasticSearch) orchestrated via docker-compose
– Apache2 with mod_wsgi or gunicorn for serving the WSGI app
– CI pipelines (GitHub Actions under .github/workflows)
– Pre-commit hooks (in .githooks rules)
Relationships & Data Flows
• Client requests (web/mobile) → API Layer → Business Logic → Persistence (MySQL / ElasticSearch) → API Layer → Client
• Feed crawler (tools/article_crawler, tools/download_feed) → Business Logic → Persistence
• NLP pipeline modules ingest raw text from DB → annotate/clean → update DB / feed ElasticSearch
• Recommender modules pull user history from MySQL → compute suggestions → store feedback
• Emailer modules read events from business logic → deliver notifications (SMTP)
• Migrations scripts (tools/migrations) → schema changes in MySQL
Architectural Patterns & Principles
• Layered Architecture (Presentation / API, Domain Logic, Persistence)
• Hexagonal/Ports & Adapters (the core modules are independent of HTTP or DB adapters)
• Modular design: each zeeguu/core/<feature> is a self-contained domain service
• Separation of concerns: CLI tools vs. long-running API service
Technologies & Frameworks
• Python 3.x, Flask-style routing, WSGI (Apache2/mod_wsgi or gunicorn)
• SQLAlchemy or similar ORM + raw SQL
• MySQL 5.7 (relational store)
• ElasticSearch (semantic & text search index)
• Docker + Docker-compose for local development & deployment
• NLP: Stanza, spaCy, custom pipelines in zeeguu/core/nlp_pipeline
• Joblib models for paywall detection, custom ML in zeeguu/core/ml_models