How ayoubchamakhi/blogrecommendation works
Project Type and Purpose
• This is a full-stack recommendation application with:
• A Streamlit-based frontend (app/app.py)
• A Python “backend” that handles data preprocessing, model training, inference, and evaluation (src/)
• File-based data persistence (data/raw → data/processed) and model storage (models/*.pkl)
• External dependency on the OpenAI API for personalized pitches
File Structure
• Top-level directories:
• app/: the Streamlit UI
• src/: core logic modules (preprocess, train, evaluate, inference, embedding_rec)
• data/: raw CSVs, Jupyter notebooks, processed pickle files
• models/: trained model artifacts (embedding_model.pkl, collaborative_model.pkl)
• Dockerfile & docker-compose.yaml: deployment configuration
• environment.yml: Python environment specification
• Notice the clear separation of concerns:
• Data ingestion & preprocessing (src/preprocess.py → data/processed)
• Model training & evaluation (src/train.py, src/evaluate.py → models/)
• Inference & recommendation logic (src/inference.py, src/embedding_rec.py)
Extract Key Architectural Insights from README
• Hybrid recommendation flow: content-based vs. collaborative fallback
• Streamlit UI flows: landing → guest vs. login → sidebar filters → main recommendations area
• External service: OpenAI for LLM-generated summaries
• No relational database—data lives in pickled files and in-memory Pandas DataFrames
Main Components and Their Interactions
1. Streamlit Frontend
• app/app.py renders UI, captures user input (login or guest) and filter settings
2. Inference Service
• src/inference.py & src/embedding_rec.py
• Loads models from models/*.pkl and processed data from data/processed
• Applies hybrid logic (content vs. collaborative)
3. OpenAI API
• External LLM call for personalized pitch generation
4. Data Layer
• data/raw → src/preprocess.py → data/processed
• Models directory for pickled artifacts
5. Training & Evaluation Pipelines
• src/train.py builds embedding and collaborative models
• src/evaluate.py runs grid search and evaluation → results/grid_search_results.csv
6. Deployment Infrastructure
• Dockerfile: containerizes the app
• docker-compose.yaml: orchestrates service with environment variables
• UI → Inference Service: user requests → recommendation results
• Inference Service → Data Layer: load processed data & models
• Inference Service → OpenAI API: HTTP call for text summaries
• Training Pipeline → Data Layer: reads raw data, writes processed data & model artifacts
• Evaluation Pipeline ↔ Models & Results: reads models & writes evaluation metrics
• Deployment Layer envelops the entire application stack
c. Architectural Patterns & Design Principles
• Layered architecture: UI, business logic (inference), data persistence, external services
• Pipeline pattern for data preprocessing and model training
• Modular code organization (one module per responsibility)
• Hybrid recommendation strategy implemented as decision logic in inference layer
d. Key Technologies & Frameworks
• Python 3.8+, Streamlit for UI
• Pandas, scikit-learn, scikit-surprise for ML pipelines
• SentenceTransformers (embedding_model.pkl) for content-based filtering
• OpenAI API for LLM summaries
• Docker & Docker Compose for containerized deployment
• Conda (environment.yml) for reproducible environments
Essential Elements but Keep It Readable
• Don’t overcrowd: group related src/ modules under a single “Recommendation Engine” block with sub-labels
• Hybrid logic decision point: “If user_ratings ≥ 5 → Collaborative Model else → Embedding Model”