How aparimeyatiwari/msq_bot_podcast works
Project Type and Purpose
• This is a full-stack AI-driven Podcast Question Generator.
• The frontend is a simple Flask-based web UI (upload a PDF or transcript, view generated questions).
• The core backend logic lives in the AI_Generator (and mirrored in my_package) Python modules that orchestrate several Ollama-deployed LLM models (chunking, theming, question generation, NER, formatting).
File Structure Analysis
Top-level:
• README.md & requirements.txt – project overview and Python dependencies
• flask_code/ – Flask app (app.py), HTML templates, static assets (CSS), upload & JSON folders
• AI_Generator/ – business-logic package: model wrappers and NLP pipelines
• extraction/ – example inputs & outputs for resume-parsing tests (can be included as sample data)
• my_package/ – duplicate of AI_Generator for packaging or testing
• frontend.py – (optional entry point)
Key patterns:
• Separation between web layer (flask_code/) vs. AI processing (AI_Generator/)
• Configuration via Ollama CLI – each model is loaded into a local Ollama daemon
• File I/O: uploads stored in flask_code/uploads; generated JSON in flask_code/json
Diagramming the Architecture
• Client Browser (uploads, views UI)
• Flask Web Server (app.py)
• File Storage (uploads folder, JSON folder)
• AI Service Layer (AI_Generator / my_package modules)
• Ollama Model Server (local Ollama runtime hosting llama3‐based models)
• External APIs (e.g., YouTube Search Agent if used)
b. Relationships and Data Flow:
1. User → Browser → HTTP POST (PDF upload) → Flask endpoint
2. Flask writes file to “uploads/”, invokes AI_Generator pipeline
3. AI_Generator:
• resume_parse.py or yt_search_agent.py reads file / fetches transcript
• text_ner.py performs named‐entity recognition via model “custom_ner_model”
• chunk_theme_model & final_theme_model produce thematic segments
• question_generation_model generates question set
• format_model post-processes into JSON/text
4. For each model call, AI_Generator makes an Ollama API/CLI call → Ollama runtime
5. Processed JSON written back to flask_code/json/
6. Flask endpoint returns JSON to client → Browser renders page2.html
c. Architectural Patterns & Principles:
• Layered architecture: Presentation (Flask templates), Application (Flask routes), Domain/AI (AI_Generator modules), Infrastructure (Ollama, file system)
• Single Responsibility: Each model wrapper module handles one task (NER, chunking, theming, Q-gen)
• Externalize LLMs: Models run in Ollama, decoupling heavy AI inference from Python code
d. Key Technologies / Frameworks:
• Python 3 + Flask
• Ollama CLI & llama3 models
• Jinja2 templates, static CSS
• Local file storage (uploads, JSON)
• (Optionally) YouTube Data API via yt_search_agent.py
Critical Details to Capture
• Each LLM model’s role and Ollama mapping (e.g., “question_generation_model → podcast_model”)
• The sequential pipeline: parse → NER → chunk/theme → question gen → format
• Where files live on disk vs. in-memory transfer
• Entry points: app.py endpoints, questions_gen.py functions