ysn-b13/ask-fsdl-2026

Purpose and Entry Points

askFSDL is a retrieval-augmented QA system over Full Stack Deep Learning educational content. Users ask questions through a Discord bot or a lightweight Gradio UI; the backend retrieves corpus passages and uses an OpenAI-backed LangChain flow to generate answers.

Primary runtime entry points:

• app.py — application backend / Gradio-facing service.

• bot.py — Discord interactions endpoint and bot frontend.

• tasks/run_backend_modal.sh, tasks/run_frontend_modal.sh — Modal deployment execution paths.

Request-to-Answer Pipeline

The online path accepts a user question, retrieves semantically relevant corpus chunks, builds an LLM prompt, and returns a generated response. The central application boundary is between request adapters (app.py, bot.py) and retrieval/generation modules (vecstore.py, prompts.py).

• app.py

• bot.py

• prompts.py

Document Store: Persistent Corpus

MongoDB Atlas stores the cleaned, structured source corpus and is the durable document-system boundary. Corpus metadata/source inventories are represented in local JSON inputs, while document access is encapsulated by the document-store layer.

• docstore.py

• data/lectures-2022.json

• data/llm-papers.json

Vector Retrieval Index

FAISS provides low-latency similarity search over document embeddings. vecstore.py is the retrieval abstraction that bridges application queries to the vector index; it should remain decoupled from Discord/UI concerns and from ETL extraction details.

• vecstore.py

• docstore.py

• utils.py

Prompting and LLM Invocation

LangChain organizes LLM calls and prompt composition. Prompt definitions are separated into prompts.py, allowing retrieval context and user input to be assembled consistently before sending requests to OpenAI. OpenAI is the cost-sensitive external generation dependency.

• prompts.py

• app.py

• .env.example

Offline ETL and Index Refresh

Batch ETL converts source materials into cleaned documents suitable for MongoDB and embedding/index construction. Format-specific extractors handle Markdown, PDFs, and videos, with shared transformation logic in the ETL package. Modal also executes these batch jobs.

• etl/markdown.py

• etl/pdfs.py

• etl/videos.py

• etl/shared.py

• Running ETL to Build the Document Corpus.ipynb

Deployment, Integrations, and Observability

Modal hosts both serverless application endpoints and batch processing. Discord invokes the bot through interactions endpoints. Gantry monitors production model behavior and gathers user feedback. MongoDB Atlas, OpenAI, Discord, Modal, and Gantry are the principal external-service boundaries.

• tasks/run_etl.sh

• tasks/setup_environment_modal.sh

• tasks/send_secrets_to_modal.sh

• setup/mongodb

• setup/openai

Operational Flow

1. ETL ingests educational sources, cleans/chunks them, and persists documents to MongoDB.

2. Embeddings/index refresh produces a FAISS retrieval artifact.

3. Discord or Gradio submits a question to the Modal-hosted backend.

4. The backend retrieves relevant chunks, applies prompt templates, and calls OpenAI.

5. The generated answer is returned to the requesting frontend; Gantry records production behavior/feedback.