How madhavbala/genai_main works
Project type and purpose
• This repository is not a single deployable application but a curated “Generative‑AI Mastery Resources” collection.
• It contains Jupyter notebooks, ZIP archives of example projects, and instructional materials organized by topic (e.g., Data Preprocessing, Fine‑Tuning, RAG, LLMOps, LangChain, LlamaIndex).
• Its purpose is to serve as a central knowledge base and hands‑on lab repository for engineers learning Generative AI techniques.
File structure
• Top‑level entries:
• .gitignore and LICENSE – standard repo metadata
• README.md – very brief title
• A handful of directories, each named after a topic area:
– “Data Preprocessing and Embeddings”
– “End to End Projects”
– “Fine Tuning LLMs”
– “Huggingface Platform and its API”
– “LLMOps”
– “LangChain - Basic to Advance”
– “Learn to use Open Source LLMs”
– “LlamaIndex - Basic to Advance”
– “Master Vector Database for LLMs”
– “Retrieval Augmented Generation (RAG)”
• Within each, either Jupyter notebooks (.ipynb) or ZIP archives containing example code and datasets.
• No code‑structured directories (e.g., src/, backend/, frontend/), build scripts, or deployment configurations are present.
README for additional insights
• The README only provides the repository title.
• No architectural guidance or technology stack is documented there.
Given that this is a static resource repository, we will model the *content lifecycle* and *user consumption* flow rather than a service‑oriented microservice system.
a. Main components
• GitHub Repository (source of truth for all ZIPs & notebooks)
• Continuous Integration / Content Pipeline (optional)
• Static Site Generator or Documentation Builder (e.g., MkDocs, Sphinx)
• Hosted Static Content Store (e.g., GitHub Pages, Netlify, S3 + CloudFront)
• Search/Index Engine (e.g., Algolia, ElasticSearch)
• End User’s Web Browser
b. Relationships and interactions
1. Content Editor pushes notebooks and ZIPs to the GitHub Repository.
2. GitHub Actions (CI) triggers on push:
– Validates notebook syntax
– Converts notebooks to HTML/Markdown
– Bundles ZIP samples for download
– Deploys output to the Static Content Store
– Updates Search Index with metadata (topic, file name, tags)
3. End Users access the Static Content Store via a browser:
– Browse categories (menus for “Data Preprocessing,” “Fine Tuning,” etc.)
– Click to view rendered notebooks or download example ZIPs
4. (Optional) Analytics service to track user engagement
c. Architectural patterns & design principles
• Content‑centric static site (no servers at runtime)
• CI/CD for content validation and publishing
• Separation of concerns:
– Source repo (authoring)
– CI/CD pipeline (build & index)
– Hosting & search (read‑only delivery)
• Modular categorization: each topic area is a logical “module”
d. Key technologies and frameworks
• Git/GitHub – version control and authoring
• GitHub Actions – build and deployment pipeline
• Jupyter Notebook – interactive content authoring
• Static site generator (MkDocs, Sphinx, Docusaurus)
• Hosting: GitHub Pages / Netlify / AWS S3 + CloudFront
• Search: Algolia or ElasticSearch for real‑time search index