How rkumar70900/dungeons-and-dragons-game works
Project Type and Purpose
• Type: Backend API service (FastAPI) orchestrating data ingestion, model training, inference, and logging.
• Purpose: Generate and manage D&D character attributes and context via REST endpoints, backed by various data pipelines and storage layers.
File Structure
Top-level:
– main.py (FastAPI app)
– logger.py (logging utility)
– pyproject.toml, test.py
Pipelines:
– feature-pipeline/ (data ingestion: CSV scrapers, cleaning, NoSQL + vector DB)
– training_pipeline/ (LLM-based training via ask_llm.py)
– inference_pipeline/ (core game logic, dice rolls, MongoDB connectors, DataFrame filters)
Data stores & config:
– Redis (caching via middleware in main.py)
– MongoDB (context_data collections)
– Vector database (feature-pipeline/vector_database.py)
– Local CSV sources (weapons.csv, armor.csv)
README for Insights
– FastAPI endpoints for each character attribute stage (race, class, abilities, etc.)
– Detailed list of inference methods (roll_dice, calculate_attack_modifier, etc.)
– Logging modules for success/failure
– Dependencies: FastAPI, Pydantic, Redis, uuid, inspect, pandas, logger, inference_pipeline
Main Components & Interactions
a. Components:
1. API Layer (FastAPI in main.py)
2. Inference Pipeline (inference_pipeline/dep.py)
3. Training Pipeline (training_pipeline/ask_llm.py)
4. Feature Pipeline (feature-pipeline scrapers, data_cleaning, vector_database)
5. Data Stores:
– Redis (cache/session)
– MongoDB (character context & raw data)
– Vector DB (embeddings or similarity search)
6. Logging (logger.py or logger/ modules)
7. External LLM (when training or inference uses LLM calls)
b. Relationships & Data Flow:
1. Offline Data Ingestion: feature-pipeline scrapes CSV → cleans → stores in MongoDB & vector DB.
2. Model Training: training_pipeline invokes ask_llm → generates embeddings or fine-tunes → updates vector DB.
3. API Request: client → FastAPI endpoints → middleware clears/reads Redis → calls inference_pipeline methods → reads/writes MongoDB → optionally queries vector DB → returns response → writes to Redis for caching.
4. Logging: each success/failure in any component → logger writes to logfile or stdout.
c. Architectural Patterns:
– Layered Architecture: API layer → Service layer (inference/training/feature) → Data layer
– Separation of Concerns: distinct pipelines for ingestion, training, inference
– Caching Pattern: Redis for repeat calls
– Adapter Pattern: dep.py adapts MongoDB, pandas, LLM calls into uniform service methods
d. Technologies & Frameworks:
– FastAPI (API layer)
– Pydantic (request/response models)
– Redis (cache)
– MongoDB & PyMongo (document store)
– Pandas (data transformations)
– Vector DB (similarity search)
– Python logging or custom logger modules
– External LLM APIs (e.g., OpenAI) used in training_pipeline
Final Checklist
– All major components distinctly labeled
– Separation between online (API & inference) and offline (feature, training) processes
– Callouts for caching, logging, external dependencies