How nom-network/melba-toast-ai works
Project type and purpose
• This is an open-source, CLI-driven inference tool (“Melba-Toast-AI”) that wraps a GGUF-format LLM via llama-cpp-python, uses ChromaDB for vector-store memory, and employs NRCLex for sentiment/Affect analysis.
• It is neither a full-stack web app nor a compiler; it’s a backend-only “inference engine” library + reference CLI interface.
File structure
• Top-level:
• README.md, LICENSE.md, docs/ (static docs), examples/ (usage sample), memories/ (seed memory text files)
• src/ (all code)
• src/:
• melbaToast.py → entry-point orchestrator & CLI interface
• LLMCore.py → wraps llama-cpp load/infer routines
• LLMUtils.py → shared utilities (tokenization, prompt formatting, logging)
• memoryDB.py + initMemoryDB.py → handle ChromaDB vector store init, persist & retrieve “memories”
• llmbackendClient.py → client API abstraction for external invocation of the engine
README for additional insights
• Requires llama-cpp-python (local GGUF model runner), ChromaDB (vector DB), NRCLex (sentiment lexicon).
• Models must be converted/quantized to GGUF beforehand.
Detail
• Six src/ modules, the “memories” directory as a seed store, example usage script, and external dependencies.