How bshraman/codebase-qna-search works
Main Components
• Configuration & Environment Setup:
• Configuration file (core/config.py) and note that environment variables (such as API keys and path settings) drive the behavior of the application.
• Core Processing Modules:
• Nodes:
• document_embedding.py (handles code/document embedding via LlamaIndex)
• query.py (handles query processing)
• vector_db.py (interfaces with ChromaDB for create/query operations)
• Graph and State:
• state_graph.py (constructs a state graph to represent workflows/interactions using LangGraph)
• state.py (manages overall state tracking)
• Routing:
• route.py (defines the workflow by checking conditions in the vector database and directing processing accordingly)
• External Services & Databases:
• ChromaDB for vector database operations.
• OpenAI API for natural language processing and interpreting queries.
• Entry Point:
• main.py: the application’s entry point that initializes and orchestrates various components.
• Documentation:
• docs folder for storing visual aids such as the compiled graph and output screenshots.
Relationships and Data Flow
• Creation and flow between various Nodes:
• Document Embedding Node uses LlamaIndex to process input documents.
• Query Node receives user queries and interacts with the Vector Database Node.
• Vector Database Node communicates with ChromaDB to either create a new collection or query an existing one.
• That once the nodes are prepared, the State Graph (build in state_graph.py using LangGraph) captures these interactions and workflows.
• The routing module (route.py) directs the process flow based on the state of the vector database (existence/absence of a collection).
Architectural Patterns and Design Principles
• Separation of Concerns: Each file in the core module (nodes, graph, routes, state) is responsible for a distinct part of the overall workflow.
• Modular Design: The project modules (configuration, nodes, routing, state graph) are well-separated which facilitates testing, maintenance, and potential expansion.
• Data Flow and State Management: The use of a state graph demonstrates a directed and managed flow. Also, using environment variables gives flexibility across deployment contexts.