How platzi/curso-langchain-qa-documents works
Main Components
• User Interface (UI): Represented by app.py which is launched via Streamlit. This component provides the front-end where users can input queries and view results.
• Ingestion Engine: The ingest.py script is used to load and perhaps index documents. This script likely calls helper functions from the src (specifically src/langchain_docs_loader.py) to process and prepare documents.
• Core LangChain Components: The src directory (and its langchain_docs_loader module) embodies the logic for handling document retrieval – especially since the course is titled “Curso de LangChain para Manejo y Recuperación de Documentos.”
• External APIs: Highlight external services that are integrated into the project. The README mentions OpenAI, Docugami, and Cohere API keys. These services provide the underlying NLP capabilities (language models, semantic processing, etc.).
• Documentation & Notebooks: Although not part of runtime, the notebooks (both exercises and resolved versions in notebooks_ejercicios and notebooks_resueltas) and docs directories play an important educational role. They show various retrieval patterns (e.g., context-aware extraction, text splitting, semantic reranking, multi-query, ensemble retrievers, etc.) and should be noted in the architecture as examples of different strategies.
Relationships & Data Flow
• Start at the User Interface (Streamlit – app.py). When the user submits a query, this interface triggers calls to the backend.
• The backend logic (which may involve aspects from ingest.py and the src modules) processes the query. This might include selecting and applying a specific retrieval algorithm discussed in the notebooks.
• External API calls (to OpenAI, Docugami, Cohere) would be invoked as part of the processing – for example, to obtain embeddings, perform semantic ranking, or language modeling.
• Once processing is complete, results flow back to the UI to be rendered for the user.
• The ingestion process (if run separately via ingest.py) sets up the document database (or vector indexing) that the UI and retrieval logic will use.
Important Architectural Patterns
• Layered Architecture: Clearly separate UI/presentation (Streamlit) from ingestion and processing logic.
• Modular Design: The src module, notebooks (as experimental and learning modules), and separate ingestion script illustrate modularity.
• Integration with External Services: Emphasize how external language APIs are integrated into the process.
• Educational Variants: The existence of multiple notebooks demonstrating different retrieval mechanisms suggests a divergence of strategies (perhaps akin to different “plugins” or configuration options) utilized in similar overall workflows.
Relevant Technologies & Frameworks
• Python as the main language.
• Streamlit for the front-end web app.
• LangChain (implicitly used within src/langchain_docs_loader.py and others) for document retrieval chains.
• External NLP APIs from OpenAI, Docugami, and Cohere.
• Poetry & pyenv for dependency and Python version management.
• Jupyter notebooks demonstrating various strategies should be mentioned as experimental/learning modules.
• Start at a high level, then break each piece down into its subcomponents.
• To capture the layered architecture (UI, processing, external services) clearly.
• With technology details, for instance, list “Streamlit” below app.py and “LangChain & Custom Loader” below the core processing module.
• Clearly label any external API integrations and how these are triggered on-demand by the processing layer.