How pat-jj/deepretrieval works
Main Components and Their Interactions
• Data Ingestion/Preprocessing:
– Represent modules under “code/data_preprocess” which contain scripts for processing various datasets (PubMed, clinical trials, BM25 and Dense configurations, SQL-based data, etc.).
– Include two paths: (a) direct data download from Huggingface (preprocessed datasets) and (b) local raw data processing option.
• Core Training Pipeline and Reinforcement Learning:
– Show the RL training components under “code/verl” and “code/src/eval”, which harness distributed training (with scripts in “code/scripts/train”) and evaluation (in “code/scripts/eval”).
– Highlight that the LLM generates reasoning steps (chain-of-thought) for query generation, and it receives rewards based on retrieval performance. Indicate the reward function modules under “code/verl/utils/reward_score” where different retrieval metrics are computed.
• Model Serving and Evaluation:
– Include evaluation components under “code/examples”, “code/src/eval”, and search engine integration modules under “code/src/eval/SearchEngine”. The system interacts with external APIs (e.g., PubMed API) to benchmark query performance.
• Distributed and Parallel Components:
– Illustrate the distributed training elements, including Ray-based orchestration (directories like “code/tests/ray” and “code/verl/single_controller/ray”) and Megatron distributed practices (see “code/verl/models/llama/megatron” and associated worker/actor structures).
• External Dependencies and Infrastructure:
– Note the use of external libraries (PyTorch, vllm, Ray, pyserini, FAISS, Java via install-jdk, etc.) that support computation, retrieval, and training setups.
– Include Docker containerization (located in “code/docker”) which is used for deployment and ensuring environment consistency.
Key Architectural Patterns and Design Principles
• Modular Design:
– The project separates data preprocessing, training, evaluation, and model-serving into distinct modules, making it easier to maintain and extend.
• Distributed and Parallel Architecture:
– Emphasize the use of distributed training strategies using Ray and Megatron. This is visible in both the “verl” (model training and orchestration) and “tests” directories, which contain worker and orchestration scripts.
• Extensibility and Testability:
– With dedicated “tests” directories for e2e, sanity, rollout, and GPU utilities, the architecture supports comprehensive testing. Also, the “docs” and “examples” folders provide clear usage examples and configuration variants.
• Data-Driven Reinforcement Learning:
– The training loop centers on LLM query generation with inbuilt chain-of-thought reasoning, using retrieval metrics as training rewards. Illustrate this feedback loop between the model’s output and the reward computation module.