How osamaashraf01/al-baheth works
Project type and purpose
• This is a full-stack search engine (“Al-baheth”) that lets users upload local files, index their contents using multiple indexing/NLP services, and run queries against those indexes.
• The README confirms a FastAPI backend (Python 3.8+), a React/Vite/TypeScript frontend, MongoDB as the primary database, and Elasticsearch (plus PyTerrier, NLTK, LangChain) for indexing & retrieval.
High-level component breakdown
a. Frontend
• A Vite-powered React/TypeScript single-page app (src/…).
• Components: file uploader, search bar, results panel, theme toggle.
• Calls backend REST endpoints.
b. Backend (FastAPI)
• API layer: routes (files, indexing, query) → controllers.
• Business logic layer: services (DirectoryService, IndexingService, ElasticSearchService, PyTerrierService, NLP services, ParsingService).
• Data layer: repositories + MongoDB documents/models.
• Environment: runs on Uvicorn, configured via .env.
c. External systems
• MongoDB (via Docker Compose).
• Elasticsearch (possibly spun up in Docker).
• NLP libs (NLTK) and retrieval frameworks (PyTerrier, LangChain) as internal/external service stubs.
d. File system
• Local directory scanning for ingestion.
Relationships & data flow
• User ↔ Frontend: HTTP/HTTPS.
• Frontend ↔ Backend: REST calls to /files (upload/index), /index (trigger index), /query (search).
• Backend → DirectoryService: reads files from disk.
• Backend → Repositories → MongoDB: store document metadata & full text.
• Backend → IndexingService → Elasticsearch & PyTerrier: push and retrieve indexed data.
• Backend → NLP & Parsing services: preprocess text, chain prompts.
• Query path: user query → QueryController → NLP/Parsing → search engines → repository fetch → response → frontend.
Architectural patterns & tech
• Layered (Controller → Service → Repository).
• Hexagonal/infrastructure-driven: core logic is framework-agnostic, injected via dependencies.py.
• Micro-libraries for indexing/NLP aggregated behind a unified service interface.