How akshayacharyan/text_summarizer_chatbot works
Project Type and Purpose
• This is a single-page web application built with Streamlit.
• It allows users to upload PDF files, extract and clean their text, generate summaries, and ask questions via a chatbot interface powered by Hugging Face models.
File Structure Analysis
• Top-level scripts:
• app.py ‑ the Streamlit application entrypoint (UI + orchestration)
• QA_chatbot.py / llama_qa.py ‑ modules for question-answering logic
• summarizer.py ‑ summarization logic
• pdf_extractor.py ‑ PDF text extraction using PyPDF2
• text_cleaner.py ‑ text preprocessing and cleaning
• test.py ‑ test harness for components
• Dev container support in .devcontainer/devcontainer.json for consistent development environment
• requirements.txt lists Python dependencies (Streamlit, Transformers, PyPDF2, etc.)
Key Insights from README
• Streamlit provides the user interface and handles HTTP interactions.
• Hugging Face Transformers power both summarization and Q&A models.
• No external database; uploaded PDFs are processed in memory or temporary storage.
a. Main Components
• User Browser (UI)
• Streamlit Server (app.py)
• PDF Extraction Module (pdf_extractor.py)
• Text Cleaning Module (text_cleaner.py)
• Summarization Module (summarizer.py)
• Q&A Module (QA_chatbot.py / llama_qa.py)
• Hugging Face Model Hosts (local or remote)
• Temporary File Storage (in-memory or disk)
b. Relationships and Data Flow
1. User uploads PDF via Streamlit UI
2. Streamlit Server invokes PDF Extraction Module
3. Extracted text passes to Text Cleaning Module
4. Clean text branches to either:
– Summarization Module → calls HF Summarization Model → returns summary
– Q&A Module → calls HF QA Model → returns answer
5. Streamlit displays results back to the user
c. Architectural Patterns
• Monolithic service with clear functional modules (extraction, cleaning, NLP tasks)
• Sequential data pipeline inside one process
• External dependency calls (Hugging Face Inference or local model)
d. Technologies & Frameworks
• Streamlit for UI & HTTP handling
• PyPDF2 for PDF text extraction
• Python modules orchestrating data flow
• Hugging Face Transformers for NLP tasks