# shresthasingh1501/legal_document_analysis architecture

> Architecture diagram of the GitHub repository shresthasingh1501/legal_document_analysis, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shresthasingh1501/legal_document_analysis
- Repository: https://github.com/shresthasingh1501/legal_document_analysis

## Overview

##### Project type and purpose
- This is a specialized NLP pipeline (“Legal Document Analysis”) that takes PDF documents as input, extracts text (with OCR fallback), preprocesses it, and then runs two parallel ML workflows: summarization (Flan-T5) and named-entity recognition (BERT). There is also an optional Streamlit UI for interactive use.

##### File structure
- Top-level:
- code/analysis.py – core pipeline orchestration (PDF→text→preprocessing→inference)
- code/bert-ocr.py – text extraction + OCR for scanned docs
- streamlit-ui.py – optional front-end service layer
- notebooks/… – Jupyter notebooks for model fine-tuning (training flow, not part of runtime)
- samplepdfs/ – directory for test PDF inputs
- requirements.txt – dependency list (PyTorch, Hugging Face Transformers, Tesseract-OCR, Streamlit)

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Input & Interface"
        PDF["PDF Documents"]:::inputOrange
        UI["Streamlit UI"]:::inputOrange
    end

    subgraph "Extraction & Preprocessing"
        TextExtractor["Text Extractor"]:::preprocessBlue
        OCR["OCR Fallback Module"]:::preprocessBlue
        Preprocessor["Preprocessing Module"]:::preprocessBlue
    end

    subgraph "Inference Services"
        Summ["Summarization Engine"]:::inferenceGreen
        NER["NER Engine"]:::inferenceGreen
    end

    Aggregator["Output Aggregator"]:::inputOrange
    Output["Results (JSON/UI)"]:::inputOrange

    subgraph "Training & Fine-Tuning"
        FlanNote["Flan-T5 Summarizer Notebook"]:::trainingGray
        NERNote["BERT NER Fine-Tune Notebook"]:::trainingGray
        NotebooksArtifacts["Model Development Artifacts"]:::trainingGray
    end

    subgraph "Documentation & Config"
        Req["requirements.txt"]:::docsGray
        Doc["readme.md"]:::docsGray
    end

    PDF -->|"raw PDF"| TextExtractor
    TextExtractor -->|"extracted text"| Preprocessor
    TextExtractor -->|"no text detected"| OCR
    OCR -->|"OCR text"| Preprocessor
    Preprocessor -->|"tokens"| Summ
    Preprocessor -->|"tokens"| NER
    Summ -->|"summary JSON"| Aggregator
    NER -->|"entities JSON"| Aggregator
    Aggregator -->|"final output"| Output
    UI -->|"uploads PDF"| TextExtractor
    NotebooksArtifacts --> FlanNote
    NotebooksArtifacts --> NERNote

    click PDF "https://github.com/shresthasingh1501/legal_document_analysis/tree/main/samplepdfs/"
    click TextExtractor "https://github.com/shresthasingh1501/legal_document_analysis/blob/main/code/analysis.py"
    click OCR "https://github.com/shresthasingh1501/legal_document_analysis/blob/main/code/bert-ocr.py"
    click Preprocessor "https://github.com/shresthasingh1501/legal_document_analysis/blob/main/code/analysis.py"
    click Summ "https://github.com/shresthasingh1501/legal_document_analysis/blob/main/code/analysis.py"
    click NER "https://github.com/shresthasingh1501/legal_document_analysis/blob/main/code/analysis.py"
    click Aggregator "https://github.com/shresthasingh1501/legal_document_analysis/blob/main/code/analysis.py"
    click UI "https://github.com/shresthasingh1501/legal_document_analysis/blob/main/streamlit-ui.py"
    click FlanNote "https://github.com/shresthasingh1501/legal_document_analysis/blob/main/notebooks/FlanT5-Summerizer.ipynb"
    click NERNote "https://github.com/shresthasingh1501/legal_document_analysis/blob/main/notebooks/Entity_ExtractionUsing_BERT_&_PyTorch_finetune.ipynb"
    click NotebooksArtifacts "https://github.com/shresthasingh1501/legal_document_analysis/tree/main/notebooks/"
    click Req "https://github.com/shresthasingh1501/legal_document_analysis/blob/main/requirements.txt"
    click Doc "https://github.com/shresthasingh1501/legal_document_analysis/blob/main/readme.md"

    classDef inputOrange fill:#FFA500,stroke:#333,stroke-width:1px
    classDef preprocessBlue fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef inferenceGreen fill:#90EE90,stroke:#333,stroke-width:1px
    classDef trainingGray fill:#D3D3D3,stroke:#333,stroke-width:1px
    classDef docsGray fill:#EEEEEE,stroke:#333,stroke-width:1px
```
