How dmmiller612/bert-extractive-summarizer works
Project type and purpose
• This is a Python library plus optional REST service for extractive text summarization using BERT (and SBERT) embeddings, clustering, and coreference resolution.
• It provides both an importable “Summarizer” API (summarizer/…) and a Flask‐based HTTP service (server.py + Dockerfile.service).
File structure
• Top‐level:
– setup.py, setup.cfg, requirements.txt: packaging and installable library
– server.py, Dockerfile.service, Makefile: dockerized Flask service for HTTP summarization
– examples/, colab/: sample usages and Jupyter notebook demo
– .github/workflows: CI for tests and publishing
• summarizer/ package:
– bert.py, sbert.py: wrappers around HuggingFace model loading and embedding extraction
– summary_processor.py: clustering and sentence selection logic
– text_processors/: sentence handlers and coreference resolution (neuralcoref + spaCy)
– transformer_embeddings/: encapsulates embedding extraction per model type
– util.py, cluster_features.py: helper functions for clustering and feature calculations
• tests/: pytest modules covering core pipeline components
README for additional insights
• Describes algorithm: sentence embedding → clustering → choose nearest‐to‐centroid sentences
• Mentions external dependencies: HuggingFace Transformers, Sentence Transformers, neuralcoref, spaCy
• Outlines the HTTP API: POST /summarize with query params for ratio, min_length, max_length