How pierrepo/mdner works
Project type and purpose
• Project type: A full-stack ML/NER pipeline in Python, focused on named-entity recognition (NER) for molecular dynamics (MD) literature.
• Purpose: Fetch raw MD text data, preprocess and annotate it (automatically and manually), optionally paraphrase for data augmentation, train/evaluate an NER model (spaCy or Transformer-based), analyze results, and expose a Streamlit web app for inference.
File structure
Top-level directories/files and their roles:
• binder/ – Conda environment definitions for reproducible runs
• scripts/
– generate_annotation.py: fetches and filters texts, emits JSON/text for annotation
– entity_annotator.py: Streamlit app for manual annotation
– mdner.py & mdner_app.py: training script (CLI) and web-app entrypoint (inference)
– other scripts: run_llm_annotation.py, run_qc_analysis.py, run_scoring_analysis.py, generate_plots_stats.py, mdner (batch build)
• annotations/: human-annotated JSON/text files (input to training)
• ai_annotations/: LLM‐generated annotation experiments
• prompt_templates/: few‐shot prompt files for LLM annotation
• results/
– models/: persisted spaCy/Transformer NER models
– outputs/: evaluation data and metrics CSV
• notebooks/: Jupyter notebooks for exploratory data analysis, QC, visualization
• docs/: markdown for annotation schema, improvement guidelines
• assets/: static images (e.g., annotator GIF)
• README.md & pyproject.toml: project description and dependencies
README for additional insights
• Workflow stages documented: annotation generation, manual annotation, paraphrasing, model training/evaluation, and web‐app usage.
• External dependencies: spaCy, HuggingFace models (mbart, Pegasus, etc.), CUDA/GPU, Streamlit.
Main components
a) Data Sources
– External MD dataset repository (Zenodo/OSF)
b) Preprocessing & Annotation Pipeline
– generate_annotation.py (text selection, threshold filtering)
– LLM paraphrasing service (HuggingFace API)
– Manual annotation service (Streamlit entity_annotator)
– annotations/ folder (JSON & TXT)
c) Data Store
– annotations/ (human) and ai_annotations/ (LLM)
– prompt_templates/
d) Training Pipeline
– mdner.py CLI (spaCy or Transformer)
– GPU/CPU compute environment (binder/app.yml)
– results/models/ (model artifacts)
e) Evaluation & Analysis
– run_scoring_analysis.py & run_qc_analysis.py
– generate_plots_stats.py
– notebooks/ for interactive EDA and stats
– results/outputs/ (eval_data, results.csv)
f) Deployment Layer
– Streamlit app (scripts/mdner_app.py)
– Web UI for live NER inference
– Model loading from results/models/
Architectural patterns & design principles
• Modular pipeline: distinct scripts for each stage.
• Data-centric storage: annotations and results persisted in folders.
• Reproducibility via Binder/Conda environments (binder/*).
• Separation of concerns:
• Data generation/annotation (scripts/generate_annotation.py, entity_annotator.py)
• Model training (scripts/mdner.py)
• Evaluation/analysis (scripts/run_*.py + notebooks)
• Deployment (scripts/mdner_app.py + Streamlit)