How prosusai/finbert works
Project Type and Purpose
• This is a Python‐based machine‐learning pipeline for financial‐domain sentiment analysis.
• It consists of two main workflows:
– Offline training (notebook, dataset scripts)
– Runtime inference (CLI/predict script)
• Core functionality is provided by the finbert Python package, which wraps Hugging Face’s BERT model for classification.
File Structure
Top-level artifacts and their roles:
• environment.yml, Dockerfile → Deployment/configuration (Conda env, container)
• config.json → Model/config parameters for .from_pretrained(...)
• finbert/finbert.py, finbert/utils.py → Core library: model loading, tokenization, inference APIs
• notebooks/finbert_training.ipynb → Training pipeline (data → model)
• scripts/datasets.py → Data ingestion & splitting (build train/val/test CSVs)
• scripts/predict.py → CLI for batch‐mode inference producing CSV outputs
• main.py → (Optional) Top‐level entry point (could orchestrate tasks)
• examples.csv, test.txt → Sample inputs/outputs
Extract Architectural Insights from the README
• Two model types:
– Language model fine‐tuning on TRC2
– Sentiment classifier on Financial PhraseBank
• Training parameters (epochs, batch size, learning rate, unfreeze technique)
• Models are stored on disk under models/... with accompanying config.json
• Inference script loads model via .from_pretrained() from local directory
1. Data Sources
– External datasets (Reuters TRC2 [offline], FinancialPhraseBank)
– Raw text files (test.txt, user-provided)
2. Data Preparation
– scripts/datasets.py
– Outputs: data/sentiment_data/{train.csv,validation.csv,test.csv}
3. Training Pipeline
– notebooks/finbert_training.ipynb (calls FinBERT API, PyTorch training loop)
– Produces model artifacts under models/classifier_model/finbert-sentiment
4. Model Registry/Storage
– Local filesystem (models/...) or Hugging Face model hub
5. Core Library
– finbert package:
• ModelLoader (loads BERT via pytorch_pretrained_bert)
• Tokenizer, inference code, utilities
6. Inference/Serving
– scripts/predict.py (CLI)
– Possibly main.py if used as entrypoint/CLI wrapper
– Outputs: CSV with sentences + softmax probabilities + sentiment score
7. Deployment/Environment
– Conda environment (environment.yml)
– Docker container (Dockerfile, .dockerignore)
• Raw Dataset → Data Preparation Script → Prepared CSV files
• Prepared CSV → Training Notebook → Model Artifacts
• Model Artifacts + config.json → Core Library (.from_pretrained)
• Core Library + Input Text → Inference Script → Output CSV
• Dockerfile & environment.yml → Provision runtime/training environment
c. Architectural Patterns and Principles
• Pipeline pattern: clear separation between data ingestion, training, inference
• Modular design: core logic in finbert package, scripts as thin wrappers
• Configuration‐driven: parameters in config.json and notebook variables
• Reproducibility: Docker + Conda environment
d. Key Technologies & Frameworks
• Python 3.x, Conda
• PyTorch via pytorch_pretrained_bert (soon transformers)
• Hugging Face model hub (external storage)
• Jupyter Notebook for interactive training
• CLI scripts for batch workflows