How rohitsharma0719/llama-3.2-sentiment-analysis-for-financial-news works
The project is a focused sentiment analysis application for financial news headlines, leveraging a fine-tuned LLaMA-3.2-1B model with LoRA-based parameter-efficient fine-tuning. Given the simple file structure (a code directory, a Jupyter Notebook, and the README), we know that the main focus is on model fine-tuning, data preprocessing, and evaluation.
Main Components
• Data Input & Preprocessing:
• Data Source: The project uses a CSV file (all-data.csv) from Kaggle.
• Preprocessing Module: Steps that split data into train, test, and evaluation sets, and generate the prompts framing the sentiment classification task.
• Model Component:
• Base Model: LLaMA-3.2-1B.
• Fine-Tuning Pipeline: Implementation of LoRA-based fine-tuning.
• Quantization: 4-bit quantization with bitsandbytes.
• Configuration & Hyperparameters: Details like epochs (3), batch size (1), gradient accumulation (8), learning rate (2e-4 with a cosine scheduler), and the optimizer (Paged AdamW).
• Evaluation:
• Metrics & Reporting: Accuracy report, precision, recall, and f1-scores for each sentiment category.
• Notebook & Code Environment:
• Jupyter Notebook (Llama-3.2-1B_code.ipynb) as an interactive development and exploratory environment.
• Code Directory (Llama-3.2-1B_code) containing the implementation scripts and modules.
Relationships and Interactions
• Data Flow: Raw financial news headlines (CSV) are first ingested into the Data Preprocessing module.
• Preprocessed data is then fed into the Training module where the base model is fine-tuned using the LoRA approach.
• Output from model training flows into the Evaluation module, which measures performance improvements.
• Feedback Loop: Evaluation outcomes help in deciding further improvements or model tuning.
Architectural Patterns & Design Principles
• Modular Design: Separation of concerns (data processing, model fine-tuning, and evaluation).
• Pipeline Architecture: A clear sequential flow from data preprocessing to training and evaluation.
• Efficiency: Using parameter-efficient fine-tuning (LoRA) and quantization (4-bit with bitsandbytes) to optimize performance.
• Experimentation Friendly: Use of Jupyter Notebook facilitates iterative development and rapid prototyping.
Relevant Technologies, Frameworks, and Libraries
• Python Frameworks: torch, transformers, and datasets for model operation.
• LoRA (peft library) for the fine-tuning technique.
• Data science libraries: numpy, pandas, and scikit-learn.
• Optimization Library: Paged AdamW.
• Others: tqdm for progress tracking and bitsandbytes for quantization.
• Component Labels:
• Data Source (all-data.csv from Kaggle)
• Data Preprocessing Module
• Model Component (Base Model: LLaMA-3.2-1B)
• Fine-tuning (LoRA integration, quantization with bitsandbytes)
• Evaluation Module (with performance metrics)
• Notebook Environment (Jupyter Notebook)
• Sequential flow: Data Ingestion → Preprocessing → Training/Fine-Tuning → Evaluation.
• Visual Differentiators:
• Annotations: