How ahmadrezaomidvar/news_classification works
Main Components
a. Data Pipeline Layer:
• Input Data Source (data/data.csv)
• Data Cleaning Process (data/clean_data.csv)
• Data Preprocessing Module (news/dataset.py)
b. Model Architecture Layer:
• Traditional ML Models (news/benchmark.py)
• Logistic Regression
• Random Forest
• Support Vector Machine
• Deep Learning Model (news/lstm.py)
• LSTM Architecture
c. Training Pipeline Components:
• Cross Validation Module (news/cross_validation.py)
• Data Balancing Techniques
• SMOTE (Oversampling)
• Undersampling
• Class Weighting
d. Evaluation System:
• Metrics Collection
• F1 Score
• Accuracy
• Precision
• Recall
• Results Storage (results/ directory)
Flow and Relationships
1. Raw Data → Data Cleaning → Preprocessed Data
2. Preprocessed Data splits into three parallel paths:
• Traditional ML Pipeline
• LSTM Pipeline
• Cross-Validation Pipeline
3. Each path should show the various techniques (SMOTE, undersampling, weighted loss)
4. All paths converge at the Evaluation System
Important Design Elements
• Rectangles: Data storage/files
• Rounded rectangles: Processing modules
• Diamonds: Decision points/model selection
• Blue: Data-related components
• Green: ML/DL models
• Orange: Evaluation components
• Gray: Utility functions and configurations
Technical Details
• Configuration Management (news/configs/config.yaml)
• Environment Management (conda/environment.yml)
• Testing Framework (tests/ directory structure)
• Utility Functions (news/utils.py)
Implementation Details
• Maintain a hierarchical layout from top (data ingestion) to bottom (evaluation)
• Version control and environment management components