How sriramanganesan/offensive-comments-detection-for-tamil-english-in-social-media-using-ann works
Main Components
• Data Sources: Begin with the raw dataset files (TSV files) and the resulting CSV files produced for both preprocessed and non-preprocessed data. These represent your input data.
• Data Preprocessing Module: Represent the “Dataset preprocessing.ipynb” notebook. This is responsible for extracting and transforming raw TSV data into usable CSV data.
• Model Training & Evaluation Module:
• ANN Component: Focus on the “ANN_with_4_hidden_layer.ipynb” as the core ANN architecture that achieved the highest accuracy.
• Comparison Classifiers Module: Include a module symbolizing “Comparison classifiers.ipynb”, where several other standard classifiers (LR, NB, SVM, etc.) are trained and evaluated.
• mBERT Module: Represent the experiments with mBERT through its two separate notebook versions (non-preprocessed and preprocessed) to highlight its performance.
• Results & Evaluation: A final evaluation block, where accuracy metrics and performance comparisons are gathered (this is implicit in the repository documentation and the notebooks).
Relationships and Data Flow
• One going directly to the ANN Model component (to show the use of both preprocessed and non-preprocessed data).
• Another leading to the module where alternative classifiers are trained and benchmarked.
• Additionally, include a branch to the mBERT module as it handles the data in its own way.
• Both the ANN and the comparison classifiers (including mBERT) feed into an Evaluation component that aggregates performance metrics.
Architectural Patterns and Design Principles
• Pipeline/Modular Architecture: Emphasize the separation of concerns by clearly isolating each stage (data ingestion, preprocessing, model training, evaluation).
• Repeatable and Extensible: The structure should allow adding more classifiers or preprocessing steps in the future, so indicate how modules could be extended.
Relevant Technologies and Frameworks
• Jupyter Notebooks are used as the development environment.
• CSV/TSV file handling illustrates the data ingestion technology.