How krt3xp/teste2projeto works
Project type and purpose
• This is a standalone machine-learning-driven text-classification pipeline (COMLProcessing) that reads pre-labeled news articles, trains multiple classification pipelines, evaluates them, picks the best model, and then applies that model to unlabeled news—storing all results in MongoDB.
File structure
• Top-level scripts and modules:
• main.py: orchestration entry point
• prompt.py: (likely handles user prompts or dynamic parameter injection)
• requirements.txt: Python dependencies
• data/ directory: data ingestion & transformation
• database.py: opens SSH tunnel & connects to remote DB
• preprocessing.py: text cleaning and normalization
• models/ directory: feature engineering & modeling
• vectorizers.py: TF-IDF or custom vectorizers
• pipelines.py: definitions of sklearn or similar classification pipelines
• evaluation/ directory: model scoring & selection
• evaluation.py: cross-validation, metric computation, best-model selector
• Other:
• caracteristicas_extraidas.csv: sample or cached features
• README.md: setup & workflow description
README for additional insights
• Workflow:
1. Read credentials from config.yaml
2. Via SSH + bind to MongoDB
3. Pull “accepted” (manually labeled) news → run feature pipelines → store all intermediate & final scores in MongoDB collection classification_pipelines
4. Pick best-performing pipeline → classify unlabeled news → write predictions back to MongoDB