How devashi-choudhary/automated-chatbot-for-college-queries works
Project Type & Purpose
• This is an NLP research/tool repository for an automated chatbot that answers college-related questions by computing semantic similarity between user queries and a private QA dataset.
• There is no web frontend or service layer – it’s a CLI/python-script toolkit that loads data, cleans it, vectorizes text with multiple embedding models, computes similarity scores, and returns the most relevant answer.
File Structure Analysis
– Top-level scripts each implement one similarity approach:
• Cosine_Similarity.py
• doc2vec.py
• fasttext+wmdistance.py
• glove.py
• word2vec+wm distance.py
– README.md holds project description, dataset details, dependencies, and execution instructions.
– No subdirectories or service/config files – it’s a single-module pipeline per script.
Key Architectural Insights from README
– Data Collection & Cleaning stages precede modeling.
– Each script loads the same cleaned Q-A dataset.
– Multiple NLP libraries (NLTK, gensim, glove, pyemd, scikit-learn) supply tokenization, embedding, and distance functions.
– The outcome of each script is a similarity ranking between an input question and candidate answers.
a. Main Components
1. Input Interface (CLI argument or prompt)
2. Data Loader & Preprocessor
– Cleans text (removes slang, non-alphanumeric chars)
3. Embedding Modules
– Word2Vec
– GloVe
– FastText
– Doc2Vec
– TF-IDF Vectorizer
4. Similarity Computation
– Cosine similarity
– Word Mover’s Distance (via pyemd)
5. Result Aggregator & Selector
6. Output (console print of best answer)
7. External: Private dataset store
b. Relationships & Data Flow
– Loader → Preprocessor → each Embedding Module
– Each Embedding Module → Similarity Computation
– All similarity scores → Aggregator → Output
– The dataset store → Loader
c. Architectural Patterns & Principles
– Modular “one-script-per-model” design for easy comparison/extension
– Shared preprocessor to enforce consistency across models
– Separation of concerns: data, embedding, similarity, aggregation
d. Technologies & Libraries
– Python 3.6+
– NLTK (tokenization, stopwords)
– gensim (Word2Vec, Doc2Vec, FastText)
– glove-python or similar wrapper
– scikit-learn (TfidfVectorizer)
– pyemd (Word Mover’s Distance)
– numpy, pandas for data handling