How kalebu/plagiarism-checker-python works
Project Type and Purpose
• Type: A CLI text-similarity tool (not a web or full-stack app).
• Purpose: Read all “.txt” files in a directory, convert each document into a numeric vector, compute pairwise cosine similarities, and output the scores.
File Structure
• Top-level files:
– app.py: main script
– requirements.txt: Python dependencies (e.g., numpy, scikit-learn or custom code)
– README.md: documentation
• Sample documents: fatma.txt, john.txt, juma.txt
• pictures/: assets for README only (not part of runtime)
Key Components
a. Input Loader
– Reads all .txt files from the project directory
– Outputs raw text strings
b. Preprocessing Module
– (Optional) Tokenization, case‐folding, stop‐word removal
– Outputs cleaned token lists or raw text if no preprocessing
c. Vectorization Module
– Transforms text into numeric arrays (e.g., term‐frequency vectors or TF-IDF using sklearn’s TfidfVectorizer)
– Outputs document‐vectors matrix
d. Similarity Engine
– Computes cosine similarity between every pair of document vectors
– Outputs a similarity score matrix or list of tuples
e. Output Module
– Formats and prints results to the console, e.g.:
(‘john.txt’, ‘juma.txt’, 0.5466)
f. External Libraries
– Python runtime
– numpy, scikit-learn (or pure-Python vector math code)
Relationships and Data Flows
– Filesystem → Input Loader
– Input Loader → Preprocessing
– Preprocessing → Vectorization
– Vectorization → Similarity Engine
– Similarity Engine → Output Module → Console
Architectural Patterns and Design Principles
• Single-responsibility: each module does one thing (load, preprocess, vectorize, compute, output).
• Pipelines/data flow: linear sequence from raw input to final output.
• Extensibility point: the vectorization block can be swapped out for the “Pysimilar” library or another algorithm.