How sundar-0311/code-plagrarism works
This repository is a programming-assignment similarity and plagiarism checker. Its main user flow is to upload Python files in the Streamlit app, preprocess and compare them with multiple signals, combine scores into a verdict, and inspect pair evidence and suspicion clusters. It also supports precomputed dataset runs, calibrated fallback scoring, and evaluation with corrected labels and fold-local tuning. The README documents several pipeline stages whose source excerpts were not provided; those links are cited to the README rather than implying their implementations were inspected.
Main entry points: app.py Streamlit app · composite.py Score calibration · token_similarity.py Jaccard signal · winnowing.py Winnowing signal
Components 17
User workflow
- Streamlit app
app.py
Similarity signals
- AST structure
pipeline/3_structural_similarity.py - Token n-grams
similarity_engine/tokenizer.py - Jaccard signal
similarity_engine/token_similarity.py - Winnowing signal
similarity_engine/winnowing.py - Pair/token loader
similarity_engine/loader.py - Pair comparison
similarity_engine/compare.py - Data-flow extraction
similarity_engine/dfg_extractor.py - Embedding signals
similarity_engine/embedding_similarity.py
Scoring and analysis
- Score calibration
similarity_engine/composite.py - Cluster and diff
pipeline/4_clustering_and_diff.py
Dataset pipeline
- Source preprocessing
pipeline/1_preprocess.py - Pair dataset
pipeline/2_build_pairs_csv.py - Assignment corpus
dataset - Preprocessed corpus
dataset_preprocessed - Processed outputs
preprocessed_output
Other components
- Analyst
Connections 24
- Analyst → Streamlit app: uploads files
- Streamlit app → Source preprocessing: preprocesses source
app.py - Assignment corpus → Source preprocessing: provides source
- Source preprocessing → Preprocessed corpus: writes normalized code
- Preprocessed corpus → Pair dataset: supplies pair paths
- Pair dataset → AST structure: provides labeled pairs
- Streamlit app → Token n-grams: tokenizes source
app.py - Streamlit app → Jaccard signal: computes signal
app.py - Streamlit app → Winnowing signal: computes signal
app.py - Jaccard signal → Token n-grams: uses n-grams
similarity_engine/token_similarity.py - Winnowing signal → Token n-grams: uses n-grams
similarity_engine/winnowing.py - Pair comparison → Pair/token loader: loads tokens
similarity_engine/compare.py - Pair comparison → Jaccard signal: computes signal
similarity_engine/compare.py - Pair comparison → Winnowing signal: computes signal
similarity_engine/compare.py - Streamlit app → Embedding signals: optionally embeds files
app.py - Embedding signals → Data-flow extraction: extracts data flow
similarity_engine/embedding_similarity.py - Streamlit app → Score calibration: scores pairs
app.py - Score calibration → AST structure: combines signal
similarity_engine/composite.py - Score calibration → Jaccard signal: combines signal
similarity_engine/composite.py - Score calibration → Winnowing signal: combines signal
similarity_engine/composite.py - Score calibration → Embedding signals: combines signal
similarity_engine/composite.py - Streamlit app → Cluster and diff: shows clusters and diffs
app.py - Assignment corpus → Preprocessed corpus: canonicalizes corpus
- Processed outputs → Streamlit app: feeds saved run