# sundar-0311/code-plagrarism architecture

> Architecture diagram of the GitHub repository sundar-0311/code-plagrarism, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sundar-0311/code-plagrarism
- Repository: https://github.com/sundar-0311/code-plagrarism

## Overview

This repository is a programming-assignment similarity and plagiarism checker. Its main user flow is to upload Python files in the Streamlit app, preprocess and compare them with multiple signals, combine scores into a verdict, and inspect pair evidence and suspicion clusters. It also supports precomputed dataset runs, calibrated fallback scoring, and evaluation with corrected labels and fold-local tuning. The README documents several pipeline stages whose source excerpts were not provided; those links are cited to the README rather than implying their implementations were inspected.

Main entry points: [`app.py`](https://github.com/sundar-0311/code-plagrarism/blob/main/app.py) (Streamlit app), [`similarity_engine/composite.py`](https://github.com/sundar-0311/code-plagrarism/blob/main/similarity_engine/composite.py) (Score calibration), [`similarity_engine/token_similarity.py`](https://github.com/sundar-0311/code-plagrarism/blob/main/similarity_engine/token_similarity.py) (Jaccard signal), [`similarity_engine/winnowing.py`](https://github.com/sundar-0311/code-plagrarism/blob/main/similarity_engine/winnowing.py) (Winnowing signal)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_product["User workflow"]
  node_app["Streamlit app<br/>[app.py]"]
end

subgraph group_signals["Similarity signals"]
  node_structural["AST structure"]
  node_tokens["Token n-grams<br/>[tokenizer.py]"]
  node_jaccard["Jaccard signal"]
  node_winnowing["Winnowing signal<br/>[winnowing.py]"]
  node_loader["Pair/token loader<br/>[loader.py]"]
  node_compare["Pair comparison<br/>[compare.py]"]
  node_dfg["Data-flow extraction<br/>[dfg_extractor.py]"]
  node_embeddings["Embedding signals"]
end

subgraph group_calibration["Scoring and analysis"]
  node_fusion["Score calibration<br/>[composite.py]"]
  node_clusters["Cluster and diff"]
end

subgraph group_corpus["Dataset pipeline"]
  node_preprocess["Source preprocessing<br/>[1_preprocess.py]"]
  node_pairdata["Pair dataset"]
  node_dataset[("Assignment corpus")]
  node_canonical[("Preprocessed corpus")]
  node_outputs[("Processed outputs")]
end

node_user(("Analyst"))

node_user -->|"uploads files"| node_app
node_app -->|"preprocesses source"| node_preprocess
node_dataset -.->|"provides source"| node_preprocess
node_preprocess -->|"writes normalized code"| node_canonical
node_canonical -.->|"supplies pair paths"| node_pairdata
node_pairdata -->|"provides labeled pairs"| node_structural
node_app -->|"tokenizes source"| node_tokens
node_app -->|"computes signal"| node_jaccard
node_app -->|"computes signal"| node_winnowing
node_jaccard -->|"uses n-grams"| node_tokens
node_winnowing -->|"uses n-grams"| node_tokens
node_compare -->|"loads tokens"| node_loader
node_compare -->|"computes signal"| node_jaccard
node_compare -->|"computes signal"| node_winnowing
node_app -->|"optionally embeds files"| node_embeddings
node_embeddings -->|"extracts data flow"| node_dfg
node_app -->|"scores pairs"| node_fusion
node_fusion -->|"combines signal"| node_structural
node_fusion -->|"combines signal"| node_jaccard
node_fusion -->|"combines signal"| node_winnowing
node_fusion -->|"combines signal"| node_embeddings
node_app -->|"shows clusters and diffs"| node_clusters
node_dataset -->|"canonicalizes corpus"| node_canonical
node_outputs -.->|"feeds saved run"| node_app

click node_app "https://github.com/sundar-0311/code-plagrarism/blob/main/app.py"
click node_preprocess "https://github.com/sundar-0311/code-plagrarism/blob/main/pipeline/1_preprocess.py"
click node_pairdata "https://github.com/sundar-0311/code-plagrarism/blob/main/pipeline/2_build_pairs_csv.py"
click node_structural "https://github.com/sundar-0311/code-plagrarism/blob/main/pipeline/3_structural_similarity.py"
click node_tokens "https://github.com/sundar-0311/code-plagrarism/blob/main/similarity_engine/tokenizer.py"
click node_jaccard "https://github.com/sundar-0311/code-plagrarism/blob/main/similarity_engine/token_similarity.py"
click node_winnowing "https://github.com/sundar-0311/code-plagrarism/blob/main/similarity_engine/winnowing.py"
click node_loader "https://github.com/sundar-0311/code-plagrarism/blob/main/similarity_engine/loader.py"
click node_compare "https://github.com/sundar-0311/code-plagrarism/blob/main/similarity_engine/compare.py"
click node_dfg "https://github.com/sundar-0311/code-plagrarism/blob/main/similarity_engine/dfg_extractor.py"
click node_embeddings "https://github.com/sundar-0311/code-plagrarism/blob/main/similarity_engine/embedding_similarity.py"
click node_fusion "https://github.com/sundar-0311/code-plagrarism/blob/main/similarity_engine/composite.py"
click node_clusters "https://github.com/sundar-0311/code-plagrarism/blob/main/pipeline/4_clustering_and_diff.py"
click node_dataset "https://github.com/sundar-0311/code-plagrarism/tree/main/dataset"
click node_canonical "https://github.com/sundar-0311/code-plagrarism/tree/main/dataset_preprocessed"
click node_outputs "https://github.com/sundar-0311/code-plagrarism/tree/main/preprocessed_output"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app toneBlue
class node_structural,node_tokens,node_jaccard,node_winnowing,node_loader,node_compare,node_dfg,node_embeddings toneAmber
class node_fusion,node_clusters toneMint
class node_preprocess,node_pairdata,node_dataset,node_canonical,node_outputs toneRose
class node_user toneIndigo
```

## Components

### User workflow

- [Streamlit app](https://github.com/sundar-0311/code-plagrarism/blob/main/app.py): `app.py`

### Similarity signals

- [AST structure](https://github.com/sundar-0311/code-plagrarism/blob/main/pipeline/3_structural_similarity.py): `pipeline/3_structural_similarity.py`
- [Token n-grams](https://github.com/sundar-0311/code-plagrarism/blob/main/similarity_engine/tokenizer.py): `similarity_engine/tokenizer.py`
- [Jaccard signal](https://github.com/sundar-0311/code-plagrarism/blob/main/similarity_engine/token_similarity.py): `similarity_engine/token_similarity.py`
- [Winnowing signal](https://github.com/sundar-0311/code-plagrarism/blob/main/similarity_engine/winnowing.py): `similarity_engine/winnowing.py`
- [Pair/token loader](https://github.com/sundar-0311/code-plagrarism/blob/main/similarity_engine/loader.py): `similarity_engine/loader.py`
- [Pair comparison](https://github.com/sundar-0311/code-plagrarism/blob/main/similarity_engine/compare.py): `similarity_engine/compare.py`
- [Data-flow extraction](https://github.com/sundar-0311/code-plagrarism/blob/main/similarity_engine/dfg_extractor.py): `similarity_engine/dfg_extractor.py`
- [Embedding signals](https://github.com/sundar-0311/code-plagrarism/blob/main/similarity_engine/embedding_similarity.py): `similarity_engine/embedding_similarity.py`

### Scoring and analysis

- [Score calibration](https://github.com/sundar-0311/code-plagrarism/blob/main/similarity_engine/composite.py): `similarity_engine/composite.py`
- [Cluster and diff](https://github.com/sundar-0311/code-plagrarism/blob/main/pipeline/4_clustering_and_diff.py): `pipeline/4_clustering_and_diff.py`

### Dataset pipeline

- [Source preprocessing](https://github.com/sundar-0311/code-plagrarism/blob/main/pipeline/1_preprocess.py): `pipeline/1_preprocess.py`
- [Pair dataset](https://github.com/sundar-0311/code-plagrarism/blob/main/pipeline/2_build_pairs_csv.py): `pipeline/2_build_pairs_csv.py`
- [Assignment corpus](https://github.com/sundar-0311/code-plagrarism/tree/main/dataset): `dataset`
- [Preprocessed corpus](https://github.com/sundar-0311/code-plagrarism/tree/main/dataset_preprocessed): `dataset_preprocessed`
- [Processed outputs](https://github.com/sundar-0311/code-plagrarism/tree/main/preprocessed_output): `preprocessed_output`

### Other components

- **Analyst**

## Connections

- Analyst → Streamlit app: uploads files
- Streamlit app → Source preprocessing: preprocesses source (evidence: `app.py`)
- Assignment corpus → Source preprocessing: provides source
- Source preprocessing → Preprocessed corpus: writes normalized code
- Preprocessed corpus → Pair dataset: supplies pair paths
- Pair dataset → AST structure: provides labeled pairs
- Streamlit app → Token n-grams: tokenizes source (evidence: `app.py`)
- Streamlit app → Jaccard signal: computes signal (evidence: `app.py`)
- Streamlit app → Winnowing signal: computes signal (evidence: `app.py`)
- Jaccard signal → Token n-grams: uses n-grams (evidence: `similarity_engine/token_similarity.py`)
- Winnowing signal → Token n-grams: uses n-grams (evidence: `similarity_engine/winnowing.py`)
- Pair comparison → Pair/token loader: loads tokens (evidence: `similarity_engine/compare.py`)
- Pair comparison → Jaccard signal: computes signal (evidence: `similarity_engine/compare.py`)
- Pair comparison → Winnowing signal: computes signal (evidence: `similarity_engine/compare.py`)
- Streamlit app → Embedding signals: optionally embeds files (evidence: `app.py`)
- Embedding signals → Data-flow extraction: extracts data flow (evidence: `similarity_engine/embedding_similarity.py`)
- Streamlit app → Score calibration: scores pairs (evidence: `app.py`)
- Score calibration → AST structure: combines signal (evidence: `similarity_engine/composite.py`)
- Score calibration → Jaccard signal: combines signal (evidence: `similarity_engine/composite.py`)
- Score calibration → Winnowing signal: combines signal (evidence: `similarity_engine/composite.py`)
- Score calibration → Embedding signals: combines signal (evidence: `similarity_engine/composite.py`)
- Streamlit app → Cluster and diff: shows clusters and diffs (evidence: `app.py`)
- Assignment corpus → Preprocessed corpus: canonicalizes corpus
- Processed outputs → Streamlit app: feeds saved run
