# adityajaiswal440/india-runs-hackathon architecture

> Architecture diagram of the GitHub repository adityajaiswal440/india-runs-hackathon, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/adityajaiswal440/india-runs-hackathon
- Repository: https://github.com/adityajaiswal440/india-runs-hackathon

## Overview

1. **What this repository is**
   - This is a **batch candidate-ranking pipeline**, not a long-running service.
   - It takes candidate profile data plus a job-description target, computes offline features, ranks candidates deterministically, and writes a submission file (`era.csv`).

2. **Core architectural boundary: precompute vs. rank**
   - The repo is split into two phases:
     - **Precomputation**: stream and normalize raw candidate data, extract features, build embeddings, and persist artifacts.
     - **Online ranking**: load those artifacts, combine scores, apply penalties/multipliers, and emit final ranked output.
   - This separation is central to the architecture because it keeps runtime memory and latency low.

3. **Data ingress and storage**
   - Raw inputs live under `data/raw/`, with intermediate workspace under `data/interim/`.
   - Precomputed outputs are stored in `artifacts/`, including candidate IDs, dense embeddings, and feature matrices.
   - The code expects offline local files only; there is no external network dependency in the ranking flow.

4. **Ingestion and normalization layer**
   - `src/data/loader.py` and the helpers in `src/utils/text_cleaner.py` and `src/challenge/text_match.py` handle JSONL loading, tokenization, and text normalization.
   - This layer is responsible for turning noisy profile text into stable, reusable inputs for scoring.

5. **Feature extraction subsystem**
   - `src/features/embedder.py` and `src/features/heuristic_extractor.py` build the candidate feature set.
   - Feature extraction produces both:
     - **semantic features** from embeddings and text similarity, and
     - **structural/heuristic features** from resume content, experience patterns, and metadata.
   - `src/challenge/features.py` wraps these into a candidate index used by later scoring stages.

6. **Domain-scoring rules live in `src/challenge/`**
   - This package contains the job-description schema, scoring heuristics, availability logic, company-year checks, and assessment handling.
   - Key modules include:
     - `jd_config.py` for skill/title taxonomies and thresholds,
     - `availability.py` for recency and engagement logic,
     - `assessment.py` for verified-skill weighting,
     - `company_matrix.py` and `career_blurb.py` for consistency checks and duplicate/template detection.
   - This is the main “business logic” boundary of the system.

7. **Semantic retrieval path**
   - `src/challenge/embeddings.py`, `semantic.py`, and `text_match.py` implement the semantic side of matching.
   - The repository uses offline embedding-based similarity plus lexical/phrase matching to capture both broad fit and exact term overlap.
   - `src/challenge/rerank.py` is an optional reranking hook, while `redrob_ranker.py` is the primary orchestrator.

8. **Anti-gaming / fraud detection boundary**
   - `src/challenge/honeypot.py` is a dedicated fraud/quality filter.
   - It checks for structural anomalies such as impossible dates, unrealistic tenure, template-like resumes, and other “honeypot” conditions.
   - This subsystem acts as a gate before candidates can rise into the top ranks.

9. **Ranking assembly**
   - `src/challenge/redrob_ranker.py` combines the component scores into the final ranking signal.
   - `src/ranking/heuristic_scorer.py` performs the heuristic weighting/fusion, and `src/challenge/rerank.py` can optionally add a second semantic pass.
   - The ranking order is deterministic, with explicit tie-breaking by candidate ID.

10. **Reasoning and output generation**
    - `src/ranking/reasoning_gen.py` produces grounded explanations for ranked candidates.
    - `src/pipeline.py` is the top-level CLI entrypoint that runs the whole flow and writes the submission output.
    - `run_pipeline.py` is the root wrapper used for invoking the pipeline from the repository root.

11. **Operational tooling and verification**
    - `scripts/precompute.py` prepares offline artifacts; the audit/sensitivity scripts in `scripts/` check submission quality and robustness.
    - `src/validate_submission.py` and the `tests/` suite enforce correctness, determinism, and output-shape expectations.
    - The tests cover data loading, embedding extraction, heuristic extraction, precompute behavior, and ranking logic.

12. **Configuration and runtime controls**
    - `src/config/defaults.py` and `src/utils/config.py` centralize runtime settings and fixed reference values.
    - `src/utils/logger.py` supports consistent pipeline logging.
    - The repo is designed around CPU-only execution, offline operation, and reproducible scoring.

13. **How to draw the architecture**
    - A clear diagram would show:
      - **Raw data inputs** → **load/clean** → **feature extraction** → **artifact storage**
      - then **artifacts** → **ranking/orchestration** → **fraud filtering + scoring fusion** → **reasoning generation** → **`era.csv` output**
    - Surrounding that path are the config layer, audit/validation tools, and test suite, which enforce determinism and guardrails.

Main entry points: [`src/challenge/redrob_ranker.py`](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/challenge/redrob_ranker.py) (Redrob ranker), [`src/data/loader.py`](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/data/loader.py) (Loader), [`src/pipeline.py`](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/pipeline.py) (Pipeline), [`src/challenge/features.py`](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/challenge/features.py) (Feature index)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_inputs["Precompute phase"]
  node_n_raw[("Raw data<br/>input dataset")]
  node_n_loader["Loader<br/>ingestion<br/>[loader.py]"]
  node_n_clean["Text clean<br/>normalization<br/>[text_cleaner.py]"]
  node_n_match["Text match<br/>lexical matching<br/>[text_match.py]"]
  node_n_embedder["Embedder<br/>embedding features<br/>[embedder.py]"]
  node_n_heur_feat["Heuristic feats<br/>structural features"]
  node_n_feat_index["Feature index<br/>candidate index<br/>[features.py]"]
  node_n_artifacts[("Artifacts<br/>offline store<br/>[features.npy]")]
end

subgraph group_group_rank["Ranking phase"]
  node_n_pipeline["Pipeline<br/>CLI orchestration<br/>[pipeline.py]"]
  node_n_run["Runner<br/>wrapper CLI<br/>[run_pipeline.py]"]
  node_n_ranker["Redrob ranker<br/>ranking orchestrator<br/>[redrob_ranker.py]"]
  node_n_honeypot["Honeypot<br/>fraud filter<br/>[honeypot.py]"]
  node_n_rules["Domain rules<br/>business logic<br/>[jd_config.py]"]
  node_n_semantic["Semantic match<br/>similarity scoring<br/>[semantic.py]"]
  node_n_rerank["Rerank<br/>[rerank.py]"]
  node_n_heur_score["Heuristic score<br/>score fusion"]
  node_n_reason["Reasoning<br/>explanations<br/>[reasoning_gen.py]"]
end

subgraph group_group_ops["Ops and guardrails"]
  node_n_paths["Data paths<br/>path policy<br/>[data_paths.py]"]
  node_n_defaults["Defaults<br/>runtime config<br/>[defaults.py]"]
  node_n_config["Config utils<br/>config helper<br/>[config.py]"]
  node_n_validate["Validate<br/>submission checks"]
  node_n_scripts["Scripts<br/>ops tooling<br/>[precompute.py]"]
  node_n_tests["Tests<br/>verification suite<br/>[test_ranker.py]"]
end

node_n_raw -->|"load"| node_n_loader
node_n_loader -->|"normalize"| node_n_clean
node_n_clean -->|"lexical prep"| node_n_match
node_n_clean -->|"semantic prep"| node_n_embedder
node_n_loader -->|"structure prep"| node_n_heur_feat
node_n_match -->|"signals"| node_n_feat_index
node_n_embedder -->|"embeddings"| node_n_feat_index
node_n_heur_feat -->|"heuristics"| node_n_feat_index
node_n_feat_index -->|"persist"| node_n_artifacts
node_n_paths -->|"resolve"| node_n_loader
node_n_paths -->|"locate"| node_n_artifacts
node_n_defaults -->|"base settings"| node_n_config
node_n_config -->|"runtime config"| node_n_pipeline
node_n_scripts -->|"precompute"| node_n_artifacts
node_n_artifacts -->|"load"| node_n_ranker
node_n_ranker -->|"gate"| node_n_honeypot
node_n_ranker -->|"apply"| node_n_rules
node_n_ranker -->|"score"| node_n_semantic
node_n_ranker -->|"fuse"| node_n_heur_score
node_n_semantic -.->|"refine"| node_n_rerank
node_n_rerank -.->|"feedback"| node_n_ranker
node_n_honeypot -->|"filter context"| node_n_reason
node_n_heur_score -->|"ranked signal"| node_n_reason
node_n_reason -->|"emit"| node_n_pipeline
node_n_pipeline -->|"invoke"| node_n_run
node_n_pipeline -->|"check"| node_n_validate
node_n_validate -->|"verify"| node_n_tests

click node_n_raw "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/data/raw/sample_candidates.jsonl"
click node_n_loader "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/data/loader.py"
click node_n_clean "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/utils/text_cleaner.py"
click node_n_match "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/challenge/text_match.py"
click node_n_embedder "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/features/embedder.py"
click node_n_heur_feat "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/features/heuristic_extractor.py"
click node_n_feat_index "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/challenge/features.py"
click node_n_artifacts "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/artifacts/features.npy"
click node_n_paths "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/challenge/data_paths.py"
click node_n_defaults "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/config/defaults.py"
click node_n_config "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/utils/config.py"
click node_n_pipeline "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/pipeline.py"
click node_n_run "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/run_pipeline.py"
click node_n_ranker "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/challenge/redrob_ranker.py"
click node_n_honeypot "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/challenge/honeypot.py"
click node_n_rules "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/challenge/jd_config.py"
click node_n_semantic "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/challenge/semantic.py"
click node_n_rerank "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/challenge/rerank.py"
click node_n_heur_score "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/ranking/heuristic_scorer.py"
click node_n_reason "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/ranking/reasoning_gen.py"
click node_n_validate "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/validate_submission.py"
click node_n_scripts "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/scripts/precompute.py"
click node_n_tests "https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/tests/test_ranker.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n_raw,node_n_loader,node_n_clean,node_n_match,node_n_embedder,node_n_heur_feat,node_n_feat_index,node_n_artifacts toneBlue
class node_n_pipeline,node_n_run,node_n_ranker,node_n_honeypot,node_n_rules,node_n_semantic,node_n_rerank,node_n_heur_score,node_n_reason toneAmber
class node_n_paths,node_n_defaults,node_n_config,node_n_validate,node_n_scripts,node_n_tests toneMint
```

## Components

### Precompute phase

Offline ingestion, normalization, feature extraction, and artifact writing.

- [Raw data](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/data/raw/sample_candidates.jsonl): input dataset · Candidate JSONL and job-description inputs stored locally under data/raw/. · `data/raw/sample_candidates.jsonl`
- [Loader](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/data/loader.py): ingestion · Reads JSONL inputs and prepares records for downstream processing. · `src/data/loader.py`
- [Text clean](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/utils/text_cleaner.py): normalization · Tokenization and text normalization for noisy resume and JD content. · `src/utils/text_cleaner.py`
- [Text match](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/challenge/text_match.py): lexical matching · Phrase and lexical matching used during normalization and feature creation. · `src/challenge/text_match.py`
- [Embedder](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/features/embedder.py): embedding features · Builds dense semantic representations for candidates and job text. · `src/features/embedder.py`
- [Heuristic feats](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/features/heuristic_extractor.py): structural features · Extracts resume structure, experience patterns, and metadata-based features. · `src/features/heuristic_extractor.py`
- [Feature index](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/challenge/features.py): candidate index · Wraps extracted features into the candidate-facing feature index used by ranking. · `src/challenge/features.py`
- [Artifacts](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/artifacts/features.npy): offline store · Persisted embeddings, feature matrices, and candidate IDs used by the ranking phase. · `artifacts/features.npy`

### Ranking phase

Artifact-driven candidate scoring, fraud filtering, reranking, and output generation.

- [Pipeline](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/pipeline.py): CLI orchestration · Top-level entrypoint that runs the end-to-end flow and writes submission output. · `src/pipeline.py`
- [Runner](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/run_pipeline.py): wrapper CLI · Root-level wrapper for invoking the pipeline from the repository root. · `run_pipeline.py`
- [Redrob ranker](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/challenge/redrob_ranker.py): ranking orchestrator · Combines semantic, heuristic, and gating signals into the final rank order. · `src/challenge/redrob_ranker.py`
- [Honeypot](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/challenge/honeypot.py): fraud filter · Rejects structural anomalies, impossible dates, template resumes, and other quality issues. · `src/challenge/honeypot.py`
- [Domain rules](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/challenge/jd_config.py): business logic · Job-description schema, availability logic, assessment weighting, and consistency checks. · `src/challenge/jd_config.py`
- [Semantic match](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/challenge/semantic.py): similarity scoring · Embedding-based and lexical semantic matching for candidate-job fit. · `src/challenge/semantic.py`
- [Rerank](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/challenge/rerank.py): optional rerank · Optional second-pass reranking hook for semantic refinement. · `src/challenge/rerank.py`
- [Heuristic score](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/ranking/heuristic_scorer.py): score fusion · Weights and fuses rule-based signals into the final candidate score. · `src/ranking/heuristic_scorer.py`
- [Reasoning](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/ranking/reasoning_gen.py): explanations · Generates grounded candidate explanations for the final submission. · `src/ranking/reasoning_gen.py`

### Ops and guardrails

Configuration, validation, scripts, and tests that keep the pipeline deterministic.

- [Data paths](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/challenge/data_paths.py): path policy · Centralized repository-local path resolution for raw, interim, and artifact files. · `src/challenge/data_paths.py`
- [Defaults](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/config/defaults.py): runtime config · Central runtime settings and fixed reference values. · `src/config/defaults.py`
- [Config utils](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/utils/config.py): config helper · Loads and manages pipeline configuration values. · `src/utils/config.py`
- [Validate](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/src/validate_submission.py): submission checks · Enforces output shape, determinism, and submission correctness. · `src/validate_submission.py`
- [Scripts](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/scripts/precompute.py): ops tooling · Offline precompute and audit/sensitivity utilities for submission quality. · `scripts/precompute.py`
- [Tests](https://github.com/adityajaiswal440/india-runs-hackathon/blob/main/tests/test_ranker.py): verification suite · Coverage for loading, embeddings, feature extraction, precompute, and ranking. · `tests/test_ranker.py`

## Connections

- Raw data → Loader: load · Raw candidate data is ingested from local files.
- Loader → Text clean: normalize · Loaded records are cleaned and tokenized.
- Text clean → Text match: lexical prep · Normalized text is prepared for phrase and term matching.
- Text clean → Embedder: semantic prep · Clean text feeds embedding generation.
- Loader → Heuristic feats: structure prep · Record structure and metadata feed heuristic extraction.
- Text match → Feature index: signals · Lexical matching contributes to the candidate feature index.
- Embedder → Feature index: embeddings · Dense semantic vectors become part of the feature index.
- Heuristic feats → Feature index: heuristics · Structural features are folded into the candidate index.
- Feature index → Artifacts: persist · Precomputed features and IDs are written to offline artifacts.
- Data paths → Loader: resolve · Repository-local paths are used for data ingress.
- Data paths → Artifacts: locate · Artifact directories and files are resolved centrally.
- Defaults → Config utils: base settings · Defaults provide the baseline runtime configuration.
- Config utils → Pipeline: runtime config · Pipeline execution uses configured runtime values.
- Scripts → Artifacts: precompute · Scripts populate offline artifacts before ranking.
- Artifacts → Redrob ranker: load · Ranking consumes the precomputed artifacts.
- Redrob ranker → Honeypot: gate · Quality and fraud checks filter candidates before ranking.
- Redrob ranker → Domain rules: apply · Domain-specific rules shape the scoring logic.
- Redrob ranker → Semantic match: score · Semantic similarity contributes to candidate ranking.
- Redrob ranker → Heuristic score: fuse · Heuristic scoring fuses multiple signal families.
- Semantic match → Rerank: refine · Optional reranking adds a second semantic pass.
- Rerank → Redrob ranker: feedback · Rerank can feed refined ordering back into the orchestrator.
- Honeypot → Reasoning: filter context · Filtered candidates move into explanation generation.
- Heuristic score → Reasoning: ranked signal · Final scores inform candidate reasoning output.
- Reasoning → Pipeline: emit · Reasoned ranking output is assembled for submission.
- Pipeline → Runner: invoke · Root wrapper launches the top-level pipeline.
- Pipeline → Validate: check · Submission output is validated before completion.
- Validate → Tests: verify · Validation logic is reinforced by the test suite.
