# nandhu270/multisource-data-transformer architecture

> Architecture diagram of the GitHub repository nandhu270/multisource-data-transformer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nandhu270/multisource-data-transformer
- Repository: https://github.com/nandhu270/multisource-data-transformer

## Overview

This is a multi-source candidate-profile transformer with a React interface and a FastAPI processing backend. The intended workflow is to configure priorities, upload recruiter and GitHub CSVs plus resumes, run the API pipeline, and inspect merged profiles, conflicts, confidence, and job-description matches. The backend stages are documented and partly sampled; the frontend excerpts show state and result views, but do not establish a live request from the UI to the API. That integration is therefore not asserted here.

Main entry points: [`Backend/pipeline.py`](https://github.com/nandhu270/multisource-data-transformer/blob/main/Backend/pipeline.py) (Pipeline orchestration), [`src/pages/Results.jsx`](https://github.com/nandhu270/multisource-data-transformer/blob/main/src/pages/Results.jsx) (Candidate results), [`src/context/PipelineContext.jsx`](https://github.com/nandhu270/multisource-data-transformer/blob/main/src/context/PipelineContext.jsx) (Pipeline state), [`src/pages/UploadSources.jsx`](https://github.com/nandhu270/multisource-data-transformer/blob/main/src/pages/UploadSources.jsx) (Source upload)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_client["Frontend client"]
  node_config["Priority configuration<br/>[ConfigEditor.jsx]"]
  node_upload["Source upload<br/>[UploadSources.jsx]"]
  node_context["Pipeline state"]
  node_mockdata["Sample profile data<br/>[mockData.js]"]
end

subgraph group_service["Backend service"]
  node_app["FastAPI endpoint<br/>[app.py]"]
end

subgraph group_processing["Profile processing"]
  node_pipeline["Pipeline orchestration<br/>[pipeline.py]"]
  node_extractor["Resume extraction<br/>[extractor.py]"]
  node_normalizer["Field normalization<br/>[normalizer.py]"]
  node_matcher["Matching algorithms<br/>[matcher.py]"]
end

subgraph group_review["Results review"]
  node_results["Candidate results<br/>[Results.jsx]"]
  node_conflicts["Conflict details<br/>[ConflictLog.jsx]"]
  node_confidence["Confidence display"]
  node_sourcebadge["Source attribution<br/>[SourceBadge.jsx]"]
  node_githubicon["GitHub icon<br/>[GithubIcon.jsx]"]
  node_experience["Experience timeline"]
  node_stages["Pipeline progress<br/>[PipelineStages.jsx]"]
end

node_recruiter["Recruiter CSV"]
node_githubcsv["GitHub CSV"]
node_resumes["Resume files"]
node_user(("Recruiter"))
node_githubapi["GitHub API"]

node_user -->|"sets priorities"| node_config
node_user -->|"selects sources"| node_upload
node_upload -->|"records uploads"| node_context
node_config -->|"saves settings"| node_context
node_context -->|"loads samples"| node_mockdata
node_user -->|"submits pipeline"| node_app
node_app -->|"invokes pipeline"| node_pipeline
node_pipeline -->|"extracts resumes"| node_extractor
node_pipeline -->|"normalizes fields"| node_normalizer
node_pipeline -->|"matches sources"| node_matcher
node_pipeline -->|"fetches repositories"| node_githubapi
node_extractor -->|"uses normalization"| node_normalizer
node_matcher -->|"uses normalization"| node_normalizer
node_recruiter -->|"ingested by"| node_pipeline
node_githubcsv -->|"ingested by"| node_pipeline
node_resumes -->|"processed by"| node_pipeline
node_pipeline -.->|"presents output"| node_results
node_results -->|"reads results"| node_context
node_results -->|"renders conflicts"| node_conflicts
node_results -->|"renders confidence"| node_confidence
node_results -->|"shows experience"| node_experience
node_conflicts -->|"labels sources"| node_sourcebadge
node_conflicts -->|"shows confidence"| node_confidence
node_sourcebadge -->|"uses icon"| node_githubicon
node_experience -->|"labels provenance"| node_sourcebadge
node_experience -->|"shows confidence"| node_confidence
node_upload -->|"shows progress"| node_stages

click node_app "https://github.com/nandhu270/multisource-data-transformer/blob/main/Backend/app.py"
click node_pipeline "https://github.com/nandhu270/multisource-data-transformer/blob/main/Backend/pipeline.py"
click node_extractor "https://github.com/nandhu270/multisource-data-transformer/blob/main/Backend/extractor.py"
click node_normalizer "https://github.com/nandhu270/multisource-data-transformer/blob/main/Backend/normalizer.py"
click node_matcher "https://github.com/nandhu270/multisource-data-transformer/blob/main/Backend/matcher.py"
click node_config "https://github.com/nandhu270/multisource-data-transformer/blob/main/src/pages/ConfigEditor.jsx"
click node_upload "https://github.com/nandhu270/multisource-data-transformer/blob/main/src/pages/UploadSources.jsx"
click node_context "https://github.com/nandhu270/multisource-data-transformer/blob/main/src/context/PipelineContext.jsx"
click node_mockdata "https://github.com/nandhu270/multisource-data-transformer/blob/main/src/data/mockData.js"
click node_results "https://github.com/nandhu270/multisource-data-transformer/blob/main/src/pages/Results.jsx"
click node_conflicts "https://github.com/nandhu270/multisource-data-transformer/blob/main/src/components/ui/ConflictLog.jsx"
click node_confidence "https://github.com/nandhu270/multisource-data-transformer/blob/main/src/components/ui/ConfidenceBadge.jsx"
click node_sourcebadge "https://github.com/nandhu270/multisource-data-transformer/blob/main/src/components/ui/SourceBadge.jsx"
click node_githubicon "https://github.com/nandhu270/multisource-data-transformer/blob/main/src/components/icons/GithubIcon.jsx"
click node_experience "https://github.com/nandhu270/multisource-data-transformer/blob/main/src/components/ui/ExperienceTimeline.jsx"
click node_stages "https://github.com/nandhu270/multisource-data-transformer/blob/main/src/components/ui/PipelineStages.jsx"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_config,node_upload,node_context,node_mockdata toneBlue
class node_app toneAmber
class node_pipeline,node_extractor,node_normalizer,node_matcher,node_githubapi toneMint
class node_results,node_conflicts,node_confidence,node_sourcebadge,node_githubicon,node_experience,node_stages toneRose
class node_recruiter,node_githubcsv,node_resumes,node_user toneIndigo
```

## Components

### Frontend client

- [Priority configuration](https://github.com/nandhu270/multisource-data-transformer/blob/main/src/pages/ConfigEditor.jsx): `src/pages/ConfigEditor.jsx`
- [Source upload](https://github.com/nandhu270/multisource-data-transformer/blob/main/src/pages/UploadSources.jsx): `src/pages/UploadSources.jsx`
- [Pipeline state](https://github.com/nandhu270/multisource-data-transformer/blob/main/src/context/PipelineContext.jsx): `src/context/PipelineContext.jsx`
- [Sample profile data](https://github.com/nandhu270/multisource-data-transformer/blob/main/src/data/mockData.js): `src/data/mockData.js`

### Backend service

- [FastAPI endpoint](https://github.com/nandhu270/multisource-data-transformer/blob/main/Backend/app.py): `Backend/app.py`

### Profile processing

- [Pipeline orchestration](https://github.com/nandhu270/multisource-data-transformer/blob/main/Backend/pipeline.py): `Backend/pipeline.py`
- [Resume extraction](https://github.com/nandhu270/multisource-data-transformer/blob/main/Backend/extractor.py): `Backend/extractor.py`
- [Field normalization](https://github.com/nandhu270/multisource-data-transformer/blob/main/Backend/normalizer.py): `Backend/normalizer.py`
- [Matching algorithms](https://github.com/nandhu270/multisource-data-transformer/blob/main/Backend/matcher.py): `Backend/matcher.py`

### Results review

- [Candidate results](https://github.com/nandhu270/multisource-data-transformer/blob/main/src/pages/Results.jsx): `src/pages/Results.jsx`
- [Conflict details](https://github.com/nandhu270/multisource-data-transformer/blob/main/src/components/ui/ConflictLog.jsx): `src/components/ui/ConflictLog.jsx`
- [Confidence display](https://github.com/nandhu270/multisource-data-transformer/blob/main/src/components/ui/ConfidenceBadge.jsx): `src/components/ui/ConfidenceBadge.jsx`
- [Source attribution](https://github.com/nandhu270/multisource-data-transformer/blob/main/src/components/ui/SourceBadge.jsx): `src/components/ui/SourceBadge.jsx`
- [GitHub icon](https://github.com/nandhu270/multisource-data-transformer/blob/main/src/components/icons/GithubIcon.jsx): `src/components/icons/GithubIcon.jsx`
- [Experience timeline](https://github.com/nandhu270/multisource-data-transformer/blob/main/src/components/ui/ExperienceTimeline.jsx): `src/components/ui/ExperienceTimeline.jsx`
- [Pipeline progress](https://github.com/nandhu270/multisource-data-transformer/blob/main/src/components/ui/PipelineStages.jsx): `src/components/ui/PipelineStages.jsx`

### Other components

- **Recruiter CSV**
- **GitHub CSV**
- **Resume files**
- **Recruiter**
- **GitHub API**

## Connections

- Recruiter → Priority configuration: sets priorities (evidence: `README.md`)
- Recruiter → Source upload: selects sources (evidence: `src/pages/UploadSources.jsx`)
- Source upload → Pipeline state: records uploads (evidence: `src/context/PipelineContext.jsx`)
- Priority configuration → Pipeline state: saves settings (evidence: `src/context/PipelineContext.jsx`)
- Pipeline state → Sample profile data: loads samples (evidence: `src/context/PipelineContext.jsx`)
- Recruiter → FastAPI endpoint: submits pipeline (evidence: `README.md`)
- FastAPI endpoint → Pipeline orchestration: invokes pipeline (evidence: `Backend/app.py`)
- Pipeline orchestration → Resume extraction: extracts resumes (evidence: `Backend/pipeline.py`)
- Pipeline orchestration → Field normalization: normalizes fields (evidence: `Backend/pipeline.py`)
- Pipeline orchestration → Matching algorithms: matches sources (evidence: `Backend/pipeline.py`)
- Pipeline orchestration → GitHub API: fetches repositories (evidence: `Backend/pipeline.py`)
- Resume extraction → Field normalization: uses normalization (evidence: `Backend/extractor.py`)
- Matching algorithms → Field normalization: uses normalization (evidence: `Backend/matcher.py`)
- Recruiter CSV → Pipeline orchestration: ingested by (evidence: `README.md`)
- GitHub CSV → Pipeline orchestration: ingested by (evidence: `README.md`)
- Resume files → Pipeline orchestration: processed by (evidence: `README.md`)
- Pipeline orchestration → Candidate results: presents output (evidence: `README.md`)
- Candidate results → Pipeline state: reads results (evidence: `src/pages/Results.jsx`)
- Candidate results → Conflict details: renders conflicts (evidence: `src/pages/Results.jsx`)
- Candidate results → Confidence display: renders confidence (evidence: `src/pages/Results.jsx`)
- Candidate results → Experience timeline: shows experience (evidence: `src/pages/Results.jsx`)
- Conflict details → Source attribution: labels sources (evidence: `src/components/ui/ConflictLog.jsx`)
- Conflict details → Confidence display: shows confidence (evidence: `src/components/ui/ConflictLog.jsx`)
- Source attribution → GitHub icon: uses icon (evidence: `src/components/ui/SourceBadge.jsx`)
- Experience timeline → Source attribution: labels provenance (evidence: `src/components/ui/ExperienceTimeline.jsx`)
- Experience timeline → Confidence display: shows confidence (evidence: `src/components/ui/ExperienceTimeline.jsx`)
- Source upload → Pipeline progress: shows progress (evidence: `src/pages/UploadSources.jsx`)
