# bhaveshg-22/data-alchemist architecture

> Architecture diagram of the GitHub repository bhaveshg-22/data-alchemist, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bhaveshg-22/data-alchemist
- Repository: https://github.com/bhaveshg-22/data-alchemist

## Overview

##### Project type and purpose
- This is a full-stack web application built with Next.js (App Router), React, and TypeScript.
- Purpose: “Data Alchemist” ingests CSV/XLSX datasets, runs comprehensive validations, applies AI-powered transformations, lets users configure business rules, and then exports cleaned data and rule sets.

##### File structure
- Top-level:
- next.config.ts, tsconfig.json, package.json → Next.js + TypeScript configuration.
- public/sample → Static sample CSV/XLSX and JSON.
- src/app/… → Next.js App Router:
- src/app/page.tsx, layout.tsx → UI shell, pages.
- src/app/api/[ai-column-mapping, ai-fix, nl-modify, validation]/route.ts → Serverless API endpoints.
- src/components → React UI components (file upload, editable tables, AI rule input, validation summary, export).
- src/hooks → Custom React hooks (state management for command templates, toast notifications, undo/redo history).
- src/utils → Pure utility modules (file parsing, export wrangling, NLP helpers).
- src/validators → Domain logic modules implementing 12+ validation rules.
- config, validators/types.ts → Shared type definitions, prioritization criteria.

##### README for insights
- Confirms Next.js 15 + React 19 + Tailwind CSS.
- Walks through core and bonus AI features, specifies which files implement which features.
- No traditional database — data flows through in-memory React state and stateless API calls, with all persistence via file export.

a. Main components
1. Client Browser (React SPA)
2. Next.js Server (App Router):
- Page rendering
- Serverless API routes
3. AI Service (external OpenAI API)
4. Validation Engine (stateless JS modules in /src/validators)
5. File Parsing & Export Utils
6. Static Assets (public/sample)
b. Relationships & data flows
- Browser → Next.js page/ui components → Redux-style React state via hooks.
- FileUpload component pushes CSV/XLSX to fileParser → data stored in client state.
- EditableDataTable ↔ ValidationView triggers client-side validator modules.
- When user invokes AI mapping/fixes or NL modifications, UI calls the respective Next.js API route → the route forwards requests to OpenAI (via LangChain/OpenAI API) → receives suggestions → returns JSON to UI → UI updates state.
- ExportButton component calls exportUtils to generate CSVs/ZIP, then triggers file-saver.
- No central database: persistence is ephemeral until user exports.
c. Architectural patterns and design principles
- Separation of concerns: UI (components) vs. domain logic (validators, utils) vs. API layer.
- Serverless functions: each feature’s AI/validation endpoint is isolated.
- Pure functions for data parsing and validation—stateless, side-effect-free modules.
- Hooks for cross-cutting client state (history, toasts, command templates).
d. Key technologies/frameworks
- Next.js App Router (React server components + API routes)
- React 19 + TypeScript
- Tailwind CSS for styling
- PapaParse & XLSX.js for file parsing
- OpenAI API (+ LangChain) for AI features
- Custom JS validator modules
- file-saver & JSZip for export

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Side
    subgraph "Client Side"
        direction TB
        FB["FileUpload"]:::ui
        EP["EditableDataTable"]:::ui
        VV["ValidationView"]:::ui
        VS["ValidationSummary"]:::ui
        EB["ExportButton"]:::ui
        NM["NLDataModifier"]:::ui
        NR["NLRuleInput"]:::ui
        CT["CommandTemplateSelector"]:::ui
        IS["IssuesSidebar"]:::ui
        DV["DiffVisualization"]:::ui
        FD["FilteredDataView"]:::ui
        TBV["TabbedDataView"]:::ui
        PW["PrioritizationWeights"]:::ui
        TC["ToastContainer"]:::ui

        subgraph "Hooks"
            direction TB
            HT1["useCommandTemplates"]:::hook
            HT2["useDataModificationHistory"]:::hook
            HT3["useToast"]:::hook
        end

        subgraph "Utils"
            direction TB
            UP["fileParser"]:::util
            UE["exportUtils"]:::util
            UN["nlDataUtils"]:::util
        end

        subgraph "Validation Engine"
            direction TB
            VI["Validator Index"]:::validator
            VT["Shared Types"]:::validator
            VU["Validator Utils"]:::validator
            V1["validateMissingColumns"]:::validator
            V2["validateDuplicateIDs"]:::validator
            V3["validateConcurrencyFeasibility"]:::validator
            V4["validateCircularCoRun"]:::validator
            V5["validateConflictingRules"]:::validator
            V6["validateMalformedLists"]:::validator
            V7["validateJSONFields"]:::validator
            V8["validateNLRules"]:::validator
            V9["validateOutOfRange"]:::validator
            V10["validateOverloadedWorkers"]:::validator
            V11["validatePhaseSaturation"]:::validator
            V12["validateReferences"]:::validator
            V13["validateSkillCoverage"]:::validator
        end

        FB -->|parses file| UP
        UP -->|stores data| State[(Client State)]:::static
        EP -->|reads/writes| State
        EP --> VV
        VV -->|calls validators| V1
        VV --> V2
        VV --> V3
        VV --> V4
        VV --> V5
        VV --> V6
        VV --> V7
        VV --> V8
        VV --> V9
        VV --> V10
        VV --> V11
        VV --> V12
        VV --> V13
        EP --> VS
        EB -->|exports| UE
        UE -->|triggers| FileSaver[[Browser FileSaver]]:::external
        NM -->|calls API| API_NL_MOD
        NR -->|calls API| API_NL_MOD
        VS --> IS
        EP --> DV
        EP --> FD
        EP --> TBV
        EP --> PW
        TC -.->|notifications| State
    end

    %% Static Assets
    SA["Sample Data (public/sample)"]:::static
    SA --> FB

    %% Server Side
    subgraph "Server Side (Next.js)"
        direction TB
        PG["Page Rendering"]:::ui
        LY["Layout Shell"]:::ui

        subgraph "API Routes"
            direction TB
            API_ACM["AI Column Mapping"]:::api
            API_AF["AI Fix"]:::api
            API_NL_MOD["Natural Language Modify"]:::api
            API_VAL["Validation Controller"]:::api
        end

        PG --> LY
        EP -->|client call POST /api/ai-column-mapping| API_ACM
        EP -->|POST /api/ai-fix| API_AF
        NM -->|POST /api/nl-modify| API_NL_MOD
        VV -->|POST /api/validation| API_VAL

        API_ACM -->|forward request| OpenAI["OpenAI API"]:::external
        API_AF --> OpenAI
        API_NL_MOD --> OpenAI
        API_VAL --> VI
    end

    %% Styles
    classDef ui fill:#D4E3FC,stroke:#0366d6;
    classDef hook fill:#E0F7FA,stroke:#00796b;
    classDef util fill:#E8F5E9,stroke:#388E3C;
    classDef validator fill:#FFF3E0,stroke:#f57c00;
    classDef api fill:#FFE0B2,stroke:#F57C00;
    classDef external fill:#E1BEE7,stroke:#6A1B9A;
    classDef static fill:#CFD8DC,stroke:#455A64;

    %% Click Events
    click PG "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/app/page.tsx"
    click LY "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/app/layout.tsx"
    click API_ACM "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/app/api/ai-column-mapping/route.ts"
    click API_AF "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/app/api/ai-fix/route.ts"
    click API_NL_MOD "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/app/api/nl-modify/route.ts"
    click API_VAL "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/app/api/validation/route.ts"
    click FB "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/components/FileUpload.tsx"
    click EP "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/components/EditableDataTable.tsx"
    click VV "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/components/ValidationView.tsx"
    click VS "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/components/ValidationSummary.tsx"
    click EB "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/components/ExportButton.tsx"
    click NM "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/components/NLDataModifier.tsx"
    click NR "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/components/NLRuleInput.tsx"
    click CT "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/components/CommandTemplateSelector.tsx"
    click IS "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/components/IssuesSidebar.tsx"
    click DV "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/components/DiffVisualization.tsx"
    click FD "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/components/FilteredDataView.tsx"
    click TBV "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/components/TabbedDataView.tsx"
    click PW "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/components/PrioritizationWeights.tsx"
    click TC "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/components/ToastContainer.tsx"
    click HT1 "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/hooks/useCommandTemplates.ts"
    click HT2 "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/hooks/useDataModificationHistory.ts"
    click HT3 "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/hooks/useToast.ts"
    click UP "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/utils/fileParser.ts"
    click UE "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/utils/exportUtils.ts"
    click UN "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/utils/nlDataUtils.ts"
    click VI "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/validators/index.ts"
    click VT "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/validators/types.ts"
    click VU "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/validators/utils.ts"
    click V1 "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/validators/validateMissingColumns.ts"
    click V2 "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/validators/validateDuplicateIDs.ts"
    click V3 "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/validators/validateConcurrencyFeasibility.ts"
    click V4 "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/validators/validateCircularCoRun.ts"
    click V5 "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/validators/validateConflictingRules.ts"
    click V6 "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/validators/validateMalformedLists.ts"
    click V7 "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/validators/validateJSONFields.ts"
    click V8 "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/validators/validateNLRules.ts"
    click V9 "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/validators/validateOutOfRange.ts"
    click V10 "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/validators/validateOverloadedWorkers.ts"
    click V11 "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/validators/validatePhaseSaturation.ts"
    click V12 "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/validators/validateReferences.ts"
    click V13 "https://github.com/bhaveshg-22/data-alchemist/blob/main/src/validators/validateSkillCoverage.ts"
    click SA "https://github.com/bhaveshg-22/data-alchemist/tree/main/public/sample/"
```
