# lryanle/smare architecture

> Architecture diagram of the GitHub repository lryanle/smare, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lryanle/smare
- Repository: https://github.com/lryanle/smare

## Overview

##### Project type and purpose
- This is a full-stack SaaS application named SMARE (Social Marketplace Automotive Risk Engine).
- Purpose: scrape social marketplaces (Craigslist, Facebook Marketplace), clean and score listings via multiple ML models to flag suspicious/insurance-fraud listings, persist results in MongoDB, and present a dashboard UI for end users.

##### File structure
- Top-level:
- frontend/   → Next.js + TypeScript client + API routes + Prisma ORM
- backend/    → Python pipeline: scrapers, cleaners, model scoring, orchestrator
- classification-chrome-extension/ → Chrome extension for labeling/classification
- documentation/ → project docs
- Key backend pieces:
- orchestrator.py         → pipeline driver
- src/scrapers/           → Selenium & BeautifulSoup scrapers
- src/cleaners/           → data cleaning
- src/models/             → six scoring models & model manager
- src/utilities/database  → MongoDB adapter (pymongo)
- src/sendGrid/notifs.py  → email notifications
- Key frontend pieces:
- app/api/                → Next.js API routes as middleware
- lib/prisma.ts           → Prisma client for MongoDB
- components/             → React UI (dashboard, datatable, shared UI)
- NextAuth for authentication
- Docker configs for local/dev/prod

##### README for additional insights
- Stack callouts: Next.js, Tailwind, Prisma, NextAuth, Selenium, BS4, Pandas, imbalanced-learn, Loguru, OpenAI API.
- High-level summary confirms pipeline segments: scraping → cleaning → modeling → dashboard.

##### Elements
- Component blocks with clear labels: “Browser UI”, “Next.js API Server”, “Prisma ORM”, “MongoDB Atlas”, “Python Scraper Service”, “Python Cleaner Service”, “ML Model Service”, “Orchestrator”, “OpenAI API”, “SendGrid”, “Chrome Extension”.
- Pipeline data flow (scrape → clean → model → store)
- Request/response flow (Browser → API → DB → Browser)
- External service calls
- Web layer (blue)
- API/database layer (green)
- Data-processing pipeline (orange)
- External services (purple)
- Chrome extension (gray)

##### Final advice
- Be very detailed: capture each pipeline step, each interaction.
- That each component block references the actual directory or file (e.g., backend/src/scrapers).

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "Web Layer"
        BrowserUI["Browser UI"]:::web
    end

    subgraph "Next.js Layer"
        NextServer["Next.js API Server"]:::api
        Auth["NextAuth"]:::api
        Prisma["Prisma ORM"]:::db
        MongoUtil["MongoDB Connection Utility"]:::db
    end

    subgraph "Database"
        Mongo["MongoDB Atlas"]:::db
    end

    subgraph "Data Pipeline"
        Orchestrator["Orchestrator"]:::pipeline
        Scrapers["Scraper Modules"]:::pipeline
        Cleaners["Cleaner Modules"]:::pipeline
        Models["Model Scoring Service"]:::pipeline
        DBAdapter["Database Adapter"]:::pipeline
    end

    subgraph "External Services"
        OpenAI["OpenAI API"]:::external
        SendGrid["SendGrid"]:::external
    end

    subgraph "Chrome Extension"
        Extension["Chrome Extension"]:::ext
    end

    BrowserUI -->|"HTTPS REST"| NextServer
    Extension -->|"POST labels"| NextServer
    NextServer -->|"auth flow"| Auth
    Auth -->|"session cookie"| NextServer
    NextServer -->|"queries"| Prisma
    Prisma -->|"connects"| Mongo
    NextServer -->|"connects"| MongoUtil
    MongoUtil -->|"connects"| Mongo

    Orchestrator -->|schedules| Scrapers
    Scrapers -->|"writes raw data"| Mongo
    Orchestrator -->|triggers| Cleaners
    Cleaners -->|"writes cleaned data"| Mongo
    Orchestrator -->|triggers| Models
    Models -->|"writes scores"| Mongo
    Models -->|"requests GPT"| OpenAI
    Models -->|"flags results"| SendGrid
    Orchestrator -->|"uses"| DBAdapter
    DBAdapter -->|"connects"| Mongo

    click BrowserUI "https://github.com/lryanle/smare/tree/main/frontend/app/"
    click BrowserUI "https://github.com/lryanle/smare/tree/main/frontend/components/"
    click NextServer "https://github.com/lryanle/smare/tree/main/frontend/app/api/"
    click Auth "https://github.com/lryanle/smare/blob/main/frontend/app/api/auth/[...nextauth]/authOptions.ts"
    click Prisma "https://github.com/lryanle/smare/blob/main/frontend/lib/prisma.ts"
    click Prisma "https://github.com/lryanle/smare/blob/main/frontend/prisma/schema.prisma"
    click MongoUtil "https://github.com/lryanle/smare/blob/main/frontend/lib/mongodb.ts"
    click Orchestrator "https://github.com/lryanle/smare/blob/main/backend/orchestrator.py"
    click Scrapers "https://github.com/lryanle/smare/tree/main/backend/src/scrapers/"
    click Cleaners "https://github.com/lryanle/smare/tree/main/backend/src/cleaners/"
    click Models "https://github.com/lryanle/smare/blob/main/backend/src/models/model_manager.py"
    click DBAdapter "https://github.com/lryanle/smare/blob/main/backend/src/utilities/database.py"
    click SendGrid "https://github.com/lryanle/smare/blob/main/backend/src/sendGrid/notifs.py"
    click Extension "https://github.com/lryanle/smare/tree/main/classification-chrome-extension/"

    classDef web fill:#cce5ff,stroke:#004085,color:#004085
    classDef api fill:#d4edda,stroke:#155724,color:#155724
    classDef db fill:#d1ecf1,stroke:#0c5460,color:#0c5460
    classDef pipeline fill:#fff3cd,stroke:#856404,color:#856404
    classDef external fill:#f5c6cb,stroke:#721c24,color:#721c24
    classDef ext fill:#e2e3e5,stroke:#6c757d,color:#6c757d
```
