# tom-bohn/sfdc-user-permissions-ai architecture

> Architecture diagram of the GitHub repository tom-bohn/sfdc-user-permissions-ai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tom-bohn/sfdc-user-permissions-ai
- Repository: https://github.com/tom-bohn/sfdc-user-permissions-ai

## Overview

##### Project Type and Purpose
- This is a data-pipeline / AI-driven analytics tool for extracting and classifying Salesforce user permissions. It ingests raw scraped data, transforms it, applies LLM-based classification/evaluation, and outputs results for further analysis or reporting (e.g., via notebooks).

##### High-Level File-Structure Analysis
- Top-level folders:
- config/ – environment and template YAML files
- data/ – three subfolders:
– raw/ (scraped MHTML from Salesforce)
– input/ (reference and sample CSVs used as lookups or processed inputs)
– output/ (final analysis results)
- src/ – core logic, organized in subpackages:
– scraping/ (permission_scraper.py)
– processing/ (json_processor.py)
– llms/ (classifier/evaluator modules)
– analysis/ (higher-level orchestration or ML workflows)
– prompts/ (prompt templates for LLM interactions)
– utils/ (data utility functions)
- notebooks/ – exploratory or presentation notebooks
- tests/ – unit tests

##### Key Components
a. External Service
– Salesforce instance (source of permission data)
b. Ingestion Layer
– PermissionScraper (src/scraping/permission_scraper.py)
– Writes MHTML to data/raw/
c. Processing Layer
– JSONProcessor (src/processing/json_processor.py)
– DataUtils (src/utils/data_utils.py)
– Transforms raw MHTML → structured JSON/CSV in data/input/
d. Reference Data Store
– data/input/*.csv (user-permission reference data)
e. LLM Analysis Layer
– LLMServices (src/llms/*_classifier.py + *_evaluator.py)
– Categories: risk, cloud, description, user-perm classification
– Prompt templates (src/prompts/templates/*.md) drive the LLM chains
f. Output Layer
– Writes final classified results to data/output/
– Notebooks (Notebooks/*.ipynb) consume output for visualization
g. Configuration & Auxiliary
– config/environment.template.yml (env variables, API keys)
– requirements.txt (dependencies)
– tests/ (src/scraping and other modules)

##### Architectural Patterns & Principles
- Layered Separation: ingestion → processing → analysis → presentation
- Single-responsibility modules (one scraper, one processor, one classifier)
- Externalize configuration (YAML/.env) and prompts (markdown templates)
- Pluggable LLM chains (easy to add new classifiers or evaluators)

##### Detail Level
- Prompt subfolder and template names if prompt orchestration is a key architectural concern

## Diagram (Mermaid)

```mermaid
flowchart TB
  %% External Services
  subgraph "External Services"
    SF["Salesforce Instance"]:::external
    LLMAPI["LLM API Provider"]:::external
  end

  %% Configuration
  subgraph "Configuration & Auxiliary"
    Env["environment.template.yml"]:::config
    Req["requirements.txt"]:::config
  end

  %% Ingestion Layer
  subgraph "Ingestion Layer"
    PS["PermissionScraper"]:::compute
  end

  %% Processing Layer
  subgraph "Processing Layer"
    JP["JSONProcessor"]:::compute
    DU["DataUtils"]:::compute
  end

  %% LLM Analysis Layer
  subgraph "LLM Analysis Layer"
    Prompts["Prompt Templates"]:::config
    Chat["LLM Chat Session Utility"]:::compute
    subgraph "Classifiers & Evaluators"
      CC["Category Classifier"]:::compute
      CEv["Category Evaluator"]:::compute
      CloC["Cloud Classifier"]:::compute
      CloE["Cloud Evaluator"]:::compute
      DC["Description Classifier"]:::compute
      DE["Description Evaluator"]:::compute
      RC["Risk Classifier"]:::compute
      RE["Risk Evaluator"]:::compute
    end
  end

  %% Data Stores
  subgraph "Data Stores"
    Raw["Raw Data Store"]:::datastore
    Ref["Reference CSVs"]:::datastore
    Out["Output Data Store"]:::datastore
  end

  %% Visualization
  subgraph "Visualization"
    NB["Notebooks"]:::compute
  end

  %% Data Flows
  SF -->|"MHTML"| PS
  PS -->|"MHTML"| Raw
  Raw -->|"MHTML"| JP
  JP -->|"JSON"| DU
  DU -->|"Structured JSON/CSV"| Ref
  Ref -->|"CSV"| Chat

  Prompts -->|"Prompt text"| Chat
  Chat -->|"Prompt text"| CC
  Chat -->|"Prompt text"| CloC
  Chat -->|"Prompt text"| DC
  Chat -->|"Prompt text"| RC

  CC -->|"Prompt text"| LLMAPI
  CloC -->|"Prompt text"| LLMAPI
  DC -->|"Prompt text"| LLMAPI
  RC -->|"Prompt text"| LLMAPI

  LLMAPI -->|"Response"| CC
  LLMAPI -->|"Response"| CloC
  LLMAPI -->|"Response"| DC
  LLMAPI -->|"Response"| RC

  CC -->|"Classification result"| CEv
  CEv -->|"Evaluated result"| Out
  CloC -->|"Classification result"| CloE
  CloE -->|"Evaluated result"| Out
  DC -->|"Classification result"| DE
  DE -->|"Evaluated result"| Out
  RC -->|"Classification result"| RE
  RE -->|"Evaluated result"| Out

  Out -->|"CSV/JSON"| NB

  Env --> PS
  Env --> JP
  Env --> Chat
  Req --> PS
  Req --> JP
  Req --> Chat

  %% Click Events
  click PS "https://github.com/tom-bohn/sfdc-user-permissions-ai/blob/main/src/scraping/permission_scraper.py"
  click Raw "https://github.com/tom-bohn/sfdc-user-permissions-ai/tree/main/data/raw/"
  click JP "https://github.com/tom-bohn/sfdc-user-permissions-ai/blob/main/src/processing/json_processor.py"
  click DU "https://github.com/tom-bohn/sfdc-user-permissions-ai/blob/main/src/utils/data_utils.py"
  click Ref "https://github.com/tom-bohn/sfdc-user-permissions-ai/blob/main/data/input/*.csv"
  click CC "https://github.com/tom-bohn/sfdc-user-permissions-ai/blob/main/src/llms/category_classifier.py"
  click CEv "https://github.com/tom-bohn/sfdc-user-permissions-ai/blob/main/src/llms/category_evaluator.py"
  click CloC "https://github.com/tom-bohn/sfdc-user-permissions-ai/blob/main/src/llms/cloud_classifier.py"
  click CloE "https://github.com/tom-bohn/sfdc-user-permissions-ai/blob/main/src/llms/cloud_evaluator.py"
  click DC "https://github.com/tom-bohn/sfdc-user-permissions-ai/blob/main/src/llms/description_classifier.py"
  click DE "https://github.com/tom-bohn/sfdc-user-permissions-ai/blob/main/src/llms/description_evaluator.py"
  click RC "https://github.com/tom-bohn/sfdc-user-permissions-ai/blob/main/src/llms/risk_classifier.py"
  click RE "https://github.com/tom-bohn/sfdc-user-permissions-ai/blob/main/src/llms/risk_evaluator.py"
  click Chat "https://github.com/tom-bohn/sfdc-user-permissions-ai/blob/main/src/llms/chat_session.py"
  click Prompts "https://github.com/tom-bohn/sfdc-user-permissions-ai/blob/main/src/prompts/templates/*.md"
  click Out "https://github.com/tom-bohn/sfdc-user-permissions-ai/tree/main/data/output/"
  click NB "https://github.com/tom-bohn/sfdc-user-permissions-ai/blob/main/Notebooks/SFDC_User_Permission_AI.ipynb"

  %% Styles
  classDef compute fill:#ADD8E6,stroke:#333,stroke-width:1px
  classDef datastore fill:#90EE90,stroke:#333,stroke-width:1px
  classDef external fill:#FFA500,stroke:#333,stroke-width:1px
  classDef config fill:#D3D3D3,stroke:#333,stroke-width:1px
```
