How tom-bohn/sfdc-user-permissions-ai works
Project Type and Purpose
• This is a data-pipeline / AI-driven analytics tool for extracting and classifying Salesforce user permissions. It ingests raw scraped data, transforms it, applies LLM-based classification/evaluation, and outputs results for further analysis or reporting (e.g., via notebooks).
High-Level File-Structure Analysis
• Top-level folders:
• config/ – environment and template YAML files
• data/ – three subfolders:
– raw/ (scraped MHTML from Salesforce)
– input/ (reference and sample CSVs used as lookups or processed inputs)
– output/ (final analysis results)
• src/ – core logic, organized in subpackages:
– scraping/ (permission_scraper.py)
– processing/ (json_processor.py)
– llms/ (classifier/evaluator modules)
– analysis/ (higher-level orchestration or ML workflows)
– prompts/ (prompt templates for LLM interactions)
– utils/ (data utility functions)
• notebooks/ – exploratory or presentation notebooks
• tests/ – unit tests
Key Components
a. External Service
– Salesforce instance (source of permission data)
b. Ingestion Layer
– PermissionScraper (src/scraping/permission_scraper.py)
– Writes MHTML to data/raw/
c. Processing Layer
– JSONProcessor (src/processing/json_processor.py)
– DataUtils (src/utils/data_utils.py)
– Transforms raw MHTML → structured JSON/CSV in data/input/
d. Reference Data Store
– data/input/*.csv (user-permission reference data)
e. LLM Analysis Layer
– LLMServices (src/llms/*_classifier.py + *_evaluator.py)
– Categories: risk, cloud, description, user-perm classification
– Prompt templates (src/prompts/templates/*.md) drive the LLM chains
f. Output Layer
– Writes final classified results to data/output/
– Notebooks (Notebooks/*.ipynb) consume output for visualization
g. Configuration & Auxiliary
– config/environment.template.yml (env variables, API keys)
– requirements.txt (dependencies)
– tests/ (src/scraping and other modules)
Architectural Patterns & Principles
• Layered Separation: ingestion → processing → analysis → presentation
• Single-responsibility modules (one scraper, one processor, one classifier)
• Externalize configuration (YAML/.env) and prompts (markdown templates)
• Pluggable LLM chains (easy to add new classifiers or evaluators)
Detail Level
• Prompt subfolder and template names if prompt orchestration is a key architectural concern