How nlpsast/smartsast works
Project type and purpose
• Project Type: A local, AI-powered Static Application Security Testing (SAST) tool
• Purpose: Analyze source code (e.g., Java, Python, etc.) in a privacy-first way by running entirely in Google Colab. It loads your code and a local Retrieval-Augmented Generation (RAG) dataset of CWEs, invokes a Large Language Model (LLM) to spot vulnerabilities, maps results to CWE entries, and emits a JSON report.
Top-level file structure analysis
• smart_sast_v1_0.py – the main Python driver that orchestrates scanning
• CWE-top25-…zip – the RAG dataset of CWE examples and explanations
• Sample/ – example code snippets (e.g., CWE-434.java, CWE89.java)
• README.md, LICENSE, CONTRIBUTING.md – docs, no additional runtime components
Key elements from README
• Runs in Google Colab with no external API calls (LLM runs locally or via integrated Colab GPU/TPU)
• Uses LLMs + RAG dataset to increase accuracy over rule-based scanners
• Produces structured JSON output for vulnerability findings
Main components
a. User environment
– Google Colab notebook (Python runtime + GPU/TPU)
– User workspace (drive-mounted folders: RAG_FOLDER, OUTPUT_FOLDER, project files)
b. Input sources
– Source code files (Sample/ or user-provided repo)
– RAG dataset (unzipped CWE-top25…zip)
c. SmartSast engine (smart_sast_v1_0.py), logically subdivided into:
1. Code Loader & Parser
• Discovers code files by extension
• Reads file contents
2. Prompt Generator & RAG Retriever
• Retrieves relevant CWE examples from RAG_FOLDER
• Assembles LLM prompts combining code snippet + CWE context
3. LLM Invocation Module
• Calls LLM (e.g., Hugging Face model or Colab local LLM)
• Handles batching, retry, temperature settings
4. Response Processor & CWE Mapper
• Parses LLM outputs to extract vulnerability descriptions
• Maps to CWE IDs and severity
5. JSON Report Writer
• Serializes findings to OUTPUT_FOLDER/vulnerabilities.json
d. Outputs
– JSON vulnerability report
– Optional logs or verbose console output
Relationships and data flows
1. User drops source code & RAG data into Colab workspace
2. smart_sast_v1_0.py loads code and CWE examples
3. Prompt Generator requests relevant RAG passages for each code snippet
4. LLM Invocation sends prompts and returns LLM responses
5. Processor extracts vulnerabilities, associates CWE metadata
6. Report Writer writes JSON to OUTPUT_FOLDER
7. User retrieves JSON for review
Architectural patterns & principles
• Single-process, pipeline-style architecture
• Clear separation of concerns: I/O (loader/writer), data enrichment (RAG), AI calls, post-processing
• RAG augments LLM prompts to improve precision
• Environment-driven configuration (RAG_FOLDER, OUTPUT_FOLDER)
Detail without over-splitting
• Every logical step in the SAST pipeline
• Do not fragment trivial utility functions—group them under their module