dheerajdamala/probing

How dheerajdamala/probing works

ISAE is a research toolkit for learning security representations, probing code-model activations, steering generation, and evaluating security and functional correctness. Its principal documented workflow prepares paired security data, loads a Mistral model, extracts representations and steering vectors, optionally intervenes during generation, then evaluates generated code on separate security and functional measures. A substantial companion SQL-injection analyzer adds project-level static analysis and report generation. The README documents the research capabilities; source-index references and excerpts support the analyzer connections, but several broader pipeline links are documented rather than verified from full implementations.

Main entry points: pipeline.py Analysis Pipeline · load_model.py Mistral Loader · steered_generate.py Steered Generation · extract_security_vectors.py Security Vectors

Components 31

Research Data

Dataset Download src/data/download_datasets.py
Dataset Preparation src/data/prepare_dataset.py
Dataset Validation src/data/validate_dataset.py
SQL Sink Extraction experiments/sqli_probe_v2/extraction/ast_sink_extractor.py

Model Runtime

Mistral Loader src/model/load_model.py
Activation Extraction src/model/extract_activations.py

Probing and Steering

Security Vectors src/steering/extract_security_vectors.py
Semantic Probe src/probes/train_semantic_probe.py
Probe Evaluation src/probes/evaluate_semantic_probe.py
Steering Hook src/steering/activation_hook.py
Steered Generation src/steering/steered_generate.py
Vector Evaluation src/steering/evaluate_vectors.py
Single-Sample Probe security_probe/src/inference.py
Probe Model security_probe/src/probe_model.py

Evaluation

Security Evaluation src/evaluation/security_eval.py
Functional Evaluation src/evaluation/functional_eval.py
Benchmark Runner src/evaluation/run_benchmarks.py

SQL Analysis

Project Input experiments/sqli_probe_v3_full_mistral/extension/input_handler.py
Analysis Pipeline experiments/sqli_probe_v3_full_mistral/extension/pipeline.py
SQL AST Analysis experiments/sqli_probe_v3_full_mistral/extension/ast_analyzer.py
Query Dataflow experiments/sqli_probe_v3_full_mistral/extension/query_dataflow.py
Interprocedural Analysis experiments/sqli_probe_v3_full_mistral/extension/interprocedural_analyzer.py
Project AST Graph experiments/sqli_probe_v3_full_mistral/extension/project_ast_graph.py
Representation Extraction experiments/sqli_probe_v3_full_mistral/extension/representation_extractor.py
Probe Prediction experiments/sqli_probe_v3_full_mistral/extension/probe_predictor.py
Vulnerability Classification experiments/sqli_probe_v3_full_mistral/extension/vulnerability_classifier.py
Evidence Builder experiments/sqli_probe_v3_full_mistral/extension/evidence_builder.py
Report Generation experiments/sqli_probe_v3_full_mistral/extension/report_generator.py
Frozen AST Analysis experiments/sqli_probe_v3_full_mistral/extension_v1_frozen/ast_analyzer.py
Frozen Interprocedural Analysis experiments/sqli_probe_v3_full_mistral/extension_v1_frozen/interprocedural_analyzer.py

Other components

Researcher

Connections 36

Researcher → Dataset Download: requests data README.md
Researcher → Analysis Pipeline: submits code
Dataset Download → Dataset Preparation: supplies datasets README.md
Dataset Preparation → Dataset Validation: prepares splits README.md
Researcher → Mistral Loader: starts experiments README.md
Mistral Loader → Activation Extraction: loads model for src/model/extract_activations.py
Mistral Loader → Semantic Probe: loads model for src/probes/train_semantic_probe.py
Mistral Loader → Steering Hook: provides model to src/steering/activation_hook.py
Activation Extraction → Semantic Probe: extracts training features src/probes/train_semantic_probe.py
Semantic Probe → Probe Evaluation: trains probe for README.md
Researcher → Security Vectors: runs vector extraction README.md
Researcher → Steered Generation: requests generation README.md
Steering Hook → Steered Generation: intervenes during README.md
Security Vectors → Steering Hook: provides direction README.md
Security Vectors → Vector Evaluation: evaluates vectors README.md
Steered Generation → Security Evaluation: checks vulnerabilities README.md
Steered Generation → Functional Evaluation: runs functional tests README.md
Researcher → Benchmark Runner: runs benchmarks README.md
Benchmark Runner → Security Evaluation: runs security tests README.md
Benchmark Runner → Functional Evaluation: runs functional tests README.md
Analysis Pipeline → Project Input: processes input experiments/sqli_probe_v3_full_mistral/extension/pipeline.py
Analysis Pipeline → SQL AST Analysis: runs AST analysis experiments/sqli_probe_v3_full_mistral/extension/pipeline.py
Analysis Pipeline → Query Dataflow: traces query flow experiments/sqli_probe_v3_full_mistral/extension/pipeline.py
Analysis Pipeline → Interprocedural Analysis: runs cross-file analysis experiments/sqli_probe_v3_full_mistral/extension/pipeline.py
Analysis Pipeline → Representation Extraction: extracts representations experiments/sqli_probe_v3_full_mistral/extension/pipeline.py
Analysis Pipeline → Probe Prediction: requests prediction experiments/sqli_probe_v3_full_mistral/extension/pipeline.py
Analysis Pipeline → Vulnerability Classification: classifies findings experiments/sqli_probe_v3_full_mistral/extension/pipeline.py
Analysis Pipeline → Evidence Builder: builds evidence experiments/sqli_probe_v3_full_mistral/extension/pipeline.py
Analysis Pipeline → Report Generation: generates report experiments/sqli_probe_v3_full_mistral/extension/pipeline.py
Interprocedural Analysis → Project AST Graph: uses project graph experiments/sqli_probe_v3_full_mistral/extension/interprocedural_analyzer.py
Interprocedural Analysis → SQL AST Analysis: uses AST analysis experiments/sqli_probe_v3_full_mistral/extension/interprocedural_analyzer.py
Representation Extraction → SQL Sink Extraction: uses sink regions experiments/sqli_probe_v3_full_mistral/extension/representation_extractor.py
Single-Sample Probe → Probe Model: scores representations security_probe/src/inference.py
Researcher → Single-Sample Probe: analyzes code
Frozen Interprocedural Analysis → Frozen AST Analysis: uses AST analysis experiments/sqli_probe_v3_full_mistral/extension_v1_frozen/interprocedural_analyzer.py
Frozen Interprocedural Analysis → Project AST Graph: references graph module experiments/sqli_probe_v3_full_mistral/extension_v1_frozen/interprocedural_analyzer.py

Drawn by GitDiagram on November 9, 2026 from the repository's file tree, README and source files. Read as Markdown ·