How deadbits/vigil-llm works
Main Components
• API Server: Represent the REST API layer (e.g., the entry point in vigil-server.py) that receives and handles incoming HTTP requests for analysis.
• Core Engine (Library): Include the main Vigil core (the vigil/vigil.py and related modules) responsible for orchestrating analyses. This component instantiates and coordinates all scanners.
• Configuration Layer: Include a component for configuration (conf folder, such as server.conf and docker.conf) that feeds settings into both the core engine and scanners.
• External Dependencies & Integration Points: Optionally include external services such as vector database services for embeddings, open-source libraries (like SentenceTransformers for embeddings, OpenAI API, YARA installation), and even the optional Streamlit-based web UI interface (streamlit_app.py).
Relationships and Interactions
• The API Server receives incoming requests (e.g., /analyze/prompt, /analyze/response, /canary/add, /canary/check) and routes them to the Vigil core.
• Each scanner interacts with underlying data/signature files (from the data folder) and may access external libraries or resources for computing embeddings or performing pattern matching.
• Configuration flow: the conf files (e.g., conf/server.conf) are read at startup by the API server and core engine to determine settings for scanners, model selections, and database connections.
Architectural Patterns and Principles
• A modular and extensible architecture where each scanner module is developed independently (as seen in the vigil/scanners directory).
• Separation of concerns (API handling, core orchestration, detection modules, data loading/configuration).
• That the design supports both library usage (for integration in other Python apps) and standalone REST API serving.
• Optionally, mention the layered security pattern: multiple detection approaches (heuristics, embeddings, token-based) to defend against prompt injection vulnerabilities.
Relevant Technologies, Frameworks, or Libraries
• Python as the main programming language.
• YARA (for heuristics scanning).
• Machine learning libraries or pre-trained transformer models for text similarity/injection detection.
• Vector database or embedding generation tools (e.g., sentence-transformers) for similarity-based detection.
• REST API frameworks (evident from the Python server script, though it may be lightweight and custom built).
• Docker for containerization (supported by both Dockerfile and the conf/docker.conf file).
• Streamlit for the optional web UI (streamlit_app.py).