How vndee/llm-sandbox works
Project type and purpose
• LLM Sandbox is an open‑source Python library (not a web app) whose goal is to let developers and AI frameworks safely run arbitrary code generated by large language models inside isolated container sandboxes (Docker, Kubernetes, Podman).
File structure
• Top‑level modules under llm_sandbox/:
• base.py – defines the abstract SandboxSession interface and lifecycle
• factory.py – selects and instantiates the correct backend implementation
• session.py – implements the context‑manager that orchestrates open/run/close
• docker.py, kubernetes.py, podman.py, micromamba.py – concrete backends for each container runtime
• security.py, utils.py, const.py, exceptions.py – supporting utilities, constants, and error classes
• examples/ shows how clients and AI frameworks (Langchain, LlamaIndex) integrate by calling SandboxSession
• tests/ exercise each backend to validate session open/run/close
README for additional insights
• Emphasizes four lifecycle steps (init → open → run → close)
• Lists supported languages (py, js, java, go, cpp, ruby)
• Mentions optional features (custom Dockerfiles, remote Docker host, Kubernetes manifests, Podman)
a. Main components (group into logical tiers)
• Client Layer:
– User application code (including examples or LLM‑agent code)
– AI Frameworks (Langchain, LlamaIndex) that invoke run_code()
• Library Core:
– SandboxSession (context manager)
– Factory module (decides backend)
– Abstract base session (defines open/run/close)
– Backend implementations: DockerBackend, K8sBackend, PodmanBackend, MicromambaBackend
– Utilities: security scanners, helpers, exception handlers
• Container Platform Layer:
– Docker daemon (local or remote via TLS)
– Kubernetes cluster (CoreV1Api client + pod_manifest)
– Podman service
• Sandbox Runtime Environment (ephemeral container or pod)
– Container image (official or custom Dockerfile)
– Mounted volumes, network namespace, exec calls
• Host OS / Underlying infrastructure (actual nodes, file system)
b. Relationships and interactions
1. Client → SandboxSession.open(): calls Factory → selects appropriate backend
2. Backend.open():
– builds/pulls image → starts container/pod
3. Client → SandboxSession.run(code, libs):
– backend.exec() → sends code to container’s shell/entrypoint → returns stdout/stderr
4. Client → SandboxSession.close():
– backend.clean(): stops & removes container (and image if keep_template=False)
5. Optional: Client ↔ session.copy_to_runtime()/copy_from_runtime() for file transfers
c. Architectural patterns & design principles
• Factory pattern: decouple core API from container runtime implementations
• Strategy pattern: each backend implements a common interface
• Context manager for deterministic resource cleanup
• Single Responsibility: separate modules for orchestration (session), platform adapters, security, and utilities
d. Key technologies, frameworks & libraries
• Docker SDK for Python
• Kubernetes Python client (CoreV1Api)
• Podman Python bindings
• Micromamba for language‑specific environment management (optional)
• Python typing, exceptions, utils (JSON, logging)