How codium-ai/alphacodium works
Project type and purpose
• This is a command-line, ML-driven code-generation pipeline (AlphaCodium) for competitive-programming problems.
• Purpose: orchestrate a multi-stage “flow engineering” process: read problem, generate candidate solutions via LLMs, run tests, self-reflect, fix code, choose best solution, and evaluate end-to-end.
File structure
• Top-level entrypoints (scripts invoked via python -m):
• solve_problem
• solve_dataset
• evaluate_dataset
• solve_my_problem
• alpha_codium/: core package
• settings/: config loader + .toml files
• llm/ & litellm/: abstraction over LLM calls (OpenAI API, proxy, token handling)
• gen/: “generation” pipeline, with stages under gen/stages/ (initial code gen, tests, fix, reflection, choose best, etc.)
• code_contests/: dataset provider, data prep, eval metrics, local execution runner
• log/: logging utilities
• tests/: unit tests for core eval and local execution
• docs/: MkDocs site for project documentation
• Ops files: Dockerfile, GitHub Actions CI, requirements.txt
README insights
• Multi-stage flow: solve → generate tests → run tests → fix code → reflect → choose best → repeat
• Configuration-driven: stages and model choice controlled via TOML
• DataProvider reads CodeContests dataset from filesystem
• Evaluation module computes pass@k and other metrics
• CLI Layer (entrypoint scripts)
• Config Loader (settings/config_loader.py)
• Generation Engine (gen/ package + pipeline orchestrator)
• LLM Service Adapter (llm/ and litellm/)
• Code Execution & Test Runner (code_contests/eval/local_exec.py)
• Dataset & Storage (CodeContests JSON files on disk)
• Evaluation & Metrics (code_contests/eval/, evaluate_dataset CLI)
• Logging & Artifacts (log files, outputs)
• External Services: OpenAI API
1. CLI → Config Loader → Generation Engine
2. Generation Engine → LLM Adapter → OpenAI API → LLM Adapter → Generation Engine
3. Generation Engine → Test Runner → Execution Sandbox → Results → Generation Engine
4. Generation Engine → Logging & File System (candidate code, logs)
5. CLI (evaluate_dataset) → Evaluation Module → Metrics → Logging/File System
c. Architectural patterns & principles:
• Pipeline pattern: each generation stage is a pluggable module (gen/stages/*.py)
• Configuration-driven: stages, prompts, model names defined in TOML
• Abstraction layer for LLM calls (ai_invoker, ai_handler)
• Separation of concerns: data prep, code generation, evaluation, logging
d. Key technologies/frameworks:
• Python 3.8+, TOML config, Pydantic-style structured output prompts
• OpenAI API (via litellm.proxy)
• Docker for reproducible environment
• GitHub Actions for CI
• MkDocs for documentation
Diagramting elements
• Labels: clear component names (e.g., “LLM Adapter”)
• Blue rectangles for internal modules
• Orange parallelograms for CLI scripts