How noahshinn/reflexion works
Project Type and Purpose
• This is a research‐oriented, open-source toolkit for running “reflexion” experiments on multiple decision-making and reasoning tasks (AlfWorld, HotPotQA, programming benchmarks, Webshop).
• It provides configurable pipelines that instantiate agents, environments/datasets, prompts, LLM interfaces, self-reflection loops, and logging for empirical evaluation of verbal reinforcement learning strategies.
High-Level Architectural Overview
The system consists of:
A. Experiment Domains (top-level subfolders)
• alfworld_runs
• hotpotqa_runs
• programming_runs
• webshop_runs
Each domain is a concrete instantiation of the Reflexion framework for a particular task.
B. Core Framework Components (reused across domains)
1. Orchestrator
– main.py (or Jupyter notebook)
– run_*.sh scripts
– configuration (YAML, command-line args)
2. Agent Module
– Defines agent classes and strategies (ReAct, CoT, simple, reflexion)
– agents.py, react.py, fewshots.py, prompts.json
3. Environment / Data Module
– wraps task environments or loads datasets
– environment.py, data loaders, mocks
4. LLM Interface
– llm.py
– wrappers over OpenAI API (or other LLM backends)
5. Prompt & Reflection Manager
– Prompt templates (JSON, TXT)
– generate_reflections.py, utils.py
– Implements self-reflection context augmentation
6. Logging & Storage
– root/<run_name> directories containing trial logs (JSON, .joblib)
– utils for saving history, results, metrics
C. External Services & Tools
• OpenAI API (or any LLM endpoint)
• Local environment simulators (AlfWorld Gym)
• Jupyter for notebooks
Relationships and Data Flows
• Orchestrator → (reads) Configuration → spawns → Agent & Environment
• Agent → (sends) prompt & context → LLM Interface → (receives) reasoning/output
• Agent → (executes) in Environment → (returns) reward/state
• Agent → (invokes) Reflection Manager → (updates) prompt/context for next trial
• All modules → (emit) logs → Logging & Storage
Architectural Patterns & Design Principles
• Pipeline pattern: sequential stages (orchestration → agent → LLM → environment → reflection → logging)
• Modular & pluggable components: swap in new agent types, environments, or reflexion strategies
• Separation of concerns: distinct modules for data, logic, I/O, reflection
• Config-driven design: experiments parameterized by YAML and shell scripts
Technologies and Frameworks
• Python 3.x as core language
• YAML for configuration
• JSON/.joblib for logs & model checkpoints
• Jupyter Notebooks for interactive querying and visualization
• Shell scripts for pipeline execution
• OpenAI API as LLM backend
• AlfWorld Gym for environment simulation