How divya-pal4/code-review-openenv works
Code Review OpenEnv is a gym-style benchmark for agents to review code snippets and receive task-specific feedback and scores. The principal workflow is: an agent resets the environment, receives a task observation, submits a review comment, and gets the next observation and grade. The baseline inference client optionally generates comments with an external language model. The README describes test-based grading, but the sampled grader implements keyword matching only; the excerpts do not show hidden-test execution.
Main entry points: app.py REST API · environment.py Task environment · inference.py Baseline client · grader.py Keyword grader
Components 10
Agent clients
- Baseline client
inference.py
Review environment
- Task environment
env/environment.py - Review tasks
env/environment.py - Action and observation models
env/models.py
Scoring
- Keyword grader
env/grader.py - Task grader adapters
graders
Other components
- Review agent
- Language model
Connections 10
- Review agent → REST API: uses REST
README.md - Baseline client → REST API: calls reset and step
inference.py - Baseline client → Language model: requests comments
inference.py - App shim → REST API: exports app
app.py - REST API → Task environment: resets and steps
server/app.py - REST API → Task environment: reads state
server/app.py - REST API → Keyword grader: grades action
server/app.py - Task environment → Action and observation models: constructs observations
env/environment.py - Task environment → Review tasks: selects task
env/environment.py - Task grader adapters → Keyword grader: delegates grading
graders/task_1.py