huggingface/repo2rlenv

1. What this repo is

Repo2RLEnv is a Python package and CLI that turns a source repository into a Harbor-formatted RL dataset. The core architectural boundary is between repo mining / synthesis on one side and Harbor-compatible task emission on the other.

2. Top-level orchestration

The repo2rlenv CLI (src/repo2rlenv/cli.py) is the entry point for the main workflows:

- generate to build tasks from a repo

- bootstrap to prepare a runnable sandbox image for runtime-based pipelines

- validate to check generated artifacts

- push / pull to publish and retrieve datasets

- auth checks and supporting commands

This CLI delegates into the pipeline, bootstrap, registry, and source layers rather than implementing generation logic directly.

3. Pipeline engine

The src/repo2rlenv/pipelines/ package is the synthesis layer. Pipelines implement a common Pipeline contract and each writes a task set under an output directory. The repo supports six named pipelines:

- stable: pr_diff, pr_runtime

- experimental: commit_runtime, cve_patches, code_instruct, equivalence_tests

Shared helpers in _eval_script.py, _function_extractor.py, _oss_instruct.py, _pr_diff_verifier.py, and _pr_runtime_verifier.py indicate that the pipelines reuse common extraction, prompting, and verification logic.

4. Main data flow

The dominant flow is:

source repository → pipeline-specific mining/extraction → task synthesis → verifier/reward materialization → Harbor task directory

Each task is content-addressed and carries provenance metadata, including pipeline identity and source commit/PR references. That metadata is what makes the output reproducible and verifiable.

5. Source repository abstraction

The sources.py, git_local.py, github.py, and gitlab.py modules define the input boundary. Pipelines can read from:

- local git checkouts

- GitHub repositories

- GitLab repositories

Some pipelines also need commit/PR APIs or OSV data, so the source layer is where those external constraints are enforced. This is important architecturally because not every pipeline accepts every source type.

6. Bootstrap subsystem for runtime pipelines

Runtime pipelines depend on src/repo2rlenv/bootstrap/. This is a separate subsystem that builds a per-repo Docker image by iterating commands in a container until the project builds and tests collect/pass. Key pieces include:

- Docker/image orchestration

- language-specific setup

- agent loop and prompt/preset logic

- caching and spec tracking

The bootstrap output is cached by repo/ref, so expensive setup happens once and is reused by downstream tasks.

7. Verification and reward generation

A central design point is that tasks are verifiable. Reward logic lives in src/repo2rlenv/reward.py, while runtime and diff verification are supported by pipeline-specific verifiers and log parsers. The log_parsers/ package contains parsers for pytest, Jest, Cargo, and Go output, which is how test-execution rewards are normalized across ecosystems. This is the boundary where “the agent’s output” becomes a machine-checkable score.

8. Output format and Harbor integration

The repo does not define its own runtime format; it emits Harbor-compatible datasets. src/repo2rlenv/emitter/harbor.py is the publishing/export layer, and the README notes that tasks include a small metadata.repo2env provenance block. That means the output boundary is:

- internal pipeline artifacts and task metadata on one side

- Harbor dataset structure, reward files, and registry metadata on the other

9. Registry, push/pull, and external services

The registry/ package handles publishing concerns: naming, visibility, auth, integration, probing, and push behavior. There are adapters for cloud/container registry environments (ecr.py, gar.py), which suggests the project can integrate with external registries when packaging runnable artifacts. Separately, hub.py, auth.py, and provider.py mediate authentication and interaction with Hugging Face Hub, GitHub, and LLM providers. These are distinct from synthesis: they are the distribution and credential boundary.

10. LLM usage is isolated by phase

The architecture deliberately scopes LLM calls to specific phases:

- some pipelines use an LLM during synthesis

- runtime pipelines may use an LLM during bootstrap

- certain verifiers may use an LLM during verification/judging

This separation matters because it keeps most generated artifacts deterministic and cacheable, while still allowing semantic scoring where needed.

11. User-facing inspection and live feedback

src/repo2rlenv/ui/ provides console/live views for generation and bootstrap progress. This is a presentation layer only; it does not change generation semantics. It exists to surface long-running pipeline and bootstrap state to the user during dataset creation.

12. Quality and release boundary

The tests/ tree is broad and maps closely to the architecture: pipeline contracts, bootstrap behavior, source integrations, registry behavior, reward/grading, log parsing, and end-to-end hub flows. CI and release workflows in .github/workflows/ reinforce that the repo’s main invariant is not “the package imports,” but “a generated dataset is valid, reproducible, and publishable.”

13. How to draw the diagram

A good architecture diagram would show:

- CLI/API entrypoints

- Source adapters feeding into

- Pipeline engine

- branching to bootstrap for sandboxed pipelines and verifiers/log parsers for scoring

- then converging into Harbor emitter

- then into registry/Hugging Face Hub push-pull

- with auth/provider integrations and LLM services as external dependencies

The key conceptual split is between dataset synthesis and dataset distribution.