How ethereum/tests works
Project type and purpose
• This is an open-source test suite (“Ethereum Execution Tests (EVM)”) not a typical application. It provides canonical test vectors (JSON, YAML) for EVM behavior, plus “fillers” that generate or transform tests, ansible scripts for provisioning, and documentation. Clients (Geth, Parity, etc.) consume the test data via the retesteth test harness.
File structure
• Top-level directories:
• BasicTests, BlockchainTests, TransactionTests, EOFTests, RLPTests, PoWTests, etc. → static test specs
• src/…Filler → scripts and templates (JavaScript/Node.js, YAML) to generate test vectors
• JSONSchema → defines schemas for validating test JSON
• ansible/ → infrastructure-as-code for spinning up environments (Docker, EC2) to run tests
• docs/ → Sphinx/RST docs about test formats, fillers, usage
• Config files: requirements.txt (Python deps), .travis.yml (CI), .gitmodules, Vagrantfile.
README for additional insights
• The repo is consumed by a variety of client implementations via retesteth.
• Tests are version-tagged; the “src” fillers auto-generate canonical tests, so one must maintain both raw and generated tests.
• Ansible + Vagrant + Docker are used to run retesteth across multiple clients and collect stats.
Elements
• Clear component labels (e.g., “Filler Generator”, “Schema Validator”, “retesteth”)
• “Templates → Filler → Test JSON”
• “Test JSON → retesteth → Ethereum Client”
• “Client Results → Hive → Web UI”