How webassembly/binaryen works
Project Type and Purpose
• Binaryen is a C++ compiler‐infrastructure library and toolchain for WebAssembly.
• It provides:
• A core IR (intermediate representation) & optimizer (many passes)
• A set of CLI tools (wasm-opt, wasm-as, wasm-dis, wasm2js, wasm-merge, etc.)
• C API header, JavaScript API (binaryen.js)
• Testing & fuzzing harness (check.py, lit, Python scripts)
High‐Level Architectural Layers
• Presentation / Interfaces (CLI & APIs)
– bin/ executables (wasm-opt, wasm-as, wasm-dis, wasm2js, wasm-shell, etc.)
– C API header: src/binaryen-c.h
– JavaScript API: src/js, templates, binaryen.js on npm
• Core Library
– IR definition & data structures (src/ir, src/wasm/*.h)
– Analysis & optimization passes (src/analysis, src/passes)
– Stack IR & flat IR (src/wasm-stack.h, src/ir/flat.h)
– Pass manager & utilities (src/pass.h, src/support, src/compiler-support.h)
• Backends / Code Generators
– WebAssembly emitter (src/wasm/wasm-binary.cpp, wasm-validator.cpp, wasm-io.cpp)
– JS emitter for wasm2js (src/wasm2js.h, src/js/binaryen.js-post.js, templates)
• Frontends / Parsers
– .wat / .wast parser (src/parser/*.cpp)
– CFG API (analysis/cfg.cpp)
• Testing & Tooling
– Tests: test/ (unit, integration, spec tests, lit runner)
– Fuzzing: scripts/fuzz_opt.py, scripts/clusterfuzz
– Build scripts: CMakeLists.txt, scripts/auto_update_tests.py
Main Components & Their Relationships
• CLI Tools (bin/wasm-opt, wasm-as, wasm-dis, wasm2js, etc.)
↓ invoke
• Core Library
• Parser → produces AST/IR
• IR (Tree IR + Stack IR)
• Analysis modules (dataflow, CFG, lattices)
• Pass Manager → sequences optimization/transformation passes
• Pass modules (CoalesceLocals, DeadCodeElimination, ReReloop, etc.)
• Code Generators
– Wasm binary writer
– JS emitter (wasm2js)
• C API & JS API wrapper around library
• Support utilities (command-line parsing, file I/O, JSON, threading)
• Third‐Party Submodules
• googletest (unit tests)
• LLVM components (DWARF parsing, LEB128, etc.)
• WABT, SpiderMonkey, V8 for test shells
↓ used by
• Testing & Fuzzing Harness
• check.py / Python scripts → orchestrate CLI tools + interpreters
• lit (LLVM Integrated Tester) suites
• clusterfuzz / fuzz-opt
Architectural Patterns & Principles
• Layered architecture: CLI → Library → IR → Codegen
• Modular pass framework: Pass Manager + plug-in style passes
• Arena allocation & interned strings (support/utilities)
• Extensions via submodules (LLVM, WABT, fuzztest)
Essential Elements to Capture
• Parser (src/parser)
• IR definitions (src/ir, src/wasm)
• Analysis & Dataflow (src/analysis, src/dataflow)
• Pass Manager & Pass modules (src/pass.h, src/passes)
• Code generators (Wasm binary, wasm2js)
• CLI tools (wasm-opt, wasm-as, wasm-dis, wasm2js, wasm-merge, wasm-shell, wasm-ctor-eval, etc.)
• C API & JS API layers
• Support utilities (src/support)
• Testing & fuzzing harness and third-party submodules