How aspnet/benchmarks works
Project type and purpose
• This is a performance-benchmarking toolset for ASP.NET Core, driven by Microsoft Crank, with many pre-built web app scenarios, network streaming tests, and database benchmarks.
• It is not a single full-stack application but rather a container-orchestrated benchmark harness.
File structure
• build/azurepipeline-*.yml & .github/workflows: CI definitions for building images and running benchmarks on Azure or GitHub Actions
• docker/: Dockerfiles & configs for proxies (Envoy, HAProxy, Nginx), databases (Postgres, MySQL, MongoDB, SQL Server, Redis), service-bus controller
• scenarios/: YAML scenario definitions consumed by Crank
• scripts/: helper scripts for pipelines or load tests
• src/Benchmarks: the core web harness (Controllers, Middleware, EF Core, Dapper, raw SQL)
• src/BenchmarksApps, src/System.Net.Benchmarks, src/TcpClient, src/WebsocketClient, etc.: individual micro-apps and clients for each scenario
README
• Confirms use of Microsoft Crank as orchestrator
• Scenarios are defined in scenarios/ and referenced in app-specific YAML files
• CI pipelines invoke these scenarios as part of continuous benchmarking
Main components and interactions
a. Build & CI layer
– GitHub Actions / Azure Pipelines (build/ & .github/workflows/)
– Produce Docker images for:
• Benchmark harness (src/Benchmarks)
• BenchmarkApps (various feature-specific web apps)
• Proxies & Databases (docker/)
• System.Net streaming servers & clients
– Push images to a container registry or keep locally
b. Orchestration layer
– Microsoft Crank CLI (invoked via scripts or CI jobs)
– Reads scenario definitions (.yml in scenarios/ and src/**/benchmarks.yml)
– Decides which containers to launch and their interconnections
c. Runtime / Docker layer
– Docker daemon (single host or multiple hosts)
– Containers by role:
• Clients (HttpClient, GRPC, System.Net streaming, WebsocketClient, TcpClient)
• Load-balancers & proxies (Envoy, HAProxy, Nginx)
• Servers (Kestrel-hosted ASP.NET apps from BenchmarksApps, System.Net servers, service-bus controller)
• Databases (Postgres, MySQL, MongoDB, SQL Server, Redis)
– All containers attached to a common network overlay
d. Metrics & results
– Crank collects performance counters, latencies, throughput
– Aggregates results into CSV, JSON, or publishes to Azure storage/Grafana
Architectural patterns and design principles
• Containerized micro-services: each test app or database runs in its own container
• Scenario-driven orchestration: YAML defines client-server-db topologies
• Separation of concerns: build pipelines vs runtime orchestration vs metrics collection
• Reusability: common Benchmark harness, shared Directory.Build.props / targets