LFDT - Continuous Benchmarking for Besu EVM

About the Organization

<p>Besu includes comprehensive EVM benchmarking tooling — including JMH microbenchmarks for 40+ EVM operations, precompile benchmarks via&nbsp;<code style="background-color: rgba(129, 139, 152, 0.12);">evmtool benchmark</code>, and full blockchain-level test execution via&nbsp;<code style="background-color: rgba(129, 139, 152, 0.12);">evmtool block-test</code>. However, these benchmarks are currently only run manually by developers on local machines. There is no automated way to track performance over time, detect regressions introduced by PRs, or visualize trends.</p><p><br></p><p>This project aims to build a continuous benchmarking pipeline for the Besu EVM, for example by integrating existing benchmark tooling into CI. The mentee will discuss a solution with Besu maintainers, design and implement automated (and/or manually triggered) workflows that run EVM benchmarks, collect results, store historical data, and produce reports that give maintainers clear visibility into performance changes.</p><p><br></p><p>An excellent reference for the kind of output we'd like to achieve is the&nbsp;<a href="https://benchmarks.ethrex.xyz/gas-benchmark-eest/" rel="noopener noreferrer" target="_blank" style="color: rgb(9, 105, 218);">ethrex gas benchmarks dashboard</a>, which provides comparative benchmark reports with sortable tables, speedup metrics, and historical tracking across runs.</p><p><br></p><p>Besu's existing benchmark tooling includes:</p><ul><li>JMH microbenchmarks&nbsp;(<code style="background-color: rgba(129, 139, 152, 0.12);">ethereum/core/src/jmh/</code>) — 40+ benchmarks covering EVM arithmetic, bitwise, shift, stack, memory, and crypto operations (reported in execution time only)</li><li>Precompile benchmarks&nbsp;(<code style="background-color: rgba(129, 139, 152, 0.12);">evmtool benchmark</code>) — benchmarks for AltBN128, EcRecover, ModExp, BLS12, P256Verify, SHA256, RIPEMD-160, and KZG Point Eval with throughput metrics (MGps)</li><li>Block-test execution&nbsp;(<code style="background-color: rgba(129, 139, 152, 0.12);">evmtool block-test</code>) — full blockchain reference test execution with timing and pass/fail reporting</li><li>Shell-based benchmark scripts&nbsp;(<code style="background-color: rgba(129, 139, 152, 0.12);">ethereum/evmtool/src/test/benchmarks/</code>) — existing scripts for static operations, loop operations, and mainnet-like contract execution</li></ul><p><br></p><p>The project scope could include:</p><ol><li>CI Workflow Design&nbsp;— Create GitHub Actions workflows (manually triggered in the first instance) that run selected benchmarks on consistent hardware.</li><li>Results Collection and Storage&nbsp;— Design a scheme for storing benchmark results over time (e.g., GitHub Pages, artifact storage, or a lightweight results repo).</li><li>Reporting and Visualization&nbsp;— Find/build a dashboard or report format that shows performance trends, highlights regressions, and allows comparison between runs.</li><li>PR Integration&nbsp;— Optionally, enable benchmark runs on PRs that touch EVM-critical code paths, with summary comments showing performance impact.</li><li>Documentation&nbsp;— Document how to run, extend, and interpret the benchmarks.</li><li>RPC Performance&nbsp;- As a stretch goal could also cover RPC performance, such as eth_call.</li></ol><p><br></p><p>Lean more at <a href="https://github.com/LF-Decentralized-Trust-Mentorships/mentorship-program/issues/69" rel="noopener noreferrer" target="_blank">https://github.com/LF-Decentralized-Trust-Mentorships/mentorship-program/issues/69</a></p>