@njelich.near [Posted on DevHub](/devgovgigs.near/widget/Post?id=2558) ## Solution: evolutionizing NEAR Protocol Testing: 'wasmcov' - A First in Wasm Coverage ###### Requested amount: 15000 USDT ###### Requested sponsor: @neardevdao.near ## Overview Hey there, it's [Noah](https://www.linkedin.com/in/njelich/)! I'm the creator of [wasmcov](https://github.com/hknio/wasmcov), the pioneering code coverage solution designed specifically for WebAssembly in embedded, limited, and blockchain environments. Collaborating with the NEAR team has me really excited about bringing this technology to the NEAR Protocol. Let's tackle the problem first: Wasm, although widely used for smart contracts, lacks support from existing coverage solutions. This poses a significant challenge. Automated testing is vital in software development, but without reliable tools, bugs slip through the cracks. This issue is especially critical in blockchain environments, where bugs can lead to severe consequences. To clarify, Rust does have coverage solutions, but they don't apply to Wasm running on the NEAR VM (or any other really), failing to test the actual code slated for deployment. For instance, you might have smart contracts with #[payable] functions and think they've been thoroughly tested because you've written tests that call them and even measured coverage using something like `cargo tarpaulin`. But you're not testing the actual code slated for deployment. You're testing the code in a test/debug environment, not on contracts bound for release. Many NEAR macros trigger only when compiling for the wasm32-unknown-unknown target. So while you're testing raw functions, you're not evaluating the final product at all. In scenarios involving payable functions, everything might seem fine locally because these macros don't execute. But once you deploy those contracts and start using funds, you might encounter bug reports or find everything seemingly broken, with no clue where the issue lies. Traditional Rust coverage commands don't provide clarity; all the code appears covered by tests. Some of you might know about near-workspaces-rs - it’s a fantastic tool for NEAR testing in the proper execution environment. However, with the current state of WASM tooling there is no coverage - meaning you have coverage for tests that don't mirror your actual environment and none for the crucial real tests. This issue bothered me, leading to the creation of wasmcov in collaboration with Bartosz Barwikowski. It's a game-changing code coverage tool for wasm — the first of its kind. It's a significant leap forward for wasm testing, working not just in embedded environments but also in blockchain settings. ## Implementation Plan Our implementation team would work together with the NEAR team to enable native code coverage support without any dependency patching - in short, fully fleshing-out and streamlining the PoC for coverage measurement using wasmcov that is [documented here](https://hknio.github.io/wasmcov/docs/NEAR). The implementation plan is as follows: ### Patch `near_bindgen` to Automate Coverage Data Generation The `near_bindgen` macro is responsible for generating the code that connects Rust contracts to the NEAR Protocol. We need to patch this macro to add coverage instrumentation to the generated code. This patch will be implemented as a feature flag, so it can be enabled or disabled at will. ```rust #[no_mangle] unsafe extern "C" fn generate_coverage() { let mut coverage = vec![]; wasmcov::capture_coverage(&mut coverage).unwrap(); // Invoke a function (e.g., `your_custom_save_coverage_function`) to preserve the coverage data or utilize `println!` for debugging. } ``` Automating coverage data generation post-execution is critical. Manually inserting calls to `generate_coverage` for each function is impractical. ### Modify the NEAR Protocol Sandbox to Allow for Higher Gas Limits To run tests with coverage, a custom NEAR Protocol sandbox with modified parameters to allow for higher gas limits needs to be created. We intend to improve upon the [near/sandbox](https://github.com/near/sandbox) and [near/nearcore](https://github.com/near/nearcore) repositories to achieve this - adding a custom coverage network setting to the sandbox and modifying the NEAR Protocol to allow for higher gas limits. ### Add support for coverage measurement to near-workspaces-rs In tests, NEAR Workspaces need to be connected to the custom test sandbox created in the previous step. An example of a local sandbox setup is available below. Based on discussion with the NEAR team we would find the best way to integrate a custom coverage sandbox connection. ```rust use wasmcov::dir::get_wasmcov_dir; ... let worker = near_workspaces::sandbox() .rpc_addr("http://localhost:3030") .validator_key(ValidatorKey::HomeDir(get_wasmcov_dir().join("bin").join(".near"))) .await?; ``` ### Collect Coverage Data in Tests The above implementation pushes coverage data through logs. After extracting the coverage data (via logs, storage, file writes, etc.), invoke `wasmcov::write_profraw(coverage_data)` to save it to a `.profraw` file. This file becomes the basis for generating a `.profdata` file. This step can be automated by modifying the contract call handlers in near-workspaces-rs. This will ensure that the coverage data is saved without any modifications to the test scripts. ```rust use wasmcov::near::near_coverage; fn main() { let contract: near_workspaces::Contract = near_workspaces::Contract::new(); let result = contract.view("get_coverage").await?; ... near_coverage(&result.logs()); } ``` ### Examples and Documentation With the above chain specific modifications executed, `wasmcov` can take over the rest of the process. The library will automatically generate a `.profdata` file from the `.profraw` file, and generate a coverage report from the `.profdata` file. This report can be used to identify gaps in test coverage. Based on the current NEAR testing workflow, the wasmcov binary is likely a good fit for implementation, but this can be adjusted based on further discussion. ```bash eval $(cargo wasmcov setup) # Your build command cargo build -p contract --target wasm32-unknown-unknown # Set up and run your tests cargo wasmcov post_build # Find the compiled .wasm files make external_tests # Run your external tests cargo wasmcov finalize ``` ## Deliverables - A patch for `near_bindgen` to automate coverage data generation - A custom NEAR Protocol sandbox with modified parameters to allow for higher gas limits - A custom coverage sandbox connection for near-workspaces-rs - A modified contract call handler in near-workspaces-rs to collect coverage data in tests - Examples and documentation for the above ## Funding and Timeline We estimate that the implementation will take 2-3 weeks. The total cost of the project is $15,000, with the following payment schedule: - 50% when at the start of the development process - 50% upon completion of all deliverables ## Team The implementation would be carried out by [Noah Jelich](https://www.linkedin.com/in/njelich/), a Lead Security Researcher at Hacken. ## About Hacken Hacken is a blockchain security auditor, making Web3 safer since 2017. As a vertically integrated company, we focus on delivering security solutions for Web3 projects, including smart contract and blockchain protocol audits. Our strength lies in our industry-leading teams of certified professionals, supported by specialized expertise for each service offering and a solid foundation in ethical hacking. Over the past seven years, we’ve made measurable strides towards evolving a safer Web3 by auditing 1,200 Web3 projects, discovering thousands of vulnerabilities, and contributing to industry standards. Our security services are recognised by Binance, Enterprise Ethereum Alliance, CoinGecko and CoinMarketCap, while our clients and partners include top-industry players, such as 1inch, Aurora, DAO Maker, European Commission, Gate.io, HTX, KuCoin, NEAR, OKX, Radix to name a few. Our goal is to continually drive innovation and reinforce the safety and integrity of the blockchain industry, providing a secure and reliable environment for all open web adopters.