---
title: 5 million flips
date: 2025-02-21
draft: false
description: Implementing storage access checks and other improvements
tags: [ status-update ]
authors: [ nazar-pc ]
---

It was a challenging week working on storage access checks for slots, but it is over, and I'm quite happy with how
things are looking right now. Some extra refactoring also allowed running tests under Miri and spotted some things that
violate the Rust safety rules.

<!--more-->

The work from the previous week continued on reworking the way slots are managed by native execution environment to
correctly handle recursive method calls and potential access violations. It finally concluded in [PR 61] with some
follow-up fixes in later PRs.

[PR 61]: https://github.com/nazar-pc/abundance/pull/61

There were several challenges with it that step from the desire to achieve high performance, while retaining efficiency
and maintainability. In the end, the following rules were established: a single recursive call can modify storage, but
multiple calls dispatched at once (meant to be parallel, but aren't right now) have only read-only view. This should fit
the expected use cases nicely and help to constrain code complexity. There are a few paragraphs that explain goals and
results in more detail in [PR 61] if you're interested to learn more.

I've been thinking about address formats some more and decided that for a global system 44 bits for addresses is really
not enough, and it should be way more than that. The addresses were also stored as `[u8; 8]` instead of `u64` to reduce
alignment requirements for data structures that might contain them, so the question became what should bigger address
look like and how much bigger should it really be. I then looked at RISC-V (planned to be used for a VM) assembly for
different operations on byte arrays. Comparing two addresses is the most common operation here, and turned out that byte
arrays comparison generates way more assembly instructions to do the same job. This is both due to RISC nature of the
ISA and the fact that alignment of the byte array is 1. x86-64 has powerful instructions to read unaligned byte ranges
into XMM registers and do comparison for all bytes at once, while RISC-V assembly (at least the way it is generated by
rustc for `riscv64imac-unknown-none-elf`) was comparing bytes [one pair at a time].

[one pair at a time]: https://rust.godbolt.org/z/ojzhTzf5v

As the result, I decided that `u128` will be the address format, which might be relaxed to a pair of `u64`s that 64-bit
to reduce alignment requirement from `16` bytes to `8` (RISC-V assembly is comparing 64-bit halves separately rather
than full 128-bit value at once anyway). This, landed in [PR 63], which also included some refactoring for slots
management, given how large a pair of addresses (owner+contract are used to identify a slot) have become.

[PR 63]: https://github.com/nazar-pc/abundance/pull/63

Based on developer interview with Shamil I have clarified and expanded on documentation in [PR 64], which I hope will
make it easier to understand.

[PR 64]: https://github.com/nazar-pc/abundance/pull/64

I did some initial benchmarks with [PR 61], turned out it is possible to create an environment instance and call
`Flipper::flip` on it about four million per second on a single CPU core, which gives you a good perspective of how much
overhead is happening in typical blockchain environments that can only do orders of magnitude fewer simple transactions
per seconds. But after slot optimizations in [PR 64] I got curious if it is possible to do better and squeezed another
million calls per second in [PR 65].

Five million calls per second on a single CPU core, ~200 ns per call! I'm sure it is possible to get even lower while
preserving necessary logic and overall architecture. That is basically the baseline, whatever cost above that is a waste
and should be minimized. `perf` stats look something like this:

```
          1 122,47 msec task-clock:u                     #    1,000 CPUs utilized             
                 0      context-switches:u               #    0,000 /sec                      
                 0      cpu-migrations:u                 #    0,000 /sec                      
               167      page-faults:u                    #  148,780 /sec                      
     5 459 682 279      cycles:u                         #    4,864 GHz                       
        74 201 852      stalled-cycles-frontend:u        #    1,36% frontend cycles idle      
    14 406 036 797      instructions:u                   #    2,64  insn per cycle            
                                                  #    0,01  stalled cycles per insn   
     2 470 197 650      branches:u                       #    2,201 G/sec                     
            14 684      branch-misses:u                  #    0,00% of all branches           

```

[PR 65]: https://github.com/nazar-pc/abundance/pull/65

With storage taken care of for now, there was a small problem that bothered me for a while: inability to run tests under
Miri. Writing unsafe code in Rust is more challenging than in languages like C, and there is quite a bit of unsafe code
due to FFI and performance reasons in the native execution environment right now. So running under Miri was very
desirable, but unfortunately [not] [possible] with `inventory` crate that was used to make execution environment aware
of all the contracts available, so implicit use of `inventory` had to go away.

[not]: https://github.com/rust-lang/miri/issues/4171

[possible]: https://github.com/rust-lang/miri/issues/450

I still wanted to have an ergonomic API though, and that proved to be its own challenge due to the need to register both
contracts themselves and traits that they implement, but traits as such aren't types. The best thing I came up with was
to instead use `dyn ContractTrait` as a type, but then I discovered that associated constants just like other generics
make traits [not object safe]. I found several discussions and summarized the conclusion with some links
on [Rust forum]. And shared in the next post an unstable (and incomplete!) feature that allows to have associated
constants in traits that are object safe, but it doesn't look likely that it'll be stabilized any time soon. Ultimately,
I had to split associated constants into a separate trait (implemented on `dyn ContractTrait`) and remove `: Contract`
bound on the `ContractTrait` itself, but it seemed like a price worth paying. In the end, [PR 66] landed a decent API
that explicitly registers contracts to be used in the native execution environment (system contracts are registered
internally automatically), looks something like this:

```rust
#[test]
fn basic() {
    let shard_index = ShardIndex::from_u32(1).unwrap();
    let mut executor = NativeExecutor::in_memory_empty(shard_index)
        .with_contract::<Flipper>()
        .build()
        .unwrap();

    // ...
}
```

That also meant tests are finally running under Miri 😱

[not object safe]: https://doc.rust-lang.org/reference/items/traits.html#r-items.traits.dyn-compatible.associated-consts

[Rust forum]: https://users.rust-lang.org/t/constants-in-dyn-traits-not-allowed-any-workaround/124380/7?u=nazar-pc

[PR 66]: https://github.com/nazar-pc/abundance/pull/66

Yeah, Miri wasn't too happy initially 😅. It took a lot more reading and some [help from the Rust community] to figure
out why, but eventually I was able to make it work in [PR 67], which also added Miri tests to CI 😊.

[help from the Rust community]: https://users.rust-lang.org/t/why-is-miri-claiming-unique-retagging-with-raw-pointer/125940?u=nazar-pc

[PR 67]: https://github.com/nazar-pc/abundance/pull/67

That was the bulk of the things I got done, with some random research and WIP stuff in a local branch that I will talk
about next time. Unfortunately, there were no interviews this week, but hopefully next time!

## Upcoming plans

The next steps related to execution environment will be to add a notion of a transaction. So far it was just calling
methods on contracts, but the actual blockchain will have inputs serialized into a transaction. While
serialization/deserialization is already happening when doing calls from contract methods, the API that developers can
use externally wasn't that. With transaction support and more explicit slots handling (ability to provide them as input
and extract afterward for persistence) the workflow will be partially complete and sufficient for further interation
into a bigger system with things like transaction pool. Transaction pool, of course, doesn't exist yet (just like most
other things), but it can be fixed 😉.

Based on developer feedback, I would also like to simplify contract API a bit, specifically remove `#[result]` and make
it a special case of `#[output]`, which will remove some code duplication in execution environment and procedural macro
and will be easier to explain.

Once those are done, I will probably conduct more developer interviews with more people (will try to hunt down
some [ink!] maintainers or users initially). If there is someone I should definitely talk to, let me know.

[ink!]: https://use.ink/

Also, hopefully more hiring interviews this time.

See you in about a week with more updates!
