Using Transient Storage With Vyper

Some Storage Is More Immutable Than Others

The “Dencun” hardfork (a portmanteau of the “Deneb” consensus layer upgrade, and the “Cancun” execution layer upgrade) went live on March 13, 2024.

Public attention was mostly centered around the new proto-danksharding capability, which allowed transactions to carry non-executable data with some fairly robust data availability guarantees. You can read more about this at the official EIP-4844 proposal.

However there was another EIP included in Cancun that got me very excited. EIP-1153 brought new opcodes: TSTORE and TLOAD, where the “T” represents “transient”. Both opcodes allow EVM to access a new type of storage which is scoped at the transaction level, can be shared across transaction frames, and is automatically cleared at the end of a transaction.

A primary driver for these opcodes is to allow developers to have more durable access to storage during a transaction, without paying the high cost of permanent storage for values that do not need to be recorded to the chain.

The gas cost difference is stark. TSTORE costs 100 gas, while SSTORE to an empty slot costs 20,000 gas, updating an existing slot costs 2,900 gas, and updating a “dirty” slot (already modified in that transaction) costs 100 gas. Cost differences for TLOAD and SLOAD are comparably wide. So if you can, try to use transient storage!

The most obvious application of this storage type is that re-entrancy locks no longer need to use “real” storage via SSTORE and SLOAD. Since a re-entrancy lock is useful for the length of a single transaction already, moving it to the transient storage instead is a “no downsides” improvement. There are some risks in reality, mostly related to gas use and de-facto revert assumptions, but they’re largely out of scope here and I won’t cover them.

Vyper Leads The Way

The Vyper devs have had a working implementation of transient storage for almost a year now, first introduced in version 0.3.8.

In the weeks before Cancun went live, I started experimenting with transient storage with Vyper version 0.3.10 and generally had a great time.

To activate the new feature, simply specify # pragma evm-version cancun at the top of your contract. This will activate the new opcodes and allow you to use the transient keyword for storage.

It’s really that easy, folks.

Why Bother?

You may be wondering what all the fuss is about. After all, none of the stuff above is obviously relevant to MEV botters.

Stick with me for a bit — it will become clear once I introduce the concept and begin adding complexity.

Data, Three Ways

We must first understand the difference between the three areas where data can be stored: storage, memory, and the stack.

I recommend reviewing this very helpful section in the Solidity documentation which covers this topic. I’ll skip covering the stack, which is irrelevant here.

High level summary: EVM memory is volatile and reset for every contract call, internal or external. EVM storage is long-lived and persists between calls.

If I have some data in memory, I can access it from the function that is manipulating it, but I cannot share it between functions. If I have some contract with two functions (func1 and func2), they can share data only via input/output and storage.

If I start with func1, it has a memory layout to work with. If during its execution I call func2, func2 will start with a new empty memory layout, access to all values recorded in storage, and the values of all arguments sent to it via calldata. The memory layout is discarded when func2 returns, the stack is emptied, but the storage state will persist.

Why would you want to persist data between calls anyway? If you’re a shy contract that never makes external calls, you don’t need to bother with persisted values between calls. You can just pass data around to internal functions as needed.

However, if you need to pass data to an external contract and resume execution without immediately returning to your current position, you need a reliable way to communicate data between “call frames”. A call frame is simply a group of operations and state changes that began with a CALL at some contract address.

The traditional method of passing data between call frames has been calldata. The most widely understood example if the Uniswap V2 swap callback.

I first covered this callback in the Smart Contract Arbitrage series. The callback allows smart contracts to execute flash borrows of one or two tokens from a liquidity pool. As long as the tokens are repaid in full by the uniswapv2Call function, which your contract must implement, you can use them however you like.

Smart Contract Arbitrage — The Flash Borrow Callback

It provides a mechanism to allow arbitrary execution between the borrow and repay phase by passing an array of bytes back to your smart contract via the data argument to swap().

In this way, you can call swap() at the V2 pool, encoding some instruction to yourself via data, decode that instruction in uniswapv2Call, then continue execution there. The instruction must be available to uniswapv2Call, because the tokens must be repaid by that call or it will revert. There is no way (currently) to modify the call frame ordering, so you cannot call two functions out-of-order.

The calldata approach has the advantage of avoiding storage completely, which is a huge gas savings. However calldata is not free, and communicating across call frames with calldata requires you to pass the same instruction twice (once to the V2 pool, and once back to your contract).

And for any contract that does not implement this calldata pass-through, well… what can you do? Assuming you’re on Ethereum mainnet post-Cancun, I’d suggest using transient storage!

Setting values to transient storage allows you to communicate arbitrary state between functions without relying on a messenger to reliably pass calldata.

Chain Availability

EIP-1153 transient storage opcodes may not be available on your chain. Deploying a contract using these opcodes on an incompatible chain may be impossible, and will not function as intended if it can be deployed.

Beyond Ethereum mainnet, Polygon has implemented them in PIP-31, and Binance Smart Chain has proposed including them in BEP-343.

There may be others by the time you read this! If you know of any, please comment below and share a link to that information.

Contract Example

For this example, I have created a dedicated folder in ~/code/tstore_project, a Python virtual environment called tstore with Ape Framework version 0.7.19.

If you are unfamiliar with any of that, please review the following lessons:

How To Easily Manage Python Versions and Virtual Environments

Introduction to Ape Framework

Testing With Ape Framework

Initialize the project with ape init and create an Ape config to allow quick execution of tests agains the appropriate EVM version.

ape-config.yaml

name: Transient Storage Test

plugins:
  - name: vyper
  - name: foundry

ethereum:
  default_network: mainnet-fork
  mainnet_fork:
    default_provider: foundry

foundry:
  host: auto
  fork:
    ethereum:
      mainnet:        
        evm_version: cancun
        upstream_provider: geth

geth:
  ethereum:
    mainnet:
      uri: http://localhost:8545

Note that I run my own node, so replace http://localhost:8545 with the appropriate URI for yours setup.

Now create a contract in the contracts folder.

tstore.vy

# pragma evm-version cancun

message: transient(Bytes[32])

@external
def func1(message: Bytes[32]):
    self.message = message
    assert message == self.func2()

@internal
def func2() -> Bytes[32]:
    return self.message

The purpose of this very simple contract is to verify that setting a message via func1() will record the value in the transient variable self.message, which can be read and returned when func2() is called.

Now write a test to verify the behavior in the tests directory:

test_tstore.py

import pytest


@pytest.fixture
def tstore_contract(project, accounts):
    return project.tstore.deploy(sender=accounts[0])


def test_message_accessible_internally(tstore_contract, accounts):
    assert tstore_contract.func1(b"Hello World", sender=accounts[0])

Running ape test confirms that calling func1() works as expected:

(tstore) btd@dev:~/code/tstore_project$ ape test

========================= test session starts =========================
platform linux -- Python 3.12.2, pytest-7.4.4, pluggy-1.5.0
rootdir: /home/btd/code/tstore_project
plugins: eth-ape-0.7.19, web3-6.18.0
collected 1 item                                                                                                                            
INFO: Starting 'anvil' process.
INFO: Connecting to existing Geth node at http://localhost:8545/[hidden].

tests/test_tstore.py .                                                                                                                [100%]

========================= 1 passed in 0.85s =========================
INFO: Stopping 'anvil' process.

Now what if we want to call some external function? Let’s create a very simplistic contract that allows us to fake a V2 callback just for testing. It will call at the V2 callback interface when someone hits call_me_maybe_v2(). This allows us to verify that calling a Uniswap V2 pool is minimally functional, at least from the perspective of the callback.

fake_callback.vy

# pragma version >=0.3.10

interface IUniswapV2Callee:    
    def uniswapV2Call(
        sender: address,
        amount0: uint256, 
        amount1: uint256, 
        data: Bytes[32]
    ): nonpayable

@external
def call_me_maybe_v2():
    IUniswapV2Callee(msg.sender).uniswapV2Call(
        msg.sender,
        0,
        0,
        b''
    )

Now let’s add a function to the tstore contract that calls our fake_callback contract, and verifies within the callback that the transient storage is accessible:

tstore.vy

# pragma evm-version cancun

interface IFakeCallback:
    def call_me_maybe_v2(): nonpayable

interface IUniswapV2Callee:    
    def uniswapV2Call(
        sender: address,
        amount0: uint256, 
        amount1: uint256, 
        data: Bytes[32]
    ): nonpayable

implements: IUniswapV2Callee

message: transient(Bytes[32])

@external
@nonpayable
def func1(message: Bytes[32]):
    self.message = message
    assert message == self.func2()

@internal
def func2() -> Bytes[32]:
    return self.message

@external
@nonpayable
def func3(message: Bytes[32], external_contract: address):
    self.message = message
    IFakeCallback(external_contract).call_me_maybe_v2()


@external
@nonpayable
def uniswapV2Call(
    sender: address,
    amount0: uint256, 
    amount1: uint256, 
    data: Bytes[32]
):
    assert self.message == b'Hello World'

I’ve hard-coded the assert inside the callback for simplicity, so the test only passes if exactly b'Hello World' is passed to func3().

Now we’ve verified that transient storage values can be set in one function and accessed in another, so the concept is valid.

If you’re curious, you can verify via the Vyper compiler that neither SSTORE or SLOAD are used by inspecting the output of vyper with the -f opcodes_runtime flag:

(tstore) btd@dev:~/code/tstore_project$ vyper contracts/tstore.vy -f opcodes_runtime | grep SSTORE
(tstore) btd@dev:~/code/tstore_project$ vyper contracts/tstore.vy -f opcodes_runtime | grep SLOAD

And similarly, verify that TSTORE and TLOAD are used instead:

(tstore) btd@dev:~/code/tstore_project$ vyper contracts/tstore.vy -f opcodes_runtime | grep -E 'TLOAD|TSTORE'

PUSH0 CALLDATALOAD PUSH1 0xE0 SHR PUSH1 0x2 PUSH1 0x3 DUP3 MOD PUSH1 0x1 SHL PUSH2 0x243 ADD PUSH1 0x1E CODECOPY PUSH0 MLOAD JUMP JUMPDEST PUSH4 0x73C3B2F9 DUP2 XOR PUSH2 0x24 JUMPI PUSH1 0x44 CALLDATASIZE LT CALLVALUE OR PUSH2 0x23F JUMPI PUSH1 0x4 CALLDATALOAD PUSH1 0x4 ADD PUSH1 0x20 DUP2 CALLDATALOAD GT PUSH2 0x23F JUMPI PUSH1 0x20 DUP2 CALLDATALOAD ADD DUP1 DUP3 PUSH1 0x40 CALLDATACOPY POP POP PUSH1 0x20 PUSH1 0x40 MLOAD ADD PUSH0 DUP2 PUSH1 0x1F ADD PUSH1 0x5 SHR PUSH1 0x2 DUP2 GT PUSH2 0x23F JUMPI DUP1 ISZERO PUSH2 0x07B JUMPI SWAP1 JUMPDEST DUP1 PUSH1 0x5 SHL PUSH1 0x40 ADD MLOAD DUP2 TSTORE PUSH1 0x1 ADD DUP2 DUP2 XOR PUSH2 0x066 JUMPI JUMPDEST POP POP POP PUSH2 0x088 PUSH1 0xC0 PUSH2 0x28 JUMP JUMPDEST PUSH1 0xC0 DUP1 MLOAD PUSH1 0x20 DUP3 ADD SHA3 SWAP1 POP PUSH1 0x40 MLOAD PUSH1 0x60 SHA3 XOR PUSH2 0x23F JUMPI STOP PUSH2 0x24 JUMP JUMPDEST PUSH4 0xC54B0D4 DUP2 XOR PUSH2 0x24 JUMPI PUSH1 0x64 CALLDATASIZE LT CALLVALUE OR PUSH2 0x23F JUMPI PUSH1 0x4 CALLDATALOAD PUSH1 0x4 ADD PUSH1 0x20 DUP2 CALLDATALOAD GT PUSH2 0x23F JUMPI PUSH1 0x20 DUP2 CALLDATALOAD ADD DUP1 DUP3 PUSH1 0x40 CALLDATACOPY POP POP PUSH1 0x24 CALLDATALOAD DUP1 PUSH1 0xA0 SHR PUSH2 0x23F JUMPI PUSH1 0x80 MSTORE PUSH1 0x20 PUSH1 0x40 MLOAD ADD PUSH0 DUP2 PUSH1 0x1F ADD PUSH1 0x5 SHR PUSH1 0x2 DUP2 GT PUSH2 0x23F JUMPI DUP1 ISZERO PUSH2 0x115 JUMPI SWAP1 JUMPDEST DUP1 PUSH1 0x5 SHL PUSH1 0x40 ADD MLOAD DUP2 TSTORE PUSH1 0x1 ADD DUP2 DUP2 XOR PUSH2 0x10 JUMPI JUMPDEST POP POP POP PUSH1 0x80 MLOAD PUSH4 0xEAF6F77C PUSH1 0xA0 MSTORE DUP1 EXTCODESIZE ISZERO PUSH2 0x23F JUMPI PUSH0 PUSH1 0xA0 PUSH1 0x4 PUSH1 0xBC PUSH0 DUP6 GAS CALL PUSH2 0x141 JUMPI RETURNDATASIZE PUSH0 PUSH0 RETURNDATACOPY RETURNDATASIZE PUSH0 REVERT JUMPDEST POP STOP PUSH2 0x24 JUMP JUMPDEST PUSH4 0x10D1E85C DUP2 XOR PUSH2 0x24 JUMPI PUSH1 0xA4 CALLDATASIZE LT CALLVALUE OR PUSH2 0x23F JUMPI PUSH1 0x4 CALLDATALOAD DUP1 PUSH1 0xA0 SHR PUSH2 0x23F JUMPI PUSH1 0x40 MSTORE PUSH1 0x64 CALLDATALOAD PUSH1 0x4 ADD PUSH1 0x20 DUP2 CALLDATALOAD GT PUSH2 0x23F JUMPI PUSH1 0x20 DUP2 CALLDATALOAD ADD DUP1 DUP3 PUSH1 0x60 CALLDATACOPY POP POP PUSH1 0xB PUSH1 0xE0 MSTORE PUSH32 0x48656C6C6F20576F726C64000000000000000000000 PUSH2 0x10 MSTORE PUSH1 0xE0 DUP1 MLOAD PUSH1 0x20 DUP3 ADD SHA3 SWAP1 POP PUSH1 0x20 PUSH0 TLOAD ADD PUSH0 DUP2 PUSH1 0x1F ADD PUSH1 0x5 SHR PUSH1 0x2 DUP2 GT PUSH2 0x23F JUMPI DUP1 ISZERO PUSH2 0x1EE JUMPI SWAP1 JUMPDEST DUP1 TLOAD DUP2 PUSH1 0x5 SHL PUSH2 0x120 ADD MSTORE PUSH1 0x1 ADD DUP2 DUP2 XOR PUSH2 0x1D8 JUMPI JUMPDEST POP POP POP PUSH2 0x120 DUP1 MLOAD PUSH1 0x20 DUP3 ADD SHA3 SWAP1 POP XOR PUSH2 0x23F JUMPI STOP JUMPDEST PUSH0 PUSH0 REVERT JUMPDEST PUSH1 0x20 PUSH0 TLOAD ADD PUSH0 DUP2 PUSH1 0x1F ADD PUSH1 0x5 SHR PUSH1 0x2 DUP2 GT PUSH2 0x23F JUMPI DUP1 ISZERO PUSH2 0x239 JUMPI SWAP1 JUMPDEST DUP1 TLOAD DUP2 PUSH1 0x5 SHL DUP6 ADD MSTORE PUSH1 0x1 ADD DUP2 DUP2 XOR PUSH2 0x225 JUMPI JUMPDEST POP POP POP POP JUMP JUMPDEST PUSH0 DUP1 REVERT ADD BASEFEE STOP LOG4 STOP XOR

New Techniques

I’ve identified some interesting techniques that are enabled by transient storage.

Callback Pre-Verification

When working with V2 and V3 callbacks, you absolutely must protect your callback functions. I wrote about the V2 callback in the link above, and the V3 callback here:

Uniswap V3 — Swap Callback

The accepted method is to verify msg.sender (which should be a pool) by retrieving its token0 and token1 addresses and regenerating the canonical pool address from a hard-coded pool init hash and factory address.

This method is not completely sufficient, and will not protect you from malicious tokens like CHUM and similar. The Uniswap callbacks only allow us to verify that a valid pool is calling the contract, and that we generated the transaction.

Since we are the ones responsible for generating the transaction in the first place, perhaps we can just trust ourselves?

NOTE this is an experimental technique that I’ve been testing out. It might fail spectacularly and end up wrecking me and/or you! Be careful.

The idea is that a smart contract can “gate” callbacks by checking transient storage for allowed addresses, rather than validating each time a callback is entered.

Something like this minimal example:

transient_callback.vy:

# pragma evm-version cancun

interface IFakeCallback:
    def call_me_maybe_v2(): nonpayable

allowed_callbacks: transient(HashMap[address, bool])

@external
@nonpayable
def entrypoint(callback_address: address):    
    self.allowed_callbacks[callback_address] = True
    IFakeCallback(callback_address).call_me_maybe_v2()
    

@external
@nonpayable
def entrypoint_oops(callback_address: address):        
    IFakeCallback(callback_address).call_me_maybe_v2()


@external
@nonpayable
def uniswapV2Call(
    sender: address,
    amount0: uint256, 
    amount1: uint256, 
    data: Bytes[32]
):
    assert self.allowed_callbacks[msg.sender]

It includes a bugged entrypoint_oops() which does not add the callback to the allowed hashmap before calling it.

Craft a test to verify that priming the entrypoint with the fake callback address works as expected, while it fails when not primed:

test_tstore_gated_callback.py

import pytest


@pytest.fixture
def tstore_gated_callback_contract(project, accounts):
    return project.tstore_gated_callback.deploy(sender=accounts[0])


@pytest.fixture
def fake_callback(project, accounts):
    return project.fake_callback.deploy(sender=accounts[0])


def test_primed_callback(tstore_gated_callback_contract, accounts, fake_callback):
    assert tstore_gated_callback_contract.entrypoint(
        fake_callback.address, sender=accounts[0]
    )


def test_nonprimed_callback_fails(
    tstore_gated_callback_contract, accounts, fake_callback
):
    with ape.reverts():
        assert tstore_gated_callback_contract.entrypoint_oops(
            fake_callback.address, sender=accounts[0]
        )

This is a nice technique for strategies that employ multiple callbacks from predictable addresses.

Nested Payload Execution

Another benefit of the approach is that payloads can be delivered in different call frames within the same transaction.

As an example, let’s create a function that records two values to a dynamic array in transient storage, popping them off in two separate callbacks.

tstore_offloaded_values.vy

# pragma evm-version cancun

interface IFakeCallback:
    def call_me_maybe_v2(): nonpayable

interface IUniswapV2Callee:    
    def uniswapV2Call(
        sender: address,
        amount0: uint256, 
        amount1: uint256, 
        data: Bytes[32]
    ): nonpayable

implements: IUniswapV2Callee

values: transient(DynArray[uint256, 2])

event printValue:
    value: uint256

@external
@nonpayable
def store_values_and_call(values: uint256[2], call_addresses: address[2]):
    for value in values:
        self.values.append(value)

    for call_address in call_addresses:
        IFakeCallback(call_address).call_me_maybe_v2()

@external
@nonpayable
def uniswapV2Call(
    sender: address,
    amount0: uint256, 
    amount1: uint256, 
    data: Bytes[32]
):
    val: uint256 = self.values.pop()
    log printValue(val)

And a test that runs the function and verifies the events were emitted as expected:

test_offloaded_values.py

import pytest


@pytest.fixture
def tstore_offloaded_values_contract(project, accounts):
    return project.tstore_offloaded_values.deploy(sender=accounts[0])


@pytest.fixture
def fake_callback(project, accounts):
    return project.fake_callback.deploy(sender=accounts[0])


def test_pop_two_values_in_callback(
    tstore_offloaded_values_contract, accounts, fake_callback
):
    tx = tstore_offloaded_values_contract.store_values_and_call(
        [69, 420],
        [fake_callback.address, fake_callback.address],
        sender=accounts[0],
    )
    assert tx.events == [
        tstore_offloaded_values_contract.printValue(value=420),
        tstore_offloaded_values_contract.printValue(value=69),
    ]

Note that the first value is 420 instead of 69, because the pop() method on Vyper’s DynArray pops the last value.

Transient Storage Arbitrage Contract

I’ve been testing out this executor contract on mainnet. It has landed a few small arbs and works as expected, and the callback pre-loading discussed above contributes to lower gas use.