Local Bundle Simulation with Anvil
Local Bundle Simulation with Anvil
Fly By Wire Forks
We’re all fans of eth_call here, but a huge limitation of this method is that it can only be used for a single transaction at a time.
There is currently no method to execute a multi-transaction simulation against a live blockchain. The ability to simulate multiple transactions is critical for a mempool-aware bot, since chain state is expected to change with each transaction and it is very difficult to do an off-chain simulation except for the most well known contracts.
For more generic simulations, you must rely on a 3rd party such as Flashbots, Tenderly, and Alchemy. Flashbots is free, but quite limited and somewhat unreliable. Tenderly and Alchemy are more robust, but require a subscription and you need to work around rate limits.
And as always, the Achilles Heel of performance is the round trip over the Internet to these external endpoints.
What if we could build our own instead?
Enter Ripped Jesus
I’d like to introduce you to the Foundry toolkit. It consists of four tools:
Forge: Ethereum testing framework (like Truffle, Hardhat and DappTools).
Cast: Swiss army knife for interacting with EVM smart contracts, sending transactions and getting chain data.
Anvil: Local Ethereum node, akin to Ganache, Hardhat Network.
Chisel: Fast, utilitarian, and verbose solidity REPL.
The one I care about here is Anvil, but I will be exploring others in the future.
Anvil is used the same way as Ganache, and implements all of the Ganache methods. It’s also quite fast!
Installing Anvil
Following the instructions at getfoundry.sh, Anvil can be installed simply via console and then kept current using the foundryup command.
[devil@dev ~]$ curl -L https://foundry.paradigm.xyz | bash
[...]
Installing foundryup...
######################################################################## 100.0%
Detected your preferred shell is bash and added foundryup to PATH. Run 'source /home/devil/.bashrc' or start a new terminal session to use foundryup.
Then, simply run 'foundryup' to install Foundry.[devil@dev ~]$ foundryup
.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx
╔═╗ ╔═╗ ╦ ╦ ╔╗╔ ╔╦╗ ╦═╗ ╦ ╦ Portable and modular toolkit
╠╣ ║ ║ ║ ║ ║║║ ║║ ╠╦╝ ╚╦╝ for Ethereum Application Development
╚ ╚═╝ ╚═╝ ╝╚╝ ═╩╝ ╩╚═ ╩ written in Rust.
.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx
Repo : https://github.com/foundry-rs/
Book : https://book.getfoundry.sh/
Chat : https://t.me/foundry_rs/
Support : https://t.me/foundry_support/
Contribute : https://github.com/orgs/foundry-rs/projects/2/
.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx
foundryup: installing foundry (version nightly, tag nightly-1143e57fd4eb706d2f2b8d040d3945c0b27a47f7)
foundryup: downloading latest forge, cast, anvil, and chisel
########################################################### 100.0%
foundryup: downloading manpages
########################################################### 100.0%
foundryup: installed - forge 0.2.0 (1143e57 2023-08-24T00:20:21.116511461Z)
foundryup: installed - cast 0.2.0 (1143e57 2023-08-24T00:20:21.126235885Z)
foundryup: installed - anvil 0.2.0 (1143e57 2023-08-24T00:20:21.121331722Z)
foundryup: installed - chisel 0.2.0 (1143e57 2023-08-24T00:20:21.130819143Z)
foundryup: done!Now run Anvil and see how similar it is to Ganache and Hardhat.
[devil@dev ~]$ anvil --port 6969
_ _
(_) | |
__ _ _ __ __ __ _ | |
/ _` | | '_ \ \ \ / / | | | |
| (_| | | | | | \ V / | | | |
\__,_| |_| |_| \_/ |_| |_|
0.2.0 (1143e57 2023-08-24T00:20:21.121331722Z)
https://github.com/foundry-rs/foundry
Available Accounts
==================
(0) "0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266" (10000.000000000000000000 ETH)
(1) "0x70997970C51812dc3A010C7d01b50e0d17dc79C8" (10000.000000000000000000 ETH)
(2) "0x3C44CdDdB6a900fa2b585dd299e03d12FA4293BC" (10000.000000000000000000 ETH)
(3) "0x90F79bf6EB2c4f870365E785982E1f101E93b906" (10000.000000000000000000 ETH)
(4) "0x15d34AAf54267DB7D7c367839AAf71A00a2C6A65" (10000.000000000000000000 ETH)
(5) "0x9965507D1a55bcC2695C58ba16FB37d819B0A4dc" (10000.000000000000000000 ETH)
(6) "0x976EA74026E726554dB657fA54763abd0C3a0aa9" (10000.000000000000000000 ETH)
(7) "0x14dC79964da2C08b23698B3D3cc7Ca32193d9955" (10000.000000000000000000 ETH)
(8) "0x23618e81E3f5cdF7f54C3d65f7FBc0aBf5B21E8f" (10000.000000000000000000 ETH)
(9) "0xa0Ee7A142d267C1f36714E4a8F75612F20a79720" (10000.000000000000000000 ETH)
Private Keys
==================
(0) 0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80
(1) 0x59c6995e998f97a5a0044966f0945389dc9e86dae88c7a8412f4603b6b78690d
(2) 0x5de4111afa1a4b94908f83103eb1f1706367c2e68ca870fc3fb9a804cdab365a
(3) 0x7c852118294e51e653712a81e05800f419141751be58f605c371e15141b007a6
(4) 0x47e179ec197488593b187f80a00eb0da91f1b9d0b13f8733639f19c30a34926a
(5) 0x8b3a350cf5c34c9194ca85829a2df0ec3153be0318b5e2d3348e872092edffba
(6) 0x92db14e403b83dfe3df233f83dfa3a0d7096f21ca9b0d6d6b8d88b2b4ec1564e
(7) 0x4bbbf85ce3377467afe5d46f804f221813b2bb87f24d81f60f1fcdbf7cbf4356
(8) 0xdbda1821b80551c9d65939329250298aa3472ba22feea921c0cf5d620ea67b97
(9) 0x2a871d0798f97d79848a013d4936a73bf4cc922c825d33c1cf7073dff6d409c6
Wallet
==================
Mnemonic: test test test test test test test test test test test junk
Derivation path: m/44'/60'/0'/0/
Chain ID
==================
31337
Base Fee
==================
1000000000
Gas Limit
==================
30000000
Genesis Timestamp
==================
1692846937
Listening on 127.0.0.1:6969You can also fork from a remote RPC like Ganache. Let’s fork off the public Ankr Ethereum endpoint:
[devil@dev ~]$ anvil --port 6969 --fork-url https://rpc.ankr.com/eth
[...]
Fork
==================
Endpoint: https://rpc.ankr.com/eth
Block number: 17981825
Block hash: 0xcff704fda5edc05bec2e779abc8ce180899eb0e07844d6422bc94d724ffc8dc0
Chain ID: 1
Base Fee
==================
16372787592
Gas Limit
==================
30000000
Genesis Timestamp
==================
1692847030
Listening on 127.0.0.1:6969NOTE: if you’re not yet an expert using the Linux console, you can terminate the running fork with CTRL+C.
Python Test
With this local fork running, let’s fire up a Python console and use web3py to interact with it:
>>> import web3
>>> w3 = web3.Web3(web3.HTTPProvider('http://localhost:6969'))
>>> w3.isConnected()
True
>>> w3.eth.get_block_number()
17981825
>>> w3.eth.get_block('latest')
[ ... a bunch of stuff from block 17981825 ]You can use Brownie if you prefer a more familiar environment, but be sure to add an appropriate network first. Anvil is directly integrated through Brownie as of version 1.19.3, so you can use it instead of Ganache if you prefer it.
Remote Control Forking
Python’s powerful subprocess module allows you to launch and control external processes from within Python. We will use it to launch Anvil directly instead of manipulating it on a separate console.
Kill Anvil and then launch a clean Python shell to practice. Please note the requirement for each argument to anvil to be provided as a separate string inside a list!
>>> import subprocess
>>> subprocess.run(["anvil","--port=6969"])
_ _
(_) | |
__ _ _ __ __ __ _ | |
/ _` | | '_ \ \ \ / / | | | |
| (_| | | | | | \ V / | | | |
\__,_| |_| |_| \_/ |_| |_|
0.2.0 (1143e57 2023-08-24T00:20:21.121331722Z)
https://github.com/foundry-rs/foundry
Available Accounts
==================
(0) "0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266" (10000.000000000000000000 ETH)
(1) "0x70997970C51812dc3A010C7d01b50e0d17dc79C8" (10000.000000000000000000 ETH)
(2) "0x3C44CdDdB6a900fa2b585dd299e03d12FA4293BC" (10000.000000000000000000 ETH)
(3) "0x90F79bf6EB2c4f870365E785982E1f101E93b906" (10000.000000000000000000 ETH)
(4) "0x15d34AAf54267DB7D7c367839AAf71A00a2C6A65" (10000.000000000000000000 ETH)
(5) "0x9965507D1a55bcC2695C58ba16FB37d819B0A4dc" (10000.000000000000000000 ETH)
(6) "0x976EA74026E726554dB657fA54763abd0C3a0aa9" (10000.000000000000000000 ETH)
(7) "0x14dC79964da2C08b23698B3D3cc7Ca32193d9955" (10000.000000000000000000 ETH)
(8) "0x23618e81E3f5cdF7f54C3d65f7FBc0aBf5B21E8f" (10000.000000000000000000 ETH)
(9) "0xa0Ee7A142d267C1f36714E4a8F75612F20a79720" (10000.000000000000000000 ETH)
Private Keys
==================
(0) 0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80
(1) 0x59c6995e998f97a5a0044966f0945389dc9e86dae88c7a8412f4603b6b78690d
(2) 0x5de4111afa1a4b94908f83103eb1f1706367c2e68ca870fc3fb9a804cdab365a
(3) 0x7c852118294e51e653712a81e05800f419141751be58f605c371e15141b007a6
(4) 0x47e179ec197488593b187f80a00eb0da91f1b9d0b13f8733639f19c30a34926a
(5) 0x8b3a350cf5c34c9194ca85829a2df0ec3153be0318b5e2d3348e872092edffba
(6) 0x92db14e403b83dfe3df233f83dfa3a0d7096f21ca9b0d6d6b8d88b2b4ec1564e
(7) 0x4bbbf85ce3377467afe5d46f804f221813b2bb87f24d81f60f1fcdbf7cbf4356
(8) 0xdbda1821b80551c9d65939329250298aa3472ba22feea921c0cf5d620ea67b97
(9) 0x2a871d0798f97d79848a013d4936a73bf4cc922c825d33c1cf7073dff6d409c6
Wallet
==================
Mnemonic: test test test test test test test test test test test junk
Derivation path: m/44'/60'/0'/0/
Chain ID
==================
31337
Base Fee
==================
1000000000
Gas Limit
==================
30000000
Genesis Timestamp
==================
1692847835
Listening on 127.0.0.1:6969The Anvil process is started and all of the typical output is now being redirected to the console. But you’ll notice that there is no input, and the only thing we can do is look at the last message. Kill Anvil by sending CTRL+C inside the Python shell, and then you will have control again:
[...]
^CTraceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/devil/.pyenv/versions/3.10.10/lib/python3.10/subprocess.py", line 505, in run
stdout, stderr = process.communicate(input, timeout=timeout)
File "/home/devil/.pyenv/versions/3.10.10/lib/python3.10/subprocess.py", line 1146, in communicate
self.wait()
File "/home/devil/.pyenv/versions/3.10.10/lib/python3.10/subprocess.py", line 1209, in wait
return self._wait(timeout=timeout)
File "/home/devil/.pyenv/versions/3.10.10/lib/python3.10/subprocess.py", line 1943, in _wait
(pid, sts) = self._try_wait(0)
File "/home/devil/.pyenv/versions/3.10.10/lib/python3.10/subprocess.py", line 1901, in _try_wait
(pid, sts) = os.waitpid(self.pid, wait_flags)
KeyboardInterrupt
>>>The subprocess.run function is more useful for standalone functions that do something, then exit. For a long-running process that you’d like to run in parallel, you need to use a lower-level class called Popen.
Let’s use Popen instead, and modify the arguments to launch a local fork from Ankr as before:
>>> fork = subprocess.Popen(
[
"anvil",
"--port=6969",
"--fork-url=https://rpc.ankr.com/eth"
]
)
[... familiar Anvil output ]
Listening on 127.0.0.1:6969It looks the same as before, but if you press Enter you’ll notice that the Python shell returns and you can continue entering commands:
Listening on 127.0.0.1:6969
>>>So let’s import web3 again and interact with our fork:
Listening on 127.0.0.1:6969
>>> w3 = web3.Web3(web3.HTTPProvider('http://localhost:6969'))
>>> w3.isConnected()
True
>>> w3.eth.get_block_number()
eth_blockNumber
17981945
[ ... a bunch of stuff from block 17981945 ]You might ask “What’s that eth_blockNumber output all about?”
It’s Anvil output being redirected to our Python console. If you watch Anvil in another terminal, you can see the RPC methods that we are sending to it. The output isn’t particularly useful here and mostly just clutters up the screen, so let’s throw away the fork by terminating the process, then waiting for it to exit:
>>> fork.terminate()
>>> fork.wait()You can now verify that the Anvil process is no longer running by opening another shell and running ps aux | grep anvil.
Now launch Anvil again with the “--silent” option:
>>> fork = subprocess.Popen(
[
"anvil",
"--port=6969",
"--fork-url=https://rpc.ankr.com/eth",
"--silent"
]
)
>>>Notice that Python shell control is given back immediately, and none of the typical Anvil output is displayed.
This configuration is much cleaner, and with some more refinement of the Anvil options, we can build a pretty decent transaction simulator.
I recommend reviewing the Anvil documentation for more on these, but here are some highlights: