Uniswap — Transaction Prediction (Part VIII)

Building to Backruns

The watcher from Part VII has been churning for a week now and appears robust enough to continue building.

As a side note, the watcher was frequently highlighting opportunities for a few random coins (BITCOIN, HAMS, UNIBOT, PC) which gave me an idea to build a generic “shitcoin interest scanner”. I’ll get back to this later, but it was interesting to observe that high swapping activity correlating with increasing interest due to price gains.

This lesson will take the backrun watcher and add half of a Flashbots relay submission function (the familiar execute_arb_with_relay). It only includes the top half, because I still need to do testing against the relay. It’s been some time since I hit the Flashbots relay, so I won’t go cowboy and burn a bunch of gas. The code will do the backrun simulation only, and then return after printing some basic profitability numbers.

Snapshot Automation

Over the past week I’ve incorporated a few nice improvements related to automatic liquidity snapshot management.

Here’s the TLDR: if you create a snapshot object and pass it to a V3 Uniswap Pool Manager, any V3 pools generated by that manager will be refresh at build time with a liquidity snapshot that is current to the newest block known by the snapshot.

Here’s an example:

(.venv) [devil@dev bots]$ brownie console --network mainnet-local
Brownie v1.19.3 - Python development framework for Ethereum

BotsProject is the active project.
Brownie environment is ready.
>>> import degenbot as bot
>>> snapshot = bot.uniswap.v3.snapshot.UniswapV3LiquiditySnapshot(
    'ethereum_v3_liquidity_snapshot.json'
)
Loaded LP snapshot: 14654 pools @ block 17799747
>>> chain.height
17800067

The current block height is 17,800,067 so we have an opportunity to demonstrating updating the snapshot to the current block:

>>> snapshot.fetch_new_liquidity_events(to_block=chain.height)
Updating snapshot from block 17799747 to 17800067
Processing Mint events
Processing Burn events
Updated snapshot to block 17800067

Now let’s create a V3 Pool Manager associated with the Uniswap V3 factory, passing the snapshot in via the snapshot argument:

>>> univ3_lp_manager = bot.UniswapV3LiquidityPoolManager(
    factory_address="0x1F98431c8aD98523631AE4a59f267346ea31F984",
    snapshot=snapshot,
)

Now let’s get the familiar WBTC-WETH pool from the manager:

>>> lp = univ3_lp_manager.get_pool(
    pool_address='0xCBCdF9626bC03E24f779434178A73a0B4bad62eD'
)

• WBTC (Wrapped BTC)
• WETH (Wrapped Ether)
WBTC-WETH (V3, 0.30%)
• Token 0: WBTC
• Token 1: WETH
• Fee: 3000
• Liquidity: 1710357355732935015
• SqrtPrice: 31356042752874473506319369422032215
• Tick: 257784

Now the moment of truth, does it have the latest liquidity info?

>>> lp.tick_bitmap
{
    -58: UniswapV3BitmapAtWord(
        bitmap=6917529027641081856, 
        block=13035555),
    -6: UniswapV3BitmapAtWord(
        bitmap=2, 
        block=12604301),
    0: UniswapV3BitmapAtWord(
        bitmap=1, 
        block=12455826),
    10: UniswapV3BitmapAtWord(
        bitmap=170141183460469231731687303715884105728, 
        block=15395244),
    11: UniswapV3BitmapAtWord(
        bitmap=28948243165212146153933944164359569828134633057888820293979968816874004676608, 
        block=15395244),
    13: UniswapV3BitmapAtWord(
        bitmap=28948022309329048855892746252171976963317620781534745845727480733890183692288, 
        block=16892417),
    14: UniswapV3BitmapAtWord(
        bitmap=54277541829991966604798899222822457154669449039060000979991340549024308527104, 
        block=17413048),
    15: UniswapV3BitmapAtWord(
        bitmap=65136584752943523138391235523122452417203593855052757758510585506938655596544, 
        block=17660056),
    16: UniswapV3BitmapAtWord(
        bitmap=115792089237316195423570985008687907853268655437644779123584680198630675569800, 
        block=17785590),
    17: UniswapV3BitmapAtWord(
        bitmap=14474011154664524432752386995444604526860721169040065308549775201060171808767, 
        block=17765033),
    18: UniswapV3BitmapAtWord(
        bitmap=5327296616471419015641141638445662208, 
        block=17638060),
    21: UniswapV3BitmapAtWord(
        bitmap=41538374868278621028243970633760768, 
        block=12651514),
    23: UniswapV3BitmapAtWord(
        bitmap=14474011154664524427946373126085988481658748083205070504932198000989141204992, 
        block=14788009),
    25: UniswapV3BitmapAtWord(
        bitmap=21267647932558653966460912964485513216, 
        block=12604301),
    28: UniswapV3BitmapAtWord(
        bitmap=21267647932558653966460912964485513216, 
        block=12455826),
    5: UniswapV3BitmapAtWord(
        bitmap=86844066927987146567678238756515930889952488499230423029593188005934847229952, 
        block=16243768),
    57: UniswapV3BitmapAtWord(
        bitmap=50216813883093446110686315385661331328818843555712276103168, 
        block=13035555),
    9: UniswapV3BitmapAtWord(
        bitmap=21741053132098019175522475746641612337257189351054491279686364306905686343680, 
        block=12760382)
}

I hand-verified each of the bitmaps against the values reported by the contract at block height 17,800,067. I encourage you to do the same by forking at this block height and querying the V3 pool contract using the TickBitmap function at the keys above.

Use The Pool Managers!

A key difficulty of running a highly-scaled bot is keeping track of all the various places that a helper might be built and referenced. Previous efforts were assembled “by hand” in many places, so keeping track was fairly straightforward. Build a pool, send the pool to an arb helper, store the pool and the arb helper in a dictionary somewhere, life goes on.

To operate smoothly, the transaction helper needed more control over the process. Specifically, it needed the ability to hot-load a liquidity pool. So I built the pool managers to reduce that burden and allow the transaction simulator (and other classes) to request pool helpers when needed.

The pool manager exists to simplify our interaction with pool helpers, and you’ll only run into heartburn if you try to compete with it.

So use the pool managers! The example code I’ll share below uses them exclusively instead of building pool helpers directly, so please review, experiment, and adapt your own bots as needed.

Ensure Success With The Pool Managers

The pool managers behave in a unique way. I want to ensure that a particular pool manager is responsible for pool helpers associated with a a particular factory address (e.g. the Uniswap V3 factory at 0x1F98431c8aD98523631AE4a59f267346ea31F984 will have an associated pool manager that knows about all pools created by that factory)

The pool managers maintain a common record of associated pool helpers, though they also allow you create more than one pool manager. This is known as a “Borg” singleton pattern, where the state data acts as a singleton but you can create instances of the class as you like.

The key takeaway here is that you should take care during startup of your bot to ensure that the snapshot is created and loaded into the appropriate pool managers prior to doing anything else. The transaction simulator and event watcher will both use a pool manager to retrieve pool helpers, so you want to be sure that the snapshot is loaded into the pool manager’s common state data structure.

Example:

snapshot.fetch_new_liquidity_events(bot_status.first_event)

pool_managers = dict()

univ2_lp_manager = bot.UniswapV2LiquidityPoolManager(
    factory_address="0x5C69bEe701ef814a2B6a3EDD4B1652CB9cc5aA6f"
)
pool_managers[
    "0x5C69bEe701ef814a2B6a3EDD4B1652CB9cc5aA6f"
] = univ2_lp_manager

univ3_lp_manager = bot.UniswapV3LiquidityPoolManager(
    factory_address="0x1F98431c8aD98523631AE4a59f267346ea31F984",
    snapshot=snapshot,
)
pool_managers[
    "0x1F98431c8aD98523631AE4a59f267346ea31F984"
] = univ3_lp_manager

sushiv2_lp_manager = bot.UniswapV2LiquidityPoolManager(
    factory_address="0xC0AEe478e3658e2610c5F7A4A2E1777cE9e4f2Ac"
)
pool_managers[
    "0xC0AEe478e3658e2610c5F7A4A2E1777cE9e4f2Ac"
] = sushiv2_lp_manager

sushiv3_lp_manager = bot.UniswapV3LiquidityPoolManager(
    factory_address="0xbACEB8eC6b9355Dfc0269C18bac9d6E2Bdc29C4F",
    snapshot=snapshot,
)
pool_managers[
    "0xbACEB8eC6b9355Dfc0269C18bac9d6E2Bdc29C4F"
] = sushiv3_lp_manager

Here I’ve created four pool managers (Uniswap V2, Uniswap V3, Sushiswap V2, and Sushiswap V3) and stored a reference to each in the pool_managers dictionary. The V3 pool managers have access to the snapshot object, so the liquidity information for those pools will be current when loaded.

In the example below, the pool managers are created before the event watcher and pending transaction routines begin processing anything. This ensures that the pool managers they create have access to the snapshots.

NOTE: when the event watcher finds an update for a particular pool, it is not obvious which DEX that pool is associated with. I am still experimenting with methods that would allow us to quickly identify which pools belong to a DEX without doing a bunch of wasteful lookups, but that’s another project for another day. My crude solution is to use get_pool with each pool manager, and the pool manager itself will throw a ManagerError exception when the factory address associated with the pool does not match the manager. e.g. if you attempt to request the WBTC-WETH UniswapV3 pool from a Sushiswap manager by address, it will throw a ManagerError with the message “WRONG DEX”.

Example:

>>> sushiv3_lp_manager = bot.UniswapV3LiquidityPoolManager( 
    factory_address="0xbACEB8eC6b9355Dfc0269C18bac9d6E2Bdc29C4F", 
    snapshot=snapshot, 
)

>>> sushiv3_lp_manager.get_pool(
    pool_address='0xCBCdF9626bC03E24f779434178A73a0B4bad62eD'
)
  File "<console>", line 1, in <module>
  File "./degenbot/uniswap/uniswap_managers.py", line 461, in get_pool
    pool_helper = find_or_build(pool_address, self._snapshot)
  File "./degenbot/uniswap/uniswap_managers.py", line 378, in find_or_build
    self._raise_if_wrong_factory(
  File "./degenbot/uniswap/uniswap_managers.py", line 128, in _raise_if_wrong_factory
    raise ManagerError("WRONG DEX")
ManagerError: WRONG DEX

Remaining To-Do Items

Since the executor contract will need to support both Uniswap and Sushiswap V3 pools, it needs some slight edits. Rather than trying to jam this in without sufficient testing, I am going to cover these in the next part of the series.

If you have an executor contract currently deployed, it will revert on simulations involving Sushiswap V3 pools. The code below excludes those pools, but the next version will support that DEX along with an updated contract.

Also I’ve been experimenting with the alternative node clients (to mixed results). Besu, in particular, behaves differently from Geth from the POV of raw transactoin fetching, so I’ve included two lines in the watch_pending_transactions function that supports both client. If you’re running Besu, check it out. If you’re using Geth, it should be OK as-is but please report bugs if you find them.

transaction = w3.eth.get_transaction(tx_hash)
raw_transaction = transaction["raw"]  # Besu 
raw_transaction = w3.eth.get_raw_transaction(tx_hash)  # Geth

Source

ethereum_backrun_watcher.py