Project: Uniswap-CurveV1 Backrunner
It Ain't Much, But That's Something
If you’ll allow me to speak plainly, the background work on Curve V1 (StableSwap) pools has been a real pain in the ass. Extending the degenbot code base to support pool types beyond Uniswap V2 & V3 was useful and necessary, and adding other ecosystems will be easier going forward.
However, the effort cut heavily into my ability to regularly publish content. The last article I published was on December 30!
A month and a half without an article? I don’t like that one bit.
So to make up for the long post delay, I have extended all active paid subscriptions by 30 days, free of charge.
Thank you for sticking with me!
About the Project
I have been running this backrunner for a few days and seeing good results. It finds arbitrage opportunities, builds bundles, and fires them off as expected. Latency is good, computation load is low, and it is performant.
I’ve made no money because Jared’s sandwiches are more profitable, but that’s pretty typical. I’ve also spent nothing since I’m just sending silently-discarded bundles instead of trying to compete in the mempool.
The point of the exercise was to build a functional backrunner that uses Curve pools. And in that regard, it’s a success! I’ve also implemented some nice refinements to the general purpose monitor/update/calculate/execute functions that you’ll see again.
Arbitrage Helper
The Uniswap → Curve → Uniswap arbitrage helper is working pretty nicely. You can (and should) find and review it the degenbot repo.
I covered the helper in Part IV, so you can review it to see the general structure.
An arbitrage helper is only as good as its pool helpers, so please spend some time checking out the CurveStableswapPool class.
Pool Helper
There are currently 424 Curve V1 pools deployed. The unfortunate part of the early Curve ecosystem is that the pools are all customized. There is no standard implementation that I can code against, so many pools require a set of subtle quirks in their calculations. But the CurveStableswapPool class handles them all.
I have written a large test suite for the Curve ecosystem, which verifies the offchain calculations for all pools against the live contract. I’ve been running the test suite regularly over the past few weeks to identify any pools that fail to return integer-perfect results. The test failure frequency has dropped dramatically, so I’m confident that the accuracy is high across the range. I’ll continue to identify and apply pool quirks, but once a pool is fixed it’s fixed forever. Hooray!
Unfortunately there are lingering differences between the amounts returned by get_dy() / get_dy_underlying() and the state-modifying exchange() / exchange_underlying() functions.
I don’t think it’s worthwhile to continue tweaking these out, so I have settled on a “fudge factor’ approach to calculations within the arbitrage helper.
It simply applies a discount to the amount calculated by get_dy(). It’s set to 0.9999, which is a 0.01% reduction in the amount. It should only be relevant for extremely small arbitrage amounts, though I have not done a comprehensive test. If the author for a particular Curve pool has chosen to use wildly divergent equations for their on-chain swap vs the read-only pre-calculation… well, I think that’s odd but smart contractor authors can just be like that.
Each arbitrage helper sets an attribute self.curve_discount_factor when it is built, which you can adjust if needed.
Gas Use Tradeoffs
Curve V1 pools always do a transfer from/to msg.sender. Uniswap pools allow you to set different destination addresses for your swap, but Curve V1 always expects that you’re paying and receiving directly. Thus, Curve V1 pools in a 3-pool (triangle) arbitrage always generate at least two more transfers than an equivalent Uniswap pool.
Swap fees vary by trade size, and transfer fees vary by fee environment, so sometimes this helps and sometimes hurts.
State Monitoring Considerations
Curve V1 pools do not emit events with enough information to construct a deterministic state. This means that instead of getting a snapshot of the pool state, and passively updating it, we need to take a more active role.
I have observed the following events emitted by state-modifying transactions:
TOKEN EXCHANGE
TOKEN EXCHANGE UNDERLYING
ADD LIQUIDITY (4 coins)
ADD LIQUIDITY (3 coins)
ADD LIQUIDITY (2 coins)
REMOVE LIQUIDITY (4 coins)
REMOVE LIQUIDITY (3 coins)
REMOVE LIQUIDITY (2 coins)
REMOVE LIQUIDITY (1 coin)
REMOVE LIQUIDITY IMBALANCE (4 coins)
REMOVE LIQUIDITY IMBALANCE (3 coins)
REMOVE LIQUIDITY IMBALANCE (2 coins)
It is simple to observe these events and call auto_update() on the appropriate pool whenever we observe a Curve V1 event that implies an updated state. Many of these pools are not frequently updated, so latency and computation loads are low.
Smart Contract
The degenbot codebase generates a standard payload format:
addresscalldatamsg.value
The smart contract is a executor to deliver these payloads, perform an optional check, and deliver a variable, optional bribe to the block builder via a block.coinbase transfer.
It imposes some limits on the number of payloads and their length, but otherwise will execute each without modification, in the submitted order.
It is very simple by design, and should look familiar to readers. It does not contain any special functionality in the V3 callback, except the factory address check.
ethereum_executor.vy
#pragma version >=0.3.10
from vyper.interfaces import ERC20 as IERC20
interface IWETH:
def deposit(): payable
interface IUniswapV3Pool:
def factory() -> address: view
def fee() -> uint24: view
def tickSpacing() -> int24: view
def token0() -> address: view
def token1() -> address: view
def swap(
recipient: address,
zeroForOne: bool,
amountSpecified: int256,
sqrtPriceLimitX96: uint160,
data: Bytes[32]
) -> (int256, int256): nonpayable
OWNER_ADDR: immutable(address)
WETH_ADDR: constant(address) = (
0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2
)
V3_POOL_INIT_CODE_HASH: constant(bytes32) = (
0xe34f199b19b2b4f47f68442619d555527d244f78a3297ea89325f843f87b8b54
)
MAX_PAYLOADS: constant(uint256) = 16
MAX_PAYLOAD_BYTES: constant(uint256) = 1024
struct payload:
target: address
calldata: Bytes[MAX_PAYLOAD_BYTES]
value: uint256
@external
@payable
def __init__():
OWNER_ADDR = msg.sender
# Wrap initial Ether to WETH
if msg.value > 0:
IWETH(WETH_ADDR).deposit(value=msg.value)
@external
@payable
def execute_payloads(
payloads: DynArray[payload, MAX_PAYLOADS],
balance_check: bool = True,
bribe_bips: uint256 = 0
):
assert msg.sender == OWNER_ADDR, "!OWNER"
weth_balance_before: uint256 = empty(uint256)
weth_balance_after: uint256 = empty(uint256)
if balance_check:
weth_balance_before = IERC20(WETH_ADDR).balanceOf(self)
for _payload in payloads:
raw_call(
_payload.target,
_payload.calldata,
value=_payload.value,
)
if balance_check:
weth_balance_after = IERC20(WETH_ADDR).balanceOf(self)
assert weth_balance_after > weth_balance_before, "WETH BALANCE REDUCTION"
# For a 1% bribe set = 100 bips
# For a 10% bribe set = 1_000 bips
# For a 100% bribe set = 10_000 bips
if bribe_bips > 0:
send(
block.coinbase,
min(
msg.value,
bribe_bips * (
weth_balance_after - weth_balance_before
) / 10_000,
)
)
@external
@payable
def uniswapV3SwapCallback(
amount0_delta: int256,
amount1_delta: int256,
data: Bytes[32]
):
# reject callbacks that did not originate from the owner's EOA
assert tx.origin == OWNER_ADDR, "!OWNER"
# get the token0/token1 addresses and fee reported by msg.sender
factory: address = IUniswapV3Pool(msg.sender).factory()
token0: address = IUniswapV3Pool(msg.sender).token0()
token1: address = IUniswapV3Pool(msg.sender).token1()
fee: uint24 = IUniswapV3Pool(msg.sender).fee()
assert msg.sender == convert(
slice(
keccak256(
concat(
0xFF,
convert(factory,bytes20),
keccak256(_abi_encode(token0, token1, fee)),
V3_POOL_INIT_CODE_HASH,
)
),
12,
20,
),
address
), "INVALID V3 LP ADDRESS"
# repay token back to pool
if amount0_delta > 0:
IERC20(token0).transfer(
msg.sender,
convert(amount0_delta, uint256)
)
elif amount1_delta > 0:
IERC20(token1).transfer(
msg.sender,
convert(amount1_delta, uint256)
)
else:
raise "REJECTED 0 LIQUIDITY SWAP"
@external
@payable
def __default__():
# accept basic Ether transfers to the contract with no calldata
if len(msg.data) == 0:
return
# revert on all other calls
else:
raiseBot Structural Improvements
The bot example includes many nice features that you might want to re-implement, backport, or extend.
Single Watcher Coroutine
Web3py version 6 includes a very nice new feature: the ability to set up a persistant websocket connection and receive multiple eth_subscribe updates with a single connection.
The relevant web3py documentation covers the method of connecting and polling for updates. The more tangible improvement here is that these familiar subscription coroutines:
watch_eventswatch_pending_subscriptionswatch_new_blocks
can now be combined into a single coroutine that watches all three!
A key advantage of this approach is that synchronization is no longer necessary, since the subscription notifications occur in a predictable order (new block, then new events, with pending subscriptions whenever).
Another nice bonus is that the notification results are passed through the web3py middleware stack. This means that all notification values have a common, predictable format. Anyone who has spend time transforming JSON results to bytes, ints, and strings will appreciate this.
Look for the subscription_watcher coroutine, and at the focused helper functions within (handle_event, handle_block, and handle_pending_transaction) that execute various actions whenever a new notification arrives.
Threaded Helper Loading
Startup can be painful depending on the number of arbitrage paths you have selected. Parsing the paths, then creating all of the pools and arbitrage helpers takes time. Most of the delay is network-based, since JSON-RPC just isn’t a high performance API. It is difficult to offload this burden, because web3py is (by default) a synchronous module that will blocks the event loop while it waits for network results.
This behavior can be improved by using the AsyncWeb3 class, but it’s currently not practical to convert all of degenbot to be fully async.
Since the arbitrage loading only occurs once, it can be easily offloaded to a separate thread. Threads in Python are light and ideal for offloading blocking tasks that are otherwise I/O bound. Not useful for arbitrage calculation (we use processes for this), but threading allows the watcher loop to makes its cycle without being blocked by the arb loading process. There is some slight complication with stopping the thread once it has started (look for use of the threading.Event class if you want to see how this is managed), but you largely don’t need to worry about it unless you’re doing frequent interrupted startups.
Self-Limiting Calculations
To keep the subscription watcher free and responsive, processing of any pending transaction is done in a separate task, created on-demand.
The downside of this approach is that during periods of high activity, many tasks can be created and choke the event loop.
The processing task now sets and unsets a processing status flag when it starts working. Any new task will check this flag before starting work, and suspend if it finds that another task is currently executing.
In this way, one pending transaction must be fully processed from start to finish. However with judicious use of process pools and pre-calculation checks, this typically takes a few-tenths of a second for a transaction that lies across ~50 arbitrage paths. Most pending transactions are discarded because they are irrelevant, so in practice the bot can keep up quite well under load. And now when tasks queue up, the performance doesn’t drop.
Concurrent programming offers no guarantee of execution order, however, so during periods of high activity you may find that transactions are processed in unexpected orders depending on when they were scheduled.
A more robust scheduler system based on recency (last-in-first-out) is likely the best approach going forward.
Self-Registering Arbitrage Helpers
The UniswapCurveCycle employs a nice technique during instantiation. It will subscribe to updates from any pool used along its path.
These registrations are recorded by the pool and available any time. So now instead of complicate parsing of arbitrage paths, you can identify relevant arbitrage helpers by calling get_arbitrage_helpers() on any pool used by a mempool transaction.
In the future this will be extended to support triggered notifications whenever certain conditions are met (such as a pool sending a notification when a price is crossed, instead of having to poll it).
Version Info
This project uses degenbot version 0.2.0, and Vyper version 0.3.10. You can install both using pip.
The minimum supported version of Python is 3.10.