To understand how I landed on Brent as the default solver in degenbot, we need to understand the major categories that classify optimization methods.

Calculus is the branch of mathematics concerning rates of change. The most familiar operation in calculus is the derivative, which expresses the rate of change of some function with respect to its input.

I studied mechanical engineering so I often think of calculus operations with respect to a mapped physical system.

Let’s consider a function that maps the displacement of some object from its origin over time:

This is a simple linear displacement function whereby the object is displaced at a constant rate.

The velocity of the object over time is modeled by the first derivative of the original function:

The d/dt notation tells us that the derivative of the enclosed function f is taken with respect to the variable t. The “with respect to” clause is somewhat redundant for a function with only one variable. The distinction becomes important when evaluating systems with multiple variables.

The mechanics of taking derivatives are not relevant here, but the expression above resolves to:

Or more plainly, the velocity of the object is a constant value.

The second derivative can expressed over the original function or its first derivative:

The second derivative of displacement represents acceleration. We know that an object at constant velocity is not accelerating, so that value is zero.

There are names for higher-order derivatives, but the first and second are usually all that you need to work through most problems.

These concepts will appear later when we study Newtonian methods.

Fermat’s Theorem

The study of the first derivative allows us to determine some of the system’s behavior. For systems that exhibit variable behavior, we can identify certain interesting points where the change becomes zero. This may indicate that the system has ceased to change (an object falls from a height and lands without bouncing), or that it has momentarily changed direction (that same object falls from a height, then bounces in the reverse direction).

In either case, the first derivative becomes zero are one or more points called local extrema. These indicates that the function has been maximized or minimized within some range. These local extrema may also represent global extrema, but not necessarily.

Black Box Functions

The example above is intentionally simplistic. For functions like these, we can solve for the minimum or maximum directly by taking the first derivative and solving that equation for all inputs that result in zero. Then we can inspect the original function at those values and easily identify the global extrema.

But life is not always easy, and we often want to optimize a system that can only be observed from the outside. These systems are “black boxes” because we have visibility on the inputs and outputs, but incomplete visibility on how they work. We can measure how the system changes empirically and approximate the first derivative, but the absence of a formal output function for that first derivative makes the direct solution infeasible.

Derivative-Free Methods

Several methods have been developed to handle the task of optimizing black box functions where the derivative cannot be calculated, or where that calculation is unacceptably costly.

These are commonly known as derivative-free algorithms, and they operate iteratively — varying the output and studying the change in output until converging on an optimum within some tolerance.

These methods are generally well understood and documented, which implies they are likely part of the AI training data used to build popular models.

Using Kimi K2.5, I described the test setup and asked it to summarize derivative-free methods available in SciPy and how they could be used to solve the problem.

This is the result:

Nelder-Mead Simplex Method

Nelder-Mead maintains a geometric simplex of n+1 vertices (a triangle in 2D, tetrahedron in 3D) where each vertex represents a candidate solution vector of trade amounts across pools. At each iteration, it evaluates the profit at all vertices by delegating to pool classes, identifies the worst-performing vertex, and attempts to improve it by reflecting it through the centroid of the remaining vertices. Depending on how the reflected point compares to the current best and second-worst values, the method may expand (take a larger step), contract (take a smaller step toward the centroid), or shrink the entire simplex toward the best vertex. This approach requires only objective comparisons—no gradients or smoothness assumptions—making it robust to the piecewise-constant regions and sudden jumps created by trading fees and liquidity limits in AMM pools.

Powell’s Method (Conjugate Direction Method)

Powell’s method performs sequential line minimizations along a set of directions that are iteratively updated to become conjugate (non-interfering) based on the observed improvement patterns. Starting with coordinate axes as initial search directions, it minimizes the profit function along each direction in turn by finding the optimal step size, then constructs a new conjugate direction from the overall displacement achieved during the cycle. The method replaces the direction that contributed least to the total improvement with this new conjugate direction, gradually learning the local curvature of the objective without ever computing derivatives. While efficient for smooth, locally quadratic objectives, Powell’s method can stall when encountering flat regions or discontinuities common in arbitrage problems where profits remain constant until crossing a fee threshold or liquidity boundary.

Differential Evolution (DE)

Differential Evolution is a population-based evolutionary algorithm that maintains a fixed-size swarm of candidate solution vectors, each representing a complete set of trade amounts across pools. For each generation, every individual in the population creates a mutant vector by adding the weighted difference of two randomly selected population members to a third base individual, effectively exploring directions suggested by the population’s diversity. The mutant then undergoes crossover with the original individual to create a trial vector, which is evaluated against the current individual by delegating both to pool classes for profit calculation; the better performer survives to the next generation. This stochastic, parallel search naturally explores multimodal landscapes with multiple local profit peaks (arising from different pool path combinations), handles discontinuities without issue, and can be embarrassingly parallelized across available compute resources.

Covariance Matrix Adaptation Evolution Strategy (CMA-ES)

CMA-ES maintains a multivariate Gaussian distribution over the search space, where the mean represents the current best estimate of optimal trade amounts and the covariance matrix captures the pairwise correlations between amounts across different pools. At each iteration, it samples a population of candidate solutions from this distribution, evaluates their profit by delegating to pool classes, and updates the distribution parameters by re-weighting samples based on their ranked performance (not absolute values). The covariance matrix adaptation learns the local structure of the objective—such as the inherent coupling between trade amounts in multi-hop paths dictated by constant product formulas—allowing the algorithm to make elongated steps along valleys and navigate ill-conditioned problems efficiently. While highly sample-efficient and self-adaptive regarding step sizes, CMA-ES assumes local quadratic behavior and may stagnate if the profit landscape contains large truly flat regions where all sampled points yield identical values.

Constrained Optimization by Linear Approximation (COBYLA)

COBYLA is a trust-region algorithm designed for constrained black-box optimization that maintains both the current iterate and a set of “trust region” points used to construct linear approximations of the objective and constraint functions. At each iteration, it solves a linear programming subproblem using these approximations to find a step direction that improves profit while respecting constraints (such as slippage limits, available liquidity, or gas bounds), then evaluates the true objective and constraints at the proposed point by delegating to pool classes. The approximations are updated by replacing the oldest trust region point or adjusting the trust region radius based on how well the linear model predicted the actual function values. Unlike penalty-based methods, COBYLA maintains feasibility throughout the optimization (when started from a feasible point) and handles hard constraints natively without requiring derivative information, making it ideal for arbitrage scenarios where constraints like maximum slippage are non-negotiable safety limits.

Bayesian Optimization (Gaussian Process-Based)

Bayesian optimization builds a probabilistic surrogate model—typically a Gaussian Process—over the profit function, treating each evaluation of the black-box pool delegation as an observation that updates the posterior belief about the objective landscape. The Gaussian Process provides both a mean prediction of profit and an uncertainty estimate at any untested point, enabling the algorithm to use an acquisition function (such as Expected Improvement or Upper Confidence Bound) to intelligently select the next trade amount vector that optimally balances exploitation (high predicted profit) against exploration (high uncertainty). After each pool evaluation, the surrogate model is refit and the acquisition function optimized to suggest the next candidate, creating a highly sample-efficient search that minimizes expensive calls to slow or rate-limited pool simulations. While exceptionally efficient when evaluations are costly, the computational overhead of fitting Gaussian Processes scales cubically with the number of observations, making this approach suitable for offline path planning or high-stakes arbitrage validation rather than millisecond-latency production trading.

Hooke-Jeeves Pattern Search

Hooke-Jeeves performs a two-phase iteration consisting of exploratory moves and pattern moves: the exploratory phase samples points along each coordinate direction from the current base point by taking fixed-size steps, accepting any move that improves profit (even marginally), and uses the resulting point as the new base if any improvement was found. The pattern move then attempts to exploit the discovered trend by jumping from the original base point in the direction of the net displacement achieved during exploration, essentially accelerating along promising directions. If pattern moves continue to succeed, the step size is maintained or increased; if exploration fails to find improvement at a given resolution, the step size is reduced for finer local search. This deterministic, direction-based approach requires no derivatives and minimal bookkeeping, but its reliance on fixed exploratory step sizes and coordinate-aligned searches can lead to slow convergence on rotated or highly correlated objective landscapes like those arising from coupled pool interactions.

Basin-Hopping with Local Refinement

Basin-hopping is a two-stage global optimization strategy that combines random perturbations with local optimization to escape local profit maxima and find the global optimum across disconnected feasible regions. The algorithm starts from an initial trade amount vector and applies a local optimizer (such as Nelder-Mead or Powell’s method) to find the nearest local maximum, then randomly perturbs this solution by adding noise scaled to the problem dimensionality and accepts or rejects the new starting point based on a Metropolis criterion or strict improvement requirement. This process iterates, effectively allowing the solver to “hop” between basins of attraction in the profit landscape, which is essential for arbitrage problems where different pool paths create multiple distinct local optima. The method is particularly effective when combined with aggressive local optimizers that quickly refine each basin to its peak, trading off the number of expensive pool evaluations per hop against the probability of discovering the globally optimal arbitrage path.

Simulated / Dual Annealing

Simulated annealing is a probabilistic meta-heuristic inspired by the metallurgical annealing process, where the algorithm starts with high “temperature” allowing large random steps and gradually cools, reducing both step sizes and the probability of accepting worsening moves according to a Boltzmann-like acceptance criterion. At high temperatures, the solver explores broadly across the trade amount space, delegating to pools and accepting occasional profit decreases to escape local optima; as temperature decreases, it increasingly rejects inferior solutions and converges toward the global maximum. Dual annealing extends this with an adaptive strategy that combines classical annealing with local search visits and uses a fast annealing schedule for global exploration. While theoretically guaranteed to find the global optimum given infinite time and logarithmic cooling, simulated annealing typically requires many more objective evaluations than population-based methods to achieve comparable convergence, making it less suitable for time-sensitive arbitrage scenarios unless profit evaluations are extremely cheap.

Just Put My Benchmark in the Bag Bro

That’s nice and all, but what we really care about it speed and accuracy. So let’s ask the model to implement and evaluate them:

Read more

Numerical optimization is a broad category concerning the question “what is the optimal input that satisfies the constraints of my problem?”

This question is highly relevant to us. After identifying a price mismatch across some asset market, we must determine the most profitable route to capture value and how to size the associated trades.

I first covered numerical optimization with a lesson covering the Brent solver implemented in SciPy.

Later I focused on convex optimization:

Playing To Your Strengths

It’s important to understand your strengths and weaknesses, and even more important to choose efforts that are helped by your strengths, not harmed by your weaknesses, or ideally both.

I’m above average at abstract thinking, verbal reasoning, working memory, and synthesis (applying techniques from multiple domains). But I’m below average at mental processing speed, spatial reasoning, and numeric intuition.

This sets me up for trouble when I’m faced with certain classes of problems, and it compounds if that problem has multiple non-obvious solutions.

When I encounter some numerical optimization problem, I often have little intuition about how to attack it, falling back on brute-force methods and taking the first solution that works without spending much time on it.

Luckily, I now have access to highly capable AI tools that can fill in these gaps.

I already showed how to use AI to boost feature development and debugging.

Then I showed how to use it to quickly iterate on developing smart contracts.

Now I’m going to demonstrate how I’m using it to refine parts of my code base with methods that would be tedious to apply by hand.

This will be a multi-part series because the scope is too broad for a single entry. Even after I reach a reasonable stopping point, I may continue adding new entries as I discover and implement new techniques.

Problem & Test Setup

We need a way to compare methods across runs, so we need to build a reproducible set of inputs that model a relevant problem in our domain.

Since the majority of our work concerns liquidity pools from Uniswap and its various forks, we need a way to generate synthetic pool states associated with a given arbitrage scenario. I gave this to an agent, which ultimately proposed this architecture:

   ┌─────────────────────────────────────────────────────────────────────┐
   │                         FixtureFactory                              │
   │  ┌───────────────────────────────────────────────────────────────┐  │
   │  │                    PoolStateGenerator                         │  │
   │  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────────┐│  │
   │  │  │  V2 Pools   │  │  V3 Pools   │  │  V4 Pools               ││  │
   │  │  │  (reserves) │  │(tick-based) │  │(pool ID + tick-based)   ││  │
   │  │  └─────────────┘  └─────────────┘  └─────────────────────────┘│  │
   │  └───────────────────────────────────────────────────────────────┘  │
   └─────────────────────────────────────────────────────────────────────┘
                                 │
                                 ▼
                      ┌──────────────────────┐
                      │ ArbitrageCycleFixture│
                      │ (frozen dataclass)   │
                      └──────────────────────┘

I’m omitting considerable detail on how this works under the hood, but the takeaway is that we can use this setup to mix and match arbitrage scenarios across the different pool types.

Scenario: Basic Two-Pool

Generate a basic scenario using two Uniswap V2 pools with a fixed price ratio and a target liquidity:

def simple_v2_arb_profitable(self) -> ArbitrageCycleFixture:
    """
    Two V2 pools with 2% price difference.

    Simple profit: arbitrage from higher to lower price pool.
    Pool A: 1 ETH = 2000 USDC (price = 2000)
    Pool B: 1 ETH = 1960 USDC (price = 1960, 2% lower)

    Arbitrage: Buy ETH in pool B (cheaper), sell in pool A (more expensive).
    """
    pool_a_address: ChecksumAddress = cast(
        "ChecksumAddress", "0x0000000000000000000000000000000000000001"
    )
    pool_b_address: ChecksumAddress = cast(
        "ChecksumAddress", "0x0000000000000000000000000000000000000002"
    )
    input_token_address: ChecksumAddress = cast(
        "ChecksumAddress", "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"
    )  # USDC

    # Generate pools with 2% price difference
    pool_a, pool_b = self._generator.generate_profitable_v2_pair(
        pool_a_address=pool_a_address,
        pool_b_address=pool_b_address,
        fee_a=Fraction(3, 1000),
        fee_b=Fraction(3, 1000),
        price_ratio=1.02,
        liquidity_base=10**21,  # ~1000 ETH equivalent
    )

    return ArbitrageCycleFixture(
        id="simple_v2_arb_profitable",
        cycle_type="v2_v2",
        pool_states={pool_a_address: pool_a, pool_b_address: pool_b},
        input_token_address=input_token_address,
        expected_optimal_input=0,  # Calculated by solver
        expected_profit=0,  # Calculated by solver
    )

SciPy Benchmark

Now let’s compare the three methods implemented by SciPy’s minimize_scalar solver (Bounded, Brent, and Golden):

Read more

I have built many execution contracts before for various projects (search the archives if you’d like a refresher on them).

They all have a similar structure:

• Validate that the caller is authorized, then process their inputs

• Record the starting balance

• Perform a sequence of validated actions across a set of smart contracts

• Record the final balance, reverting if it has decreased

This is a well-understood pattern with plenty of examples, which is an ideal setup for agentic AI. My last entry on agentic coding was well-received, so I expect this will be useful too.

Agent Setup

For this effort, I created a new project in directory ~/code/aave_v3_liquidator and created a minimal Python 3.12 virtual environment using uv with these packages:

• eth-ape with the recommended-plugins extra

• degenbot 0.6.0a2

• vyper 0.4.3

I used Pi version 0.66.1, calling Kimi K2.5 through Synthetic.

Session Transcript

You can review the complete transcript if you’re curious.

Let’s break down the session with some context to help read between the lines:

The Task

I started the session with a task definition:

Help me prepare a plan for this project: create a Vyper smart contract that can execute an atomic Aave V3 liquidation against a given user. It will use the Uniswap V4 PoolManager as its "bank" and "market", i.e. it should perform an unlock, then within the V4 callback, execute a liquidationCall at the Aave V3 Pool and a swap at the V4 PoolManager to handle these operations: a withdrawal of the debt token (sent to the Aave V3 Pool), a deposit of the collateral token (received from the Aave liquidation), and optionally another swap to consolidate profit in a given token. There are some subtleties regarding execution order that we will have to study, but I can retrieve contract source code as needed.

Then it thought for a bit, proposed a design, and gave a summary:

Next steps:

• I can retrieve the Aave V3 Pool source code to verify liquidationCall return values

• I can retrieve Uniswap V4 PoolManager interface to confirm unlockCallback signature

• Do you have a specific network in mind (Ethereum mainnet, testnet)?

Would you like me to start implementing the contract, or should we fetch and analyze the dependency contracts first to nail down the exact interfaces?

So I gave it some more context:

We are working on Ethereum mainnet. Retrieve the Aave V3 Pool source code by using cast. First retrieve the implementation address from the proxy contract at 0x87870Bca3F3fD6335C3F4ce8392D69350B4fA4E2, then get its source. The Uniswap V4 PoolManager contract is not a proxy, so retrieve its source directly at 0x000000000004444c5dc75cB358380D2e3dE08A90

It read the cast skill to learn how to use Foundry correctly, then downloaded the contract source code for the Aave V3 Pool implementation contract at 0x8147b99df7672a21809c9093e6f6ce1a60f119bd and the Uniswap V4 PoolManager. It read through them to discover the execution flow for the key functions, and then summarized the operations:

I’ve retrieved both contracts. Here are the key findings:

Summary of Retrieved Contracts

Aave V3 Pool (Proxy: 0x87870Bca3F3fD6335C3F4ce8392D69350B4fA4E2)

• Implementation: 0x8147b99df7672a21809c9093e6f6ce1a60f119bd

• Key Function:

function liquidationCall(
    address collateralAsset,
    address debtAsset,
    address borrower,
    uint256 debtToCover,
    bool receiveAToken  // false = receive underlying collateral
) external;

• No return value - collateral is transferred directly to msg.sender

Uniswap V4 PoolManager (0x000000000004444c5dc75cB358380D2e3dE08A90)

• Key Interface:

function unlock(bytes calldata data) external returns (bytes memory);

// Callback that MUST be implemented
interface IUnlockCallback {
    function unlockCallback(bytes calldata data) external returns (bytes memory);
}

function take(Currency currency, address to, uint256 amount) external;
function settle() external payable returns (uint256 paid);
function settleFor(address recipient) external payable returns (uint256 paid);

function swap(PoolKey memory key, SwapParams memory params, bytes calldata hookData)
    external returns (BalanceDelta swapDelta);

struct SwapParams {
    bool zeroForOne;
    int256 amountSpecified;  // negative = exact input, positive = exact output
    uint160 sqrtPriceLimitX96;
}

Updated Execution Plan

Based on the source code, here’s the refined execution flow:

┌──────────────────┐
│  unlock(data)    │
│   (caller = us)  │
└────────┬─────────┘
         ↓
┌──────────────────┐     take(debtAsset, amount)       ┌─────────────┐
│ unlockCallback() │  ───────────────────────────────→ │  PoolManager│
│   (us via PM)    │                                   │             │
└────────┬─────────┘                                   └─────────────┘
         ↓
┌──────────────────┐
│approve(debtAsset)│  →  liquidationCall(...)  →  receive collateral
│  (to Aave Pool)  │
└────────┬─────────┘
         ↓
┌──────────────────┐     Optional: swap collateral→profit
│  Optional swap   │  ───────────────────────────────────────────────→
│ (if consolidate) │
└────────┬─────────┘
         ↓
┌──────────────────┐     swap/debt operations to acquire debtAsset
│  Acquire debt    │  ───────────────────────────────────────────────→
│  for repayment   │
└────────┬─────────┘
         ↓
┌──────────────────┐     settle() - repay flash borrow
│     settle()     │  ───────────────────────────────────────────────→
│  (debtAsset)     │
└──────────────────┘
         ↓
    Validation: All deltas must be zero
         ↓
    Return profit to caller

Key Implementation Details

1. Execution Order (Critical)

# Inside unlockCallback:
1. take(debtAsset, self, debt_amount)    # Create liability
2. approve(debtAsset, aave_pool, debt_amount)
3. liquidationCall(...)                   # Receive collateral
4. [Optional] swap(collateral→profit)    # Consolidate
5. swap(profit→debtAsset) or take        # Acquire repayment funds
6. settle(debtAsset)                      # Clear liability

2. Vyper Interface Definitions

# Aave V3 Pool
interface IPool:
    def liquidationCall(
        collateralAsset: address,
        debtAsset: address,
        borrower: address,
        debtToCover: uint256,
        receiveAToken: bool
    ): nonpayable

# Uniswap V4 PoolManager
interface IPoolManager:
    def unlock(data: Bytes[1024]) -> Bytes[1024]: nonpayable
    def take(currency: address, to: address, amount: uint256): nonpayable
    def settle() -> uint256: payable
    def swap(
        key: PoolKey,
        params: SwapParams,
        hookData: Bytes[256]
    ) -> BalanceDelta: nonpayable

# Must implement
interface IUnlockCallback:
    def unlockCallback(data: Bytes[1024]) -> Bytes[1024]: nonpayable

3. Critical Subtleties

• Callback validation: Check msg.sender == pool_manager inside unlockCallback

• PoolKey ordering: V4 requires currency0 < currency1; need sorting helper

• BalanceDelta: Track swap returns - negative means received, positive means owed

• Token approvals: Infinite approve Aave Pool and PoolManager in constructor

• Native ETH: Use WETH for ETH positions; PoolManager handles via Currency type

4. Profit Flow

1. Debt tokens are taken from PoolManager (creates liability)

2. Liquidation repays debt, receives collateral (+ liquidation bonus)

3. Optional: Swap collateral for preferred profit token

4. Acquire debt tokens to settle the liability

5. Remaining tokens = profit → transfer to caller

Would you like me to proceed with implementing the Vyper contract now, or do you need any clarification on the execution flow?

Most models are trained to be helpful and maintain velocity, so you’ll notice that it “wants” to guide us down the route of progress: either begin implementing the contract immediately, or offer explanation of its proposed design.

Here is the point where things could go off the rails. There are a lot of assumptions baked into the proposed design, and I like to bring them to the surface.

So I choose a third route:

Read more

To properly evaluate the health of a particular user, we need to know all of their collateral and debt positions. In Part V, we did this by hand using mentally mapped asset prices.

Now we want to replace that process (with its assumptions and simplifications) with one that scales across all of Aave’s assets. To do this, we need to look closely at how asset prices are determined so that we can track them in the same way.

Price Oracles

I’ve discussed price oracles before in a limited way, focusing on the price of a single asset.

Aave uses an oracle contract that acts as a bridge between its internal operations and the external market. The current oracle is available at the PoolAddressProvider contract via the getPriceOracle function. The oracle was set shortly after the mainnet market deployment and has not changed since, but that is not a guarantee that it will remain that way. It can be changed at any time by governance.

Asset Sources

The oracle maintains a single source for each of the registered assets. Whenever this source is updated, the oracle emits an AssetSourceUpdated event. The Aave updater in degenbot tracks these events and uses them to update the database.

Asset Prices

The oracle’s price discovery mechanism is the getAssetPrice function, which retrieves the source for a given asset, then calls source.latestAnswer():

/// @inheritdoc IPriceOracleGetter
function getAssetPrice(address asset) public view override returns (uint256) {
  AggregatorInterface source = assetsSources[asset];

  if (asset == BASE_CURRENCY) {
    return BASE_CURRENCY_UNIT;
  } else if (address(source) == address(0)) {
    return _fallbackOracle.getAssetPrice(asset);
  } else {
    int256 price = source.latestAnswer();
    if (price > 0) {
      return uint256(price);
    } else {
      return _fallbackOracle.getAssetPrice(asset);
    }
  }
}

latestAnswer() should be familiar if you’ve looked at a Chainlink contract. Note that the oracle does not have to be a Chainlink contract, but most of them are. The only requirement of the oracle is that is provides the latestAnswer interface.

So all we need is the source for each asset, and a way to observe the price when it changes.

Let’s consider Aave’s Ether/WETH asset. The oracle tells us that the price source is contract 0x5424384B256154046E9667dDFaaa5e550145215e:

And that the asset’s current price is 205429135979 ($2,054):

We can also inspect latestAnswer() directly at the source contract:

The values match, which we expected.

Chainlink SVR

It’s not obvious from reading the contract source, but this is a special kind of feed contract labeled as a Chainlink Smart Value Recapture (SVR) feed also specific to the Aave protocol.

Read more

Coincidentally I’ve published a new degenbot release with Aave V3 position scraping enabled. If you’re interested in testing it, you can install it using pip install degenbot==0.6.0a1.post1.

You can populate your database with positions using these commands:

• Upgrade the database using degenbot database upgrade

• Activate the market using degenbot aave activate ethereum_aave_v3

• Scrape the positions using degenbot aave update

In addition to Aave V3, this release is notable because it’s the first one distributed with integrated Rust components via PyO3.

I will be integrating more Rust into the package, so please report any bugs that you encounter on these 0.6 releases.

The installation should work cleanly on Linux (x86_64, aarch64), MacOS (x86_64 and ARM64), and Windows (x86_64). If you have exotic hardware and want to help me test the package against those, please let me know!

But that’s a naive approach that ignores a lot of detail. Critically, it scales poorly — every successive update would have to scan for new events since the last block, discover new users, add them to the previous users, then do a full re-scrape of all of the previous plus new positions.

So instead of rely on that structure, I built the updater from the ground up. Along the way, I dug through many layers of unexpected complexity. It required a lot of untangling, but yielded valuable information about how Aave handles position accounting, upgrades, and changing incentives.

I will cover several areas where position tracking required additional layers of decoding and tracking. This will be particularly useful if you need to understand how the Aave contracts are wired together across a particular market.

Read more

I’ve learned a lot, and will certainly refer to these techniques again. So instead of folding it into an Aave specific write-up where it could be missed, I’m giving it proper treatment with its own introduction.

AI Fatigue

Every week I’m being told that it’s all over: a script will take my job, the robots are coming to kill me, RAM will never be affordable again, I should shut up and get in the pod.

It’s neither as bad nor as good as they say. Relentless media treatment has made “AI” a container word — people hear it, fill in ideas and things they’ve heard about it, and then stop thinking.

You can respond to AI any way you like. Here are my very important opinions on it:

• Seeing fake AI avatars everywhere is annoying, but I like generating coloring pages for my kids and diagrams for technical presentations.

• AI bots clogging up the replies on X is infuriating, but I like to get a summary of long documents.

• The chat bot on my car dealership’s website is stupid, but the one integrated into open-source tool documentation is very useful.

Coding Tools

AI has been around for many decades, and its sophistication and usefulness has been trending upwards. It crossed the usefulness threshold in recent years, and appears to be here to stay.

I only started paying attention to this a few years ago.

Autocomplete

I saw folks talking about GitHub Copilot, so tried it out. After a few days I turned it off — I didn’t find the auto-completion particularly useful, and was frequently distracted by the phantom text appearing when I stopped typing.

Web Chat

Then Claude launched with the familiar web interface. I saw more people talking about it, saying that Opus 4 was a good coding model, so I signed up for a few months.

I found it was very good at generating small scale code snippets that I could copy and paste into my project. But the experience fell apart quickly when the code didn’t work as expected — I had to dump long tracebacks into the chat and ask it for fixes, then perform another copy-paste cycle. I felt limited by the browser interface and frustrated by how poorly it performed as the session grew in length. I cancelled.

Agentic

Then I started seeing chatter about agentic coding techniques. The promise was that agents could be spawned which would operate independently towards some goal.

I was specifically interested in the coding agents. A coding agent is simply a tool that tracks user inputs, calls an AI service, interprets responses, executes tools on behalf of the AI and manages I/O.

Editor Based

The first coding agents tools I saw were hooked to a graphical editor via a plug-in or natively: Cursor, Windsurf, Cline, Zed, Roo.

I tried them and enjoyed the ability to easily focus the tool on a section of code or an error message in the terminal. I continued to get nice results on small-scale tasks, but large refactoring efforts felt quite clumsy and hard to track if they needed multiple phases. I had the feeling I had traded writing code (which I enjoy very much) for micromanaging a capable but easily distracted person (which I dislike very much).

Command Line Based

Then Claude Code dropped in the middle of last year and everything changed!

It’s difficult to separate the improvement in coding tools from the improvement in coding-focused AI models, so I won’t try. But I will assert that console tools like Claude Code are a huge improvement over both web tools and editor tools.

The web comparison is easy to demonstrate. Imagine using Claude Opus 4.5 through the web interface. Still very capable, but having to copy and paste is cumbersome and not scalable.

The IDE tool comparison is harder, but the weakness shared with the web tool is the mode of user interaction. You still enter prompts with a keyboard and approve actions with a mouse. Very compatible with the typical way people work, but still limited by user attention and availability.

Code Is Now Cheap

In a post-AI world, code has become cheap so you should write less of it!

Your time is now too expensive to spend writing your own loops. But what should you do instead?

You Can Just (Not) Do Things

Appearing busy is glamorous: you look important and get the warm feeling that you’ve maximized your productivity.

But the result of your efforts can vary wildly depending on the tasks you select.

For coders, this shows up as a ratio between features and products shipped vs. time spent creating it.

In terms of processing speed and I/O, humans are the bottleneck in any process involving computers. So learning how to extract yourself from the “hot” loop is critical.

Several approaches have come from this realization.

Vibe Coding

Andrej Karpathy’s famous “vibe coding” tweet has pushed many people towards giving the AI more control. Instead of trying to act as the brain and letting the AI be your fingers, vibe coding encourages stepping away and reviewing the output later, then nudging the result to taste.

The tweet literally says it’s “not too bad for throwaway weekend projects”, so it’s obviously not a good technique for writing production software.

Once you see the spaghetti generated from a long vibe coding session, you’ll realize the problem. AIs are good at generating code — the more you use it the more you get.

The usual AI solution to handling complex inputs tends towards branching and special-case fixes. Ask it to consolidate the code using DRY techniques and it will just extract a bunch of helper functions or hide the branching in some new method, without resolving the architectural issue. It can quickly spiral away when not properly constrained.

Ralph Wiggum

I recommend watching this primer on the Ralph Wiggum loop by its creator, Geoffrey Huntley:

Here is the basic Ralph technique:

• Generate a highly detailed plan and specifications for what you want to build

• Run a coding agent in a loop with these instructions:

  • Read the plan

  • Read the specifications

  • Select the most important single task from the plan

  • Complete it

  • Run the tests

  • Commit

  • Mark the task as complete

  • Add important lessons learned to a file

  • If all tasks are complete, take some action that terminates the loop

In this way, each iteration of the loop starts fresh with a context window focused on important items and a simple request: implement one task.

Each loop starts the agent with its context reset, so that it can load only what it needs instead of holding on to many irrelevant instructions that degrade accuracy.

Automation

I’m a huge fan of using command line interface (CLI) tools. They are flexible, high performance, and portable. But the killer feature is they can be automated!

This whole newsletter is about building autonomous stuff, so this shouldn’t surprise you. If I can eliminate a click or button press, or make something run on a schedule, I will. I’m willing to spend a lot of time converting a manual process to automatic.

So naturally when I code, I do it with agentic tools on the CLI that call AI models specifically trained for coding.

I have a strong bias towards open source, so I prefer using an open source agent calling open weight models through a neutral provider. Currently I use OpenCode with Kimi K2.5 via Synthetic (referral link, we’ll both get a promotional credit if you sign up).

Automatic Plan Summary

My first automation was simple: generate a daily summary of the open plans.

OpenCode allows you to define a custom command via a Markdown file. So I wrote one with instructions on how to do that:

---
description: Review project plans and provide a summary
agent: plan
model: synthetic/hf:moonshotai/Kimi-K2.5
---

Use the Glob tool to find work plans organized in `.opencode/plans`. Simple plans in the root directory are saved as individual files. Complex plans spanning multiple files are saved in a subdirectory.

For each plan, delegate the following actions:
- Review the progress described in the plan versus the referenced code
- Provide a summary of actual progress made
- Recommend updates to the plan to synchronize it with actual progress
- Recommend improvements to the plan

Then:
- Summarize the state of each plan and action items
- Prioritize the pending work
- Recommend one action item to make progress on each plan

This command can be executed by calling OpenCode with the appropriate option. Here you can see that it looks for the plans using the Glob tool (an abstraction over find) in the working directory, then delegates review of each plan to a subagent that starts fresh and can concentrate on the one plan it was given:

summarize.md

btd@dev:~/code/degenbot$ opencode run --command summarize

> plan · hf:moonshotai/Kimi-K2.5

✱ Glob ".opencode/plans/**/*" 7 matches
→ Read .opencode/plans/hybrid_configuration/implementation_plan.md
→ Read .opencode/plans/aave_transaction_refactor.md
→ Read .opencode/plans/uniswap_v3_pool_refactoring/plan.md
→ Read .opencode/plans/uniswap_v3_pool_refactoring/architecture_diagram.md
→ Read .opencode/plans/work-plan-aave-todos.md
→ Read .opencode/plans/organization_improvement.md
→ Read .opencode/plans/bootstrap_system.md
• Review hybrid configuration plan Explore Agent
• Review Aave transaction refactor plan Explore Agent
• Review Uniswap V3 pool refactoring plan Explore Agent
• Review Aave TODO work plan Explore Agent
• Review organization improvement plan Explore Agent
• Review bootstrap system plan Explore Agent

Now I'll delegate review tasks for each plan to assess progress against the referenced code and provide recommendations.

✓ Review hybrid configuration plan Explore Agent
✓ Review Aave transaction refactor plan Explore Agent

[...]

The result is a nice summary that I can review each morning before starting work. This can be easily triggered by a systemd script or a cron job:

Transaction Inspection

My next automation was more complex. I found that during debugging, I was often repeating instructions for the same set of actions: inspect an Etherscan page, tell me what happened during a particular transaction, tell me which contracts were used.

So I defined a read-only single task subagent that would do these things:

evm-investigator.md

---
description: Analyzes Ethereum or EVM-compatible blockchain transactions, accounts, and smart contracts.
mode: subagent
tools:
  write: false
  edit: false
  task: false
---

You are in Ethereum Virtual Machine (EVM) transaction investigation mode. Focus on:
- Smart contract source code
- Inspecting transactions using a block explorer (Etherscan or similar)
- User calldata
- Event logs
- Storage states
- Transaction tracing
- Function call arguments and return values

Within the terminal user interface (TUI), trigger its use by mentioning it with the @ symbol:

Pretty cool!

Automating Debugging

It’s also simple to write a command that glues the output of something run in a terminal to a subagent. This command runs my Aave updater after a failure, instructs the model to delegate investigating the transaction that caused it, then start a hands-off debugging effort that records lessons learned to a log:

aave_debug.md

---
description: Debug Aave update failures
agent: build
model: synthetic/hf:moonshotai/Kimi-K2.5
---

!`uv run degenbot aave update --no-progress-bar --one-chunk`

## DIRECTION: Investigate and debug this failed Aave update command

## PROCESS:
### 1. Gather Information
- Parse the output to identify the transaction hash associated with the event triggering the processing error
- Inspect the transaction using @evm-investigator

### 2. Investigate Code
- Review `aave_update` and determine the execution path leading to the error
- Check for related issues involving this error or a common execution path
- Generate a failure hypothesis

### 3. Validate Execution Path and Failure Hypotheses
- If the execution path is unclear, apply the `log_function_call` decorator to confirm function calls, e.g.,
    ```python
    @log_function_call
    def some_func(...): ...
    ```
- Determine if a debugging env var is useful:
    - `DEGENBOT_VERBOSE_USER=0x123...,0x456...`
    - `DEGENBOT_VERBOSE_TX=0xabc...,0xdef...`
    - `DEGENBOT_VERBOSE_ALL=1`
    - `DEGENBOT_DEBUG=1`
    - `DEGENBOT_DEBUG_FUNCTION_CALLS=1`
- Run with any verbosity flags prepended, e.g., `DEGENBOT_DEBUG=1 uv run degenbot aave update --no-progress-bar --one-chunk`

### 4. Fix & Validate
- If a hypothesis is validated and the root cause is clear, implement a fix and run the update again

### 5. Document Findings
Append to @aave_debug_progress.md. Follow this format:
- **Issue:** Brief title
- **Date:** Current date
- **Symptom:** Error message verbatim
- **Root Cause:** Technical explanation
- **Transaction Details:** Hash, block, type, user, asset
- **Fix:** Code location and changes
- **Key Insight:** Lesson learned for future debugging
- **Refactoring:** Concise summary of proposed improvements to code that processes these transactions

It’s now available in the TUI as slash command:

After running for a minute, it narrowed down and identified the bug:

And then gives me a nice little summary of what it did:

Unattended Ralph Loop

Here’s where we max the technique out, setting up an unattended development loop where your agent can develop new features, make fixes, do code review, and run experiments while you sleep.

Read more

Now let’s zoom out and build a means to monitor these positions over the long term.

We will focus on developing a database structure for Aave V3 using SQLAlchemy, writing an extractor function to pull the necessary events and update the database, then run some limited tests to verify the data behaves as expected.

This entry is heavily influenced by ideas from Domain-Driven Design. If you find it useful, consider reading the books “Domain-Driven Design: Tackling Complexity in the Heart of Software” by Eric Evans and “Implementing Domain-Driven Design” by Vernon Vaughan.

Read more

An Aave user can have multiple collateral deposits and borrows across the platform. But Aave tracks only that user’s overall health factor. We saw this when calling the getUserAccountData function on the Pool contract.

The collateral, debt, and available borrowing capacity values reported by that function are expressed in terms of a common base currency. This base currency is not defined by Aave, it is largely a function of the oracles used for each reserve asset. For example, to determine the collateral value of WETH against the debt value of USDC, an intermediate base currency is used. The Mainnet market uses USD as this intermediate.

So we must always be aware that Aave measures user health against this base currency. Aave reserve assets move against USD and against each other, which makes accounting more complex.

Example: if you’re considering a liquidation of a WBTC loan backed by WETH, you must be aware of at least three prices:

Read more

There is no onchain view that exposes a simple list of active loan positions or their users, so we’ll need to dig through transaction logs to discover the borrowers and look up their positions.

Each new loan operation emits a Borrow event, which is defined in the IPool.sol interface:

event Borrow(
    address indexed reserve,
    address user,
    address indexed onBehalfOf,
    uint256 amount,
    DataTypes.InterestRateMode interestRateMode,
    uint256 borrowRate,
    uint16 indexed referralCode
);

Using Ape, connect to Ethereum mainnet and query for Borrow events starting at block 23,000,000:

btd@dev:~$ ape console --network :mainnet
INFO:     Connecting to existing Reth node at http://localhost:8545.

In [1]: pool = Contract('0x87870Bca3F3fD6335C3F4ce8392D69350B4fA4E2')

In [2]: borrows = pool.Borrow.query('*', start_block=23_000_000)

In [3]: borrows
Out[3]: 
0       0x807c5ad511680ecb97cc3a9f9caaacb6f55c98799596... 
1       0xa40948b206cde4493e8772bb9cf2dd38ef020151a453... 
2       0x8ed9da1f0a1c0daedd0b1c7a98811078cb09945076e0... 
3       0xab6b82ec1d7df25fcf8826d9410013d7813b9c5fb514... 
4       0x6f8dd67ab48078d522144126929d194d12b49b89196c... 
...
...
191965  0xc24c257bbdfd99a3693912169190f863874c134e2ec0... 
191966  0xf906e5d7b5926444dabd3750a510df36e13658433383... 
191967  0x8c0a4488312fda581f4a5d01e7bab4446d8aa9412628... 
191968  0x893e30290be406185abc59cabfef540335f53428a80d... 
191969  0xcd5c580418e3cb18cd5b2e730f671459dd90cf03e63e... 

[191970 rows x 8 columns]

Calls to query() return a Pandas DataFrame with columns corresponding to the event’s position in the block, its transaction, and the data in the event itself. You can view these via the columns attribute on the DataFrame:

In [4]: borrows.columns
Out[4]: 
Index(
    [
        ’block_hash’, 
        ‘block_number’, 
        ‘contract_address’, 
        ‘event_arguments’,
        ‘event_name’, 
        ‘log_index’, 
        ‘transaction_hash’, 
        ‘transaction_index’
    ],
    dtype=’object’
)

Read more

Continuing the Aave Series, let’s build to the ultimate goal of liquidating unhealthy debt positions.

We’ll cover the five primary functions of the Pool contract, ordered to build layered understanding until we reach our target: supply, withdraw, borrow, repay, and liquidationCall.

The lesson is structured as a series of tests using Ape Framework that interacts with each of the five contract functions, plus some accessory calls to other contracts that will support our efforts.

This entry assumes you are familiar with the Aave concepts and Pool contract covered in Part II, how to set up Ape Framework and how to use it for executing tests against a forked chain.

Project Setup

I will be working from a new Ape project in ~/code/aave.

Create a virtual environment:

btd@dev:~/code/aave$ uv venv --python 3.13
Using CPython 3.13.8
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate

Install Ape with the set of recommended plugins:

btd@dev:~/code/aave$ uv pip install eth-ape[recommended-plugins]
...
Installed 133 packages in 420ms
...

Create the Ape project using uv run ape init and edit the configuration so that the default network used by tests is a local fork from Ethereum mainnet using Foundry:

ape-config.yaml

name: Aave Pool

default_ecosystem: ethereum

ethereum:
  default_network: mainnet-fork
  mainnet:
    default_provider: node
  mainnet_fork:
    default_provider: foundry

node:  
  ethereum:
    mainnet:
      uri: http://localhost:8545 # EDIT THIS

plugins:
  - foundry
  - etherscan

Create a connection test to confirm that ape test launches the fork from the correct chain:

tests/test_aave_pool.py

from ape.managers.networks import NetworkManager
from ape_foundry.provider import FoundryForkProvider

def test_fork_is_launched(networks: NetworkManager):
    assert isinstance(networks.provider, FoundryForkProvider)
    assert networks.provider.chain_id == 1

Execute using uv run ape test and confirm that test passes. Now we can be confident that tests will run against the fork.

supply

The supply function from Pool.sol:

function supply(
    address asset,
    uint256 amount,
    address onBehalfOf,
    uint16 referralCode
) public virtual override {
    SupplyLogic.executeSupply(
        _reserves,
        _reservesList,
        _usersConfig[onBehalfOf],
        DataTypes.ExecuteSupplyParams({
        user: _msgSender(),
        asset: asset,
        interestRateStrategyAddress: RESERVE_INTEREST_RATE_STRATEGY,
        amount: amount,
        onBehalfOf: onBehalfOf,
        referralCode: referralCode
        })
    );
}

We need to provide four inputs:

• asset — the supplied asset address

• amount — how much we will supply

• onBehalfOf — the user address to credit the supply

• referralCode — a unique referral code (set to zero, the referral program is currently inactive)

We will be using WETH in various tests, so let’s make a fixture that creates a contract instance for use in other tests:

WETH_ADDRESS = "0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2"

@pytest.fixture
def weth():
    return Contract(WETH_ADDRESS)

And we’ll need a tester account with a balance:

@pytest.fixture
def tester_account(accounts):
    return accounts[0]

Create a test that asserts that tester_account has a balance, and that wrapping at the WETH contract works as expected:

def test_wrap_ether(weth: Contract, tester_account):
    starting_ether_balance = tester_account.balance
    starting_weth_balance = weth.balanceOf(tester_account)

    deposit_amount = 100 * 10**18
    assert starting_ether_balance > deposit_amount

    weth.deposit(value=deposit_amount, sender=tester_account)
    assert weth.balanceOf(tester_account) == (
        deposit_amount + starting_weth_balance
    )
    assert tester_account.balance < starting_ether_balance

Now we can write a test that wraps 100 Ether, approves it for use by the Pool contract, and supplies it:

AAVE_V3_POOL = “0x87870Bca3F3fD6335C3F4ce8392D69350B4fA4E2”

@pytest.fixture
def aave_pool():
    return Contract(AAVE_V3_POOL)


def test_supply(aave_pool: Contract, weth: Contract, tester_account):
    deposit_amount = 100 * 10**18
    weth.deposit(value=deposit_amount, sender=tester_account)
    weth.approve(
        aave_pool.address, 
        deposit_amount, 
        sender=tester_account
    )

    starting_weth_balance = weth.balanceOf(tester_account)
    tx = aave_pool.supply(
        weth.address, 
        deposit_amount, 
        tester_account.address, 
        0, 
        sender=tester_account
    )
    assert weth.balanceOf(tester_account) == (
        starting_weth_balance - deposit_amount
    )

The test works as expected, but doesn’t tell us much. Add a simple tx.show_trace() at the end and run the test with -s to see the output:

Read more

The largest market is on Ethereum mainnet (chain ID 1), accessible via the Aave front-end by going to the Markets tab and choosing “Core” from the drop-down menu.

Many assets are enabled for this market, including the familiar Wrapped ETH, Tether USD, and USD Coin tokens.

We will use this market for the remainder of our study. Be sure you connect to an appropriate RPC and use Etherscan (or equivalent) as your block explorer.

It’s nice to see some data through the front-end, but our primary focus is on the contracts.

Concepts

Aave has several domain-specific terms and concepts. Not all of these concepts are relevant to us, but some merit a brief overview.

Liquidity Pool

Uniswap has the concept of individual liquidity pools holding a pair of assets. Each pool has a distinct contract address that holds the balance of each assets.

Aave refers to an instance of a market (and all of its related contracts) as a liquidity pool. Assets are not held by a single contract, rather they are managed by several distinct ones.

Reserve

A reserve is a generic name for a token held by an Aave liquidity pool. Individual reserve assets are approved and enabled by the protocol for use. The characteristics of each asset vary, so reserves have several parameters to control how they can be used by the protocol:

• Loan-to-Value (LTV) — a percentage specifying the maximum size of the debt that can be created against the reserve

• Liquidation Threshold — a percentage specifying the LTV where a position is considered undercollateralized and eligible for liquidation

• Borrowing Enabled — true/false

• Caps — upper limits on both the supplied and borrowed amounts

• Interest Rate Model — parameters to set the shape of the interest rate curve

aToken

An aToken is an ERC-20 that represents a claim on some Aave reserve asset. When a liquidity provider supplies an asset, an amount of some aToken is minted to them. To withdraw that asset, the corresponding aToken is burned and the asset is returned.

The aToken contract is standalone and holds its underlying asset. The asset can be withdrawn only by the Pool contract calling burn on behalf of a user.

vToken

A vToken is an ERC-20 that represents a variable rate debt position. It is non-transferable and used to track debts for by a particular user.

Interest Rate

The interest rate is a percentage of the loan amount that must be repaid by borrowers to close their debt position. There are two interest rates: one for borrowing; one for supplying. The rate for borrowing is higher than the rate for supplying, and the difference is kept as a fee to the protocol.

Each interest rate curve is upward sloping with respect to usage ratio, the fraction of all open debt positions against the collateral. The interest rate has two slopes. The higher of the two is used after the LTV for the market crosses an optimal point.

The net effect of this dual-slope strategy is that loan creation is encouraged by low borrowing rates when LTV is low, and discouraged by high borrowing rates when LTV is high.

Since interest rates are continuously variable, the total interest paid to a liquidity provider is calculated instantaneously at withdrawal time.

Liquidity providers are paid their accumulated interest through a scaling mechanism whereby their share of the reserve asset is multiplied by a ratio corresponding to the current liquidity index which is dynamically adjusted.

Utilization Rate

Each asset has a target utilization rate set by protocol governance. The protocol-wide LTV for a particular asset is rolled into this value. The transition between the two slopes on the interest rate curve occurs at this target optimal utilization rate.

This value is shown on the front-end, and can also be read from the asset’s holding contract.

Efficiency Mode (E-Mode)

For correlated assets like stablecoins or ETH derivates, Aave provides a special mode for borrowing that allows for higher LTVs.

Isolation Mode

The protocol is at maximum risk when a volatile asset backs a loan for another volatile asset.

Nevertheless, some new assets may be desirable, so Aave allows them to be used as collateral against less volatile assets only. A typical use case is allowing a new volatile asset to be supplied as collateral for loans of stablecoins.

Console Exploration

Throughout the series, console code will be provided. The underlying tool is Ape Framework, most often connected to an Ethereum mainnet RPC.

Read more

The individual mechanisms are simple:

• A depositor can supply an asset to the protocol

• A depositor can withdraw their previously supplied asset, plus accumulated interest, up to the unborrowed amount currently held by the protocol

• A depositor can also be a borrower, and can create a loan against some other assets

• Borrowers must repay the full amount of their loan, plus accumulated interest, to close their loan

The value of a deposit must exceed the loan amount at the time of generation, a practice known as “overcollaterilization”. This provides a safety margin so the protocol and its users can react appropriately to fluctuating collateral values.

Bad Debt

Since the protocol distributes assets deposited by its users, there is a risk of insolvency if loans are not repaid.

For a given loan, if the value of its collateral assets falls below the value of the loaned assets, it is unlikely that the borrower will repay it — a rational borrower would prefer to hold the loaned assets instead of returning it for their devalued deposit. Loans like this are labeled “bad debt” because they create unsatisfied obligations to two groups of depositors — one supplying the collateral asset, and one supplying the loaned asset.

Bad debt hurts both groups because a deposit asset can only be withdrawn up to the unborrowed amount — that is, only the amount of an asset held by the pool and not currently given to a borrower.

In the case of excessive bad debt, a lending protocol can experience a “bank run” event where only a fraction of users can withdraw unborrowed assets, and everyone after is unable to withdraw.

Liquidations

Since bad debt is dangerous, the protocol needs safeguards and mechanisms to reduce or remove bad debt.

Some protocols manage a separate asset reserve that can make depositors whole following a liquidity event. This is outside the scope of our study, so we won’t concern ourselves with this.

The first defense against accumulating bad debt is to incentivize third parties to liquidate (repay) unhealthy loans. As the asset value ratio of collateral vs. borrowed falls, liquidators are incentivized to repay a portion of the loan in exchange for an inflated share of the collateral.

The primary effect is reducing the outstanding obligation to the loaned asset’s depositors. But the secondary effect is increasing the outstanding obligation to the collateral asset’s depositors.

These effects seemingly antagonize the two groups — the loaned asset’s depositors are being made whole at the cost of the collateral asset’s depositors!

However, remember that every loan is overcollateralized. Thus each collateral asset across the protocol should have a net positive amount of pooled liquidity. This extra balance is what pays for the liquidation incentive, and lending protocols often implement variable interest rates to encourage users to shift their behavior towards either deposits or borrowing.

Aave

According to DefiLlama, Aave is the largest lending protocol by total volume locked (TVL).

There are three major versions of Aave, presented below with a link to the white paper, date of first deployment to Ethereum mainnet, and a snapshot of TVL:

• V1 — Jan 2020, $10.3 million

• V2 — Dec 2020, $187.8 million

• V3 — March 2022, $30.3 billion

Aave V3 manages a majority of current lending activity, so we will focus on it. V3 is deployed to several chains, so the strategies we develop here will be portable.

Moving Forward

We are here to find profitable opportunities, and the primary way to do this on Aave is to act as a liquidator.

But before we can develop a liquidation strategy we must study: how deposits are managed; how loans are created, repaid, and closed; and how loan health is tracked.

The next entry in the series will cover the most important smart contract on Aave — Pool.

You should see it hit PyPi today once I finalize some release checks and final tests. As always, install it with pip or uv.

Multiple Routes

In the previous post I mentioned the ability to provide a mapping between equivalent tokens. However as I continued scaling the path finder to work for arbitrary depth path searches, I realized it was clunky and refactored it.

I ultimately refactored the equivalent tokens map with the more general ability to specify a list of start and end tokens. This accomplishes the same goal while simplifying the control flow.

Here’s a sample function call taken from a test:

def test_generic_algo_multiple_tokens():
    depth = 2

    # UniswapV4 pools hold both native and WETH pairs, so paths to 
    # and from both tokens can be found using this single type
    pool_types: list[type] = [UniswapV4PoolTable]

    generic_paths_weth_to_weth = list(
        find_paths(
            chain_id=BASE_CHAIN_ID,
            start_tokens=[WETH_BASE_ADDRESS],
            end_tokens=[WETH_BASE_ADDRESS],
            max_depth=depth,
            pool_types=pool_types,
        )
    )

    generic_paths_weth_to_native = list(
        find_paths(
            chain_id=BASE_CHAIN_ID,
            start_tokens=[WETH_BASE_ADDRESS],
            end_tokens=[ZERO_ADDRESS],
            max_depth=depth,
            pool_types=pool_types,
        )
    )

    generic_paths_weth_to_weth_or_native = list(
        find_paths(
            chain_id=BASE_CHAIN_ID,
            start_tokens=[WETH_BASE_ADDRESS],
            end_tokens=[WETH_BASE_ADDRESS, ZERO_ADDRESS],
            max_depth=depth,
            pool_types=pool_types,
        )
    )

    assert len(
        generic_paths_weth_to_weth_or_native
    ) == len(
        generic_paths_weth_to_weth
    ) + len(
        generic_paths_weth_to_native
    )

In this test, three distinct path finding searches are performed through the set of known Uniswap V4 pools on Base mainnet:

• WETH → WETH

• WETH → Ether

• WETH → WETH | Ether (a hybrid search with multiple destinations)

At the end, the number of paths found by the hybrid search is confirmed to match the sum of the paths found by the two single-token searches.

I also included a more complex assertion after that that confirms that all of the paths match exactly, but that required some sorting and transformation, so it’s not relevant here.

You can also run a multiple start / multiple end search:

generic_paths_weth_or_native_to_weth_or_native = list(
    find_paths(
        chain_id=BASE_CHAIN_ID,
        start_tokens=[WETH_BASE_ADDRESS, ZERO_ADDRESS],
        end_tokens=[WETH_BASE_ADDRESS, ZERO_ADDRESS],
        pool_types=pool_types,
    )
)

Ultimately this searches four routes, returning results as the Cartesian product of the inputs:

• WETH → WETH

• WETH → Ether

• Native Ether → WETH

• Native Ether → Ether

Variable Depth

The default argument to find_paths will yield all paths with minimum depth equal to 2. A path’s depth is measured by the number of pools it contains.

There is an optional maximum depth argument, which defaults to None! Beware of leaving this at the default, because the search will continue “forever” if allowed.

There is a finite limit on the number of routes through a graph since an edge (pool) can only be used once, but for practical purposes you should always set the maximum depth to a value between 2-4. The number of possible routes increases exponentially as you increase depth.

# Find 2-pool & 3-pool paths
find_paths(
    chain_id=BASE_CHAIN_ID,
    start_tokens=[WETH_BASE_ADDRESS],
    end_tokens=[WETH_BASE_ADDRESS],
    min_depth=2,
    max_depth=3,
)

To perform a fixed length search, set the minimum and maximum depth equal:

# Find only 2-pool paths
find_paths(
    chain_id=BASE_CHAIN_ID,
    start_tokens=[WETH_BASE_ADDRESS],
    end_tokens=[WETH_BASE_ADDRESS],
    min_depth=2,
    max_depth=2,
)

# Find only 3-pool paths
find_paths(
    chain_id=BASE_CHAIN_ID,
    start_tokens=[WETH_BASE_ADDRESS],
    end_tokens=[WETH_BASE_ADDRESS],
    min_depth=3,
    max_depth=3,
)

As a curiosity, Uniswap V4 allows for a WETH-Ether pair, which technically satisfies a mixed asset 1-pool path. Since this pool is a special case and a bit of a nuisance (deposits and withdrawals through the Wrapped Ether contract are always 1:1), I implicitly exclude it by setting the default depth to 2.

It’s usually not what you want, but in the interest of flexibility you can set the minimum depth to 1.

Moving Forward

With all this wrapped I’m putting two related topics on the roadmap: liquidations and lending protocols, beginning with Aave.

Uniswap V4

The UniswapV4Pool class now performs a lookup from the database at construction time. If you have activated and updated either of the built-in exchanges (base_uniswap_v4 or ethereum_uniswap_v4), the class can skip many onchain calls needed to look up tokens, fee, tick spacing, etc.

Consequently many of the inputs to the construction are now optional. If you build a V4 pool that exists in the database, you only need to provide its pool manager address and pool ID. If you aren’t using the database, you can continue passing the values in as before.

Path Finding

I have also included an arbitrage path finding function find_paths(). It uses the database to discover pools and their tokens, then works through them to find pool paths in increasing depth.

This is a key unlock because it means that paths can be built during runtime, instead of needing to prebuild them with a separate script.

Here is an example of the simplest case, a two-pool cycle using some arbitrary profit token:

for arb_path in find_paths(
    chain_id=8453,
    start_token=WETH_ADDRESS,
    end_token=WETH_ADDRESS,
    max_depth=2,
):
    for pool in arb_path:
        # [create pools and build the arbitrage helper here]

The path finding function also supports limiting the pools to a subset of types that identify the database types:

for arb_path in find_paths(
    chain_id=8453,
    start_token=WETH_ADDRESS,
    end_token=WETH_ADDRESS,
    max_depth=2,
    pool_types=[
        UniswapV2PoolTable,
        UniswapV4PoolTable,
    ],
):
    for pool in arb_path:
        # [create pools and build the arbitrage helper here]

This is very useful if you have a specialized contract that is specific to V4-V2 arbs like this one I shared:

It also accepts a list of equivalent tokens, for instance Ether & WETH. This is very useful for cases where an execution contract can perform wrapping and unwrapping in between pools. This example would find all paths that take the forms WETH-X → Ether→X and Ether-X → WETH-X:

for arb_path in find_paths(
    chain_id=8453,
    start_token=WETH_ADDRESS,
    end_token=V4_NATIVE_ADDRESS,
    max_depth=2,
    equivalent_tokens=[(WETH_ADDRESS, V4_NATIVE_ADDRESS)],
):
    for pool in arb_path:
        # [create pools and build the arbitrage helper here]

In the case of two equivalent profit tokens, the ordering of the inputs is arbitrary. Reversing the start and end works as expected and yields the same pools.

Currently the only supported paths are two-pool, but I am working on pools of arbitrary length and will include that in the coming stable release.

Talkin' 'bout My Generator

The function is a generator which must be consumed.

You can either loop through it using for or next, or capture all of the paths in a container (tuple, list, deque, etc). It’s simple enough once you’ve seen it, but don’t panic when a familiar operation like len(find_paths(…)) or find_paths(…)[0] fails.

The generator approach has two key benefits:

• Path finding can start immediately — improving time-to-first-calculation

• Memory use is minimized — each path is built and given directly to the consuming function instead of being held until the path search is complete

Final Release Soon

I’m pleased with the new features in testing, and have already retired my Cryo extraction and two-pool path finding scripts. After I convert the remaining path finding scripts for 3 or more tokens, it should be much simpler to manage and consolidate all of this stuff.

This should be the final beta release ahead of 0.5.0, so please report bugs if you find them.

• Pancakeswap — V2, V3

• Sushiswap — V2, V3

• Uniswap — V2, V3, V4

These can be activated and updated like the Base mainnet exchanges featured in the previous beta releases.

I am targeting two more features before the official 0.5.0 release: pathfinding directly from the database, and integrating database lookups into the Uniswap V4 pool class.

I encourage you to try out the beta release in a virtual environment and build a minimal database of V2 pools on Base mainnet, which is fast to populate with a fairly simple set of commands:

degenbot activate base_aerodrome_v2
degenbot activate base_pancakeswap_v2
degenbot activate base_sushiswap_v2
degenbot activate base_swapbased_v2
degenbot activate base_uniswap_v2

# pause here to make sure an RPC is defined in config.toml and to update # database schema

degenbot pool update

The V3 / V4 updaters will take much longer to run, and can be done later once you’re comfortable.

SQLAlchemy

I’ve used SQLAlchemy as the object-relational mapper (ORM) to translate the information in the database to our running applications. The classes are available under the degenbot.database.models module. Any class that derives from Base defines a database table.

These classes have fairly straightforward names that suggest the information contained in each.

Examples:

• Erc20TokenTable holds information for ERC-20 tokens

  • Chain ID

  • Address

  • Decimals

  • Name

  • Symbol

• LiquidityPoolTable is a polymorphic table class that holds a subset of information common to all liquidity pools, and a link to other subclasses with exchange-specific info as needed

  • Chain ID

  • Address

  • Kind ­— a label which maps different liquidity pools associated with a particular exchange to separate tables

    • An entry with kind “uniswap_v2” identifies this pool as Uniswap V2, and each entry in UniswapV2PoolTable will refer back to this base table with a foreign key.

  • Token0 ID & Token1 ID — both map to entries in Erc20TokenTable so that token specific data can be looked up via foreign key instead of duplicated in pool tables

  • Fee

• LiquidityPositionTable holds information about Uniswap V3 liquidity positions

  • Pool ID — a foreign key mapping the position to a given pool

  • Tick

  • Liquidity (Net)

  • Liquidity (Gross)

• UniswapV4PoolTable holds information about Uniswap V4 liquidity pools. These are all managed by a single Pool Manager contract, so they all have a common address. Thus they are tracked independently of the pools defined in LiquidityPoolTable, which are uniquely identified by their address

  • Pool Hash — referred to as the Pool ID by Uniswap, this is a 32 byte hash that uniquely identifies the pool at the Pool Manager

  • Hooks — an address that identifies the various hooks enabled by the pool

  • Currency0 ID & Currency1 ID — like the ERC-20 references from above

  • Fee

  • Tick spacing

  • Kind — a label to identify the particular exchange

• ManagedPoolLiquidityPositionTable holds liquidity positions for managed pools. Essentially identical to LiquidityPositionTable except that it points to managed pools in ManagedLiquidityPoolTable

• PoolManagerTable which holds information about individual Pool Manager contracts

  • Chain ID

  • Address

  • Kind — a label to identify the particular exchange

Relationships

A clear win from using an ORM is that links between tables can be expressed as a relationship and accessed like an attribute, instead of having to write complex JOIN queries.

Read more

I spent last week working on the feature. The integration is largely complete, so after some final QA I will publish a 0.5 beta version to PyPi.

If everything goes as expected, I should have it published in the next day or two. I will edit this post with a link when it’s ready.

The 0.5 beta release is available. By default, pip will ignore alpha, beta, and release candidate versions — to install, run pip install degenbot==0.5.0b2.

Command Line

The 0.5 releases of degenbot added command line integration with a set of useful commands.

Read more

If you’ve done concurrent programming using coroutines, threads, or multiple processes, you know how difficult it can be. You must balance protection from data races and corruption against the risk of deadlock and slowdown.

The Rust promise is that the ownership model mitigates the impact of both: the former because simultaneous read/write issues are prohibited; the latter because locks become mostly unnecessary for common concurrency patterns.

We will study how to spawn threads in Rust, the Arc and Mutex synchronization primitives, the Sync and Send traits, parallel iteration using Rayon, and async functions using Tokio.

Threads

Rust offers threading support through the standard library. This example from the book can be copied directly into Playground and executed:

use std::thread;
use std::time::Duration;

fn main() {
    thread::spawn(|| {
        for i in 1..10 {
            println!(”hi number {i} from the spawned thread!”);
            thread::sleep(Duration::from_millis(1));
        }
    });

    for i in 1..5 {
        println!(”hi number {i} from the main thread!”);
        thread::sleep(Duration::from_millis(1));
    }
}

Closures

The only thing new here is the strange argument to spawn, which we will review briefly.

Rust allows for anonymous functions that can capture the values of variables outside their scope at runtime. Python’s lambda is similar.

In the example above, the closure takes this form:

|| {
    for i in 1..10 {
        println!(”hi number {i} from the spawned thread!”);
        thread::sleep(Duration::from_millis(1));
    }
}

The || is where the inputs to the closure are placed. Since this closure doesn’t take any input, it is empty. The actions of the closure are defined just like any other function, including the curly braces.

If you wanted to be more explicit, you can assign the closure to a variable:

let spawned_hi = || {
    for i in 1..10 {
        println!(”hi number {i} from the spawned thread!”);
        thread::sleep(Duration::from_millis(1));
    }
};

Which you can call like a standard function::

spawned_hi();

Or pass it to the thread:

thread::spawn(spawned_hi);

Read more

Python uses exceptions to signal that something has gone wrong. A good Python app will anticipate what exceptions might be raised by various function or method calls and define the behavior when those exceptions occur.

Rust does not have exceptions. Instead it follows the “error as value” approach, where instead of raising an exception and breaking the normal control flow of the program, it will return an error instead of a value. The responsibility for handling both options falls to the calling code. Rust operations that can fail typically return a Result type which can hold one of two things: the value or an error.

Part II included Rust code with expressions ending with .unwrap() but I did not elaborate on why. Let’s get into it!

Result is a type of Rust struct called an enum, which organizes and labels related values. It is defined in std::result:

pub enum Result<T, E> {
    Ok(T),
    Err(E),
}

Angled brackets make another appearance here, which signifies that Result is generic over types T and E. The enum holds two variants (Ok, and Err) which are named variants that wrap a value of a generic type.

Whenever you call a function that might fail, you should expect to receive a Result. The Result will either be an Ok wrapping the good value, or an Err wrapping the error value.

If you control the function and the input and know it will not fail, you might choose to skip the error checking and just access the value directly. Calling unwrap() on an Ok will extract the value from that wrapper. This is what I was doing in Part II.

Put this into Rust Playground:

pub fn get_69() -> Result<u8, String> {
    Ok(69)
}

fn main() {
    let known_good_value = get_69().unwrap();
    println!("{:?}", known_good_value);
}

And run it to see:

69

Now let’s change the function to give an error instead:

pub fn get_69() -> Result<u8, String> {
    Err(String::from("Can't get 69, not nice"))
}

Running this blows up with a panic message:

thread 'main' panicked at src/main.rs:6:37:
called `Result::unwrap()` on an `Err` value: "Can't get 69, not nice"

The unwrap method quickly peels off the wrapper holding a successful value. However if you unwrap an Err, the runtime will panic, triggering an immediate cleanup and shutdown.

Python is like this, too. If a program raises an exception which is not caught, it will eventually reach the entry point of the program and the interpreter will clean up and shut down.

Matching

Dealing with Rust errors is similar to using Python’s structural pattern matching (see PEP-064), whereby you can match directly on the value inside the wrapping variant:

pub fn get_69() -> Result<u8, String> {
    Ok(69)
    // Err(String::from("Can't get 69, not nice"))
}

fn main() {
    let unknown_value = get_69();
    match unknown_value {
        Ok(good_value) => println!("Got a value: {}", good_value),
        Err(message) => println!("Got an error: {}", message),
    }
}

Running this, we get:

Got a value: 69

And after changing the function to return the Err instead:

Got an error: Can't get 69, not nice

The match keyword allows us to set control flow depending on the wrapper variant, and to assign the wrapped value to a variable that we could use. This allows us to offload the “did the call succeed?” flow to the match block, and concentrate on the “then do this…” logic in each arm that follows the match.

The ? Operator

Sometimes an error is unrecoverable, so there’s no point handling it. For situations where defining a complete match block would be overkill, we can use the ? operator which will extract the wrapped values for either the Ok or the Err variant without using a full match block.

Try it out:

pub fn get_69() -> Result<u8, String> {
    // Ok(69)
    Err(String::from("Can't get 69, not nice"))
}

fn main() {
    let only_good_value = get_69()?;
}

This fails because the ? operator can cause main to return a String instead of nothing, which is the default return type. We can adjust the function signature to specify the two possibilities expressed as a Result:

pub fn get_69() -> Result<u8, String> {
    // Ok(69)
    Err(String::from("Can't get 69, not nice"))
}

fn main() -> Result<(), String> {
    get_69()?;
    Ok(())
}

The Ok variant runs successfully with no output, and the Err variant results in the message:

Error: "Can't get 69, not nice"

Notably, using the ? operator on an Err variant does not trigger a panic like unwrap would — it just returns the wrapped error.

Type Conversions

The built-in Rust types are simple to convert. Like types can be converted in-line using cast in an assignment expression:

fn main() {
    let i: u64 = 420;
    let j: u32 = i as u32;
    
    println!(”{i}”);
    println!(”{j}”);
}

Output:

420
420

However, please be careful! Casting between different bit length types can result in wrapping if the source value exceeds the range of the destination type:

fn main() {
    let i: u64 = 420;
    let j: i8 = i as i8;
    
    println!(”{i}”);
    println!(”{j}”);
}

Here, the i8 (8 bit signed integer) wraps from 127 to -128 when it reaches the upper limit of its range:

420
-92

Complex Conversions

As you move beyond the built-ins, you may want to perform more complex type conversions.

Read more