Degenbot PyPI Release

It's Official!

I’ve published the initial 0.0.1 release of degenbot to PyPI. Pushing changes to Github has worked fine until now, but it is time to move to a more robust release model.

In order to improve the structure of the file repository, I have converted the file layout from a “flat” structure to a “src” structure. I don’t want anyone getting confused next time they do a git pull and notice that the code is structured differently. You can read more about src layout vs flat layout if you are curious.

A side effect of this new structure is that the local repo folder cannot be simply imported into scripts as before. The new method offers many improvements, so I think you’ll find it worthwhile and quite to repeat once you’ve seen it.

If you have a working setup and don’t want to mess around, feel free to keep it going. You can transition on your own time, or set up a separate bot in parallel to test the waters before switching.

Pulling From Github

The current process for using degenbot is to git clone the Github repo into some directory where your bot script is stored, and import degenbot at the top. Simple!

Once the flat layout has been converted to src layout, the “pull & import” method will no longer work. But don’t fret, the new method is still quite simple and everything else will work.

Editable Installation

The Package Installer for Python (aka pip) is typically used to install packages from PyPI. It also allows installation of packages from the local filesystem. A particularly useful option is called editable install. An editable install allows a user to install a package to their Python environment from a local directory instead of PyPI. It also provides a link back to the local directory, so that the user can make modifications to the package from the original directory without reinstalling through pip each time.

So now instead of cloning or linking the degenbot code into your bot’s working directory, you can perform a one-time editable installation and then continue working as normal.

To illustrate, I’ll demonstrate an editable installation from a fresh copy of the new repo structure, which I store in ~/code/degenbot:

[btd@main bots]$ python3 -m venv .venv
sour[btd@main bots]$ source .venv/bin/activate
(.venv) [btd@main bots]$ pip install -e ~/code/degenbot/
Obtaining file:///home/btd/code/degenbot
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Installing backend dependencies ... done
  Preparing editable metadata (pyproject.toml) ... done
Collecting eth-abi<5,>=4.2.0
  Downloading eth_abi-4.2.1-py3-none-any.whl (28 kB)
Collecting eth-typing<4,>=3.4.0
  Downloading eth_typing-3.5.1-py3-none-any.whl (14 kB)

[...]


Collecting urllib3<3,>=1.21.1
  Downloading urllib3-2.0.7-py3-none-any.whl (124 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 124.2/124.2 kB 54.9 MB/s eta 0:00:00
Collecting certifi>=2017.4.17
  Downloading certifi-2023.7.22-py3-none-any.whl (158 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 158.3/158.3 kB 82.0 MB/s eta 0:00:00
Installing collected packages: lru-dict, bitarray, websockets, urllib3, ujson, typing-extensions, toolz, rpds-py, regex, pyunormalize, pycryptodome, protobuf, pluggy, packaging, numpy, multidict, iniconfig, idna, hexbytes, frozenlist, eth-hash, charset-normalizer, certifi, attrs, async-timeout, yarl, scipy, requests, referencing, pytest, parsimonious, eth-typing, cytoolz, aiosignal, jsonschema-specifications, eth-utils, aiohttp, rlp, jsonschema, eth-keys, eth-abi, eth-rlp, eth-keyfile, eth-account, web3, degenbot
  DEPRECATION: pyunormalize is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
  Running setup.py install for pyunormalize ... done
  DEPRECATION: parsimonious is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
  Running setup.py install for parsimonious ... done
Successfully installed aiohttp-3.8.6 aiosignal-1.3.1 async-timeout-4.0.3 attrs-23.1.0 bitarray-2.8.2 certifi-2023.7.22 charset-normalizer-3.3.1 cytoolz-0.12.2 degenbot-0.0.1 eth-abi-4.2.1 eth-account-0.9.0 eth-hash-0.5.2 eth-keyfile-0.6.1 eth-keys-0.4.0 eth-rlp-0.3.0 eth-typing-3.5.1 eth-utils-2.3.0 frozenlist-1.4.0 hexbytes-0.3.1 idna-3.4 iniconfig-2.0.0 jsonschema-4.19.1 jsonschema-specifications-2023.7.1 lru-dict-1.2.0 multidict-6.0.4 numpy-1.26.1 packaging-23.2 parsimonious-0.9.0 pluggy-1.3.0 protobuf-4.24.4 pycryptodome-3.19.0 pytest-7.4.3 pyunormalize-15.0.0 referencing-0.30.2 regex-2023.10.3 requests-2.31.0 rlp-3.0.0 rpds-py-0.10.6 scipy-1.11.3 toolz-0.12.0 typing-extensions-4.8.0 ujson-5.8.0 urllib3-2.0.7 web3-6.11.1 websockets-12.0 yarl-1.9.2

Observe that the working directory is otherwise empty, except for the virtual environment:

(.venv) [btd@main bots]$ ls -al
total 0
drwxr-xr-x.  3 btd  btd   60 Oct 27 20:45 .
drwxrwxrwt. 34 root root 800 Oct 27 20:46 ..
drwxr-xr-x.  5 btd  btd  140 Oct 27 20:45 .venv

Now launch a Python shell and confirm that importing degenbot works as normal:

(.venv) [btd@main bots]$ python3
Python 3.10.13 (main, Sep  2 2023, 00:54:40) [GCC 13.2.1 20230728 (Red Hat 13.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import degenbot
>>>

The editable installation process can be repeated for any other virtual environments you have, and any changes in that local directory structure will be automatically reflected into any of those environments.

So if you’ve been copying or symlinking directories around to multiple places (guilty as charged), you can quit that and maintain the code from a single location.

PyPI Releases

If you prefer to use more stable versions of the degenbot classes, you can install it from PyPI the normal way (pip install degenbot) instead of doing a git clone of the latest Github code.

The initial version published to PyPI is 0.0.1, which corresponds to the latest Github code in the main branch as of October 27, 2023.

I will use the ZeroVer version numbering scheme because they have the much funnier website compared to SemVer.

I will use the initial set of 0.0.X releases to add project metadata, refine the directory structure, add an appropriate README, etc. Basically housekeeping stuff.

I’ll probably release 0.1.0 along with the next project, which is a backrun bot that watches the MEVBlocker searcher relay.

Historical Versions

A key benefit of this approach is that I can now publish examples and projects against a fixed version of the code. A common frustration of readers working through the archive is that old examples may use deprecated methods, rely on names or paths that have changed, or rely on dependencies with breaking changes in newer releases.

So if you’re reading this post a year from now, you will find that projects from here will feature a degenbot version tag. You can pass this tag to pip to install a fixed version of the code so your environment matches mine.

Dependencies Included

Another huge benefit of moving to PyPI is that dependencies can be managed automatically by pip during installation. You won’t have to deal with hand-installing random dependencies anymore, just pip install degenbot and you will be ready to go.