Giter VIP home page Giter VIP logo

market-maker-keeper's Introduction

market-maker-keeper

Build Status codecov

The DAI Stablecoin System incentivizes external agents, called keepers, to automate certain operations around the Ethereum blockchain.

Market Maker Keeper Setup Guide

Guide Outline

  1. Introduction
  2. Prerequisites
  3. Installation
  4. Testing
  5. Bands and Bands Configuration
    • Example
  6. Order Rate Limitation
  7. GasPrice configuration
  8. Data Templating Language
  9. Price Feed Configuration
    • Example
  10. Running Keepers
    • Example (OasisDEX)
  11. Known limitations
  12. Install and run using Docker and Docker Compose
  13. Token Configuration
  14. Support Information
  15. License

1. Introduction

A big part of the DAI Stablecoin System (DSS) is the incentivization of external agents, called Keepers (which can be human but are typically automated bots). In DSS, there are two primary forms: Market Maker Keepers and Auction Keepers. Market Maker Keepers facilitate market making on supported exchanges, and thereby ensure DAI markets are liquid with minimal slippage. Auction Keepers enable bidding on auctions that secure the DSS. This repository is focused on Market Maker Keepers. Auction Keepers are available here.

Market Maker Keepers work by creating a series of orders in so-called bands (defined later), which are configured with a JSON file containing parameters like spreads, maximum engagement, etc. These bands work in coordination with methods to check your available balance, as well as retrieving price from external price feeds, to ensure that liquiditiy is being placed at the correct amounts, and correct prices. Ultimately, the keepers trade an exchange pair motivated by the expected long-term convergence towards the indicated Target Price.

This guide is dedicated to showing you how to create your very own Market Maker Keeper Bot as well as educate the community and help both users and developers understand the value of this incredible software. We are proud to say that all of the code needed to get a Market Maker Keeper bot up and running is open-sourced.

List of current exchanges that Market Maker Keeper Bots can be built for

  • Binance US (binance-us-market-maker-keeper)
  • Bitso (bitso-market-maker-keeper)
  • Bittrex (bittrex-market-maker-keeper)
  • Coinbase (coinbase-market-maker-keeper)
  • Coinone (coinone-market-maker-keeper)
  • DDEX (ddex-market-maker-keeper)
  • DyDx (dydx-market-maker-keeper)
  • ErisX (erisx-market-maker-keeper)
  • EtherDelta (etherdelta-market-maker-keeper)
  • Ethfinex (ethfinex-market-maker-keeper)
  • Etoro (etoro-market-maker-keeper)
  • GoPax (gopax-market-maker-keeper)
  • Kraken (kraken-market-maker-keeper)
  • Korbit (korbit-market-maker-keeper)
  • Kucoin (kucoin-market-maker-keeper)
  • Liquid (liquid-market-maker-keeper)
  • OasisDEX (oasis-market-maker-keeper)
  • OKEX (okex-market-maker-keeper)
  • Okcoin (okcoin-market-maker-keeper)
  • 0x (0x-market-maker-keeper)
  • Paradex (paradex-market-maker-keeper)
  • RadarRelay and ERCdEX (0x-market-maker-keeper)
  • TheOcean (theocean-market-maker-keeper)
  • UniswapV2 (uniswapv2-market-maker-keeper)

2. Prerequisite

  • Git
  • Python v3.6.6
  • virtualenv
    • This project requires virtualenv to be installed if you want to use Maker's python tools. This helps with making sure that you are running the right version of python and checks that all of the pip packages that are installed in the install.sh are in the right place and have the right versions.
  • X-code (for Macs)

3. Getting Started (Installation)

  1. Clone the market-maker-keeper repository and switch into its directory:
git clone https://github.com/makerdao/market-maker-keeper.git
cd market-maker-keeper
  1. Initializing the git submodules that will bring in both the pymaker and the pyexchange library.
git submodule update --init --recursive
  1. Check to make sure you have the correct version (Python 3.6.6) of Python by running:
python3 -V
  1. To set up the virtual environment and install requirements, run the following script:
./install.sh
  1. Source the new locally created virtual environment
source _virtualenv/bin/activate

Potential errors that could arise:

  • Needing to upgrade to pip version 19.2.2

    • Run: pip install --upgrade pip to fix.
  • Installing jsonnet (if running macOS Mojave)

    To fix, run the following:

  • xcode-select --install

  • open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

  • pip3 install jsonnet==0.9.5

  • Lastly, re-run: pip3 install $(cat requirements.txt $(find lib -name requirements.txt | sort) | sort | uniq | sed 's/ *== */==/g')

Other Potential Installation Issues:

Read the following document for other known Ubuntu and macOS issues (here).

4. Testing

There is quite a lot of value in running all the unit tests to make sure market-maker keeper has been installed properly. After the repository has been cloned and the installation has been completed, you can run unit tests by executing the following commands.

Firstly, the following command will install the libraries required to run unit tests:

pip3 install -r requirements-dev.txt

To run the unit tests (py.test, etc..), use the following script:

./test.sh

Example output:

    ===================================== test session starts =====================================
    platform darwin -- Python 3.6.6, pytest-3.3.0, py-1.8.0, pluggy-0.6.0
    rootdir: /Users/charlesst.louis/market-maker-keeper, inifile:
    plugins: timeout-1.2.1, mock-1.6.3, cov-2.5.1, asyncio-0.8.0
    collected 97 items

    tests/test_airswap_market_maker_keeper.py ................                              [ 16%]
    tests/test_band.py ......                                                               [ 22%]
    tests/test_etherdelta_market_maker_keeper.py ..........................                 [ 49%]
    tests/test_feed.py .                                                                    [ 50%]
    tests/test_limit.py .......                                                             [ 57%]
    tests/test_oasis_market_maker_cancel.py ...                                             [ 60%]
    tests/test_oasis_market_maker_keeper.py ...................                             [ 80%]
    tests/test_price_feed.py ............                                                   [ 92%]
    tests/test_reloadable_config.py .......                                                 [100%]

    ---------- coverage: platform darwin, python 3.6.6-final-0 -----------
    Name                                                    Stmts   Miss  Cover
    ---------------------------------------------------------------------------
    market_maker_keeper/__init__.py                             0      0   100%
    market_maker_keeper/airswap_market_maker_keeper.py        252    142    44%
    market_maker_keeper/band.py                               260     37    86%
    market_maker_keeper/bibox_market_maker_keeper.py           93     93     0%
    market_maker_keeper/bitinka_market_maker_keeper.py         96     96     0%
    market_maker_keeper/bittrex_market_maker_keeper.py         96     96     0%
    market_maker_keeper/coinbase_market_maker_keeper.py       103    103     0%
    market_maker_keeper/coinbene_market_maker_keeper.py        95     95     0%
    market_maker_keeper/control_feed.py                         7      5    29%
    market_maker_keeper/ddex_market_maker_keeper.py           126    126     0%
    market_maker_keeper/ercdex_market_maker_keeper.py          12     12     0%
    market_maker_keeper/etherdelta_market_maker_keeper.py     193    142    26%
    market_maker_keeper/ethfinex_market_maker_keeper.py        94     94     0%
    market_maker_keeper/feed.py                                84     46    45%
    market_maker_keeper/gas.py                                 20     10    50%
    market_maker_keeper/gateio_market_maker_keeper.py         106    106     0%
    market_maker_keeper/gopax_market_maker_keeper.py           99     99     0%
    market_maker_keeper/hitbtc_market_maker_keeper.py          98     98     0%
    market_maker_keeper/idex_market_maker_keeper.py           193    193     0%
    market_maker_keeper/imtoken_pricing_server.py              51     51     0%
    market_maker_keeper/imtoken_utils.py                       97     97     0%
    market_maker_keeper/kucoin_market_maker_keeper.py         108    108     0%
    market_maker_keeper/limit.py                               46      0   100%
    market_maker_keeper/liquid_market_maker_keeper.py          97     97     0%
    market_maker_keeper/mpx_market_maker_keeper.py            137    137     0%
    market_maker_keeper/oasis_market_maker_cancel.py           38     22    42%
    market_maker_keeper/oasis_market_maker_keeper.py          133     94    29%
    market_maker_keeper/okex_market_maker_keeper.py            92     92     0%
    market_maker_keeper/order_book.py                         219    188    14%
    market_maker_keeper/order_history_reporter.py              38     26    32%
    market_maker_keeper/paradex_market_maker_keeper.py        131    131     0%
    market_maker_keeper/price_feed.py                         187     86    54%
    market_maker_keeper/reloadable_config.py                   67      3    96%
    market_maker_keeper/setzer.py                              24     17    29%
    market_maker_keeper/spread_feed.py                          7      5    29%
    market_maker_keeper/tethfinex_market_maker_keeper.py      149    149     0%
    market_maker_keeper/theocean_market_maker_keeper.py       129    129     0%
    market_maker_keeper/util.py                                 8      4    50%
    market_maker_keeper/zrx_market_maker_keeper.py            177    177     0%
    market_maker_keeper/zrxv2_market_maker_keeper.py           26     26     0%
    ---------------------------------------------------------------------------
    TOTAL                                                    3988   3232    19%


    ================================== 97 passed in 4.04 seconds ==================================

5. Understanding Bands Configuration

The Bands configuration file is directly related to how your Market Maker Keeper will work. As mentioned in the introduction, these Keepers continuously monitor and adjust their positions in the order book, maintaining open buy and sell orders in multiple bands at the same time. For each buy and sell band, the Keepers aim to have open orders for at least the minAmount. In both cases, they will ensure that the price of open orders stays within the <minMargin, maxMargin> range from the current price. When running, Keepers place orders for the average amounts (avgAmount) in each band by using use avgMargin to calculate the order price.

As long as the price of orders stays within the set band(s) (i.e. it's in between the <minMargin,maxMargin> range from the current price), the Keepers are kept open/running. If some orders leave the band, they either enter another adjacent band or fall outside all bands. In the case of the latter, they would be immediately canceled. In the case of the former, Keepers can keep these orders open as long as their amount is within the <minAmount,maxAmount> ranges for the band they just entered. If it is above the maximum, some of the open orders will get canceled and potentially a new one will be created to bring the total amount back within the range. If it is below the minimum, a new order gets created for the remaining amount so that the total amount of orders in this band is equal to avgAmount. The same process will happen if the total amount of open orders in a band falls below the minAmount as a result of other market participants taking these orders. In this case, a new order gets created for the remaining amount so the total amount of orders in this band is equal to avgAmount. There are some Keepers (which operate on decentralized exchanges) that will constantly use gas to cancel orders (ex: OasisDEX, EtherDelta and 0x) and create new ones (OasisDEX) as the price changes. Gas usage can be limited by setting the margin and amount ranges wide enough as well as by making sure that the bands are always adjacent to each other/set to the same value.

File format

The bands configuration file consists of two main sections:

  1. buyBands
  2. sellBands

Note: Each section is an array containing one object (band).

The minMargin and maxMargin fields in each band object represent the margin (spread) range of that band. These ranges may not overlap for bands of the same type (buy or sell), and should be adjacent to each other for better Keeper performance (where fewer orders will get canceled if the bands are adjacent to each other). The avgMargin represents the margin (spread) of newly created orders within a band.

Glossary

  1. minAmount - the minimum amount for keeper engagement for a band.
  2. avgAmount - the target amount for keeper engagement for a band.
  3. maxAmount - the maximum amount for keeper engagement for a band.
  4. dustCutoff - a field for the minimum amount of every single order created in each individual band (expressed in buy tokens for buy bands and in sell tokens for sell bands).
    • By setting the dustCutoff to a non-zero value prevents Keepers from creating a lot of very tiny orders, which can cost a lot of gas. For example, in the case of OasisDEX, it can result in a very small order getting rejected by other exchanges.

Setting up your own Bands Configuration File:

1. Creating your bands.json file

To start, take the sample configuration file below and copy-paste it to a .json file within the root directory of your market-maker-keeper folder. For ease of use, we sugges to name it bands.json. This bands file will get configured as a command-line argument when we start up the Market Maker Keeper.

Sample bands.json file containing two Bands

This example shows bands for the ETH-DAI pair, where ETH represents the base currency and DAI as the quote currency:

{
    "_buyToken": "DAI",
    "buyBands": [
        {
            "minMargin": 0.005,
            "avgMargin": 0.01,
            "maxMargin": 0.02,
            "minAmount": 20.0,
            "avgAmount": 30.0,
            "maxAmount": 40.0,
            "dustCutoff": 0.0
        },
        {
            "minMargin": 0.02,
            "avgMargin": 0.025,
            "maxMargin": 0.03,
            "minAmount": 40.0,
            "avgAmount": 60.0,
            "maxAmount": 80.0,
            "dustCutoff": 0.0
        }
    ],
    "buyLimits": [],

    "_sellToken": "ETH",
    "sellBands": [
        {
            "minMargin": 0.005,
            "avgMargin": 0.01,
            "maxMargin": 0.02,
            "minAmount": 2.5,
            "avgAmount": 5.0,
            "maxAmount": 7.5,
            "dustCutoff": 0.0
        },
        {
            "minMargin": 0.02,
            "avgMargin": 0.025,
            "maxMargin": 0.05,
            "minAmount": 4.0,
            "avgAmount": 6.0,
            "maxAmount": 8.0,
            "dustCutoff": 0.0
        }
    ],
    "sellLimits": []
}

This bands.json file should be adequate enough for you to copy and paste it and run it as is. Of course, you are free to configure it however you would like.

Note: Since this example will be focused on getting a Market Maker Keeper set up on Kovan, you need to make certain that you have enough Kovan ETH (K-Eth) to get your Keeper up and running. To receive Kovan ETH, join the following Gitter Channel: https://gitter.im/kovan-testnet/faucet and post your ETH address from your MetaMask account to the main chat. The Kovan faucet will then populate your wallet with the test funds. This process could take a couple of minutes or a couple of hours as it is done manually by the Gitter channel’s administrator.

2. Setting Amounts

You will need to set this up if you are going to be trading small amounts to start. This will need to be set up such that the amounts are sufficiently meaningful on the exchange you want to work with (based on the rules of the exchanges that you want to work. For example, their specificly set dust limits).

As mentioned above, there is one parameter in the configuration file (dustCutoff) that will be used to determine the minAmount of trade quantity. The dustCutoff will need to be set higher than/or equal to the minAmount trade quantity of the specific exchange. This is to make sure you don't create trades that are less than an exchanges' minimum trade requirements. Note that in your configuration file, you can also lower the required quantities to make it easier on yourself. Reducing the K-ETH amounts will be helpful in reducing wait times as you likely won't want to wait long periods in order to get enough K-ETH to run your Keeper).

Bands Example

Here, we will be going over some example interactions using the bands.json file described above. These examples assume that it is denominated in DAI and that the price of 10 DAI is 1 ETH.

Using Band 1

  • If we look at the first buy band, the initial buy order will be 30 DAI (avgAmount) with the price of -> price - (price * avgMargin) -> 0.1 - (0.1 * 0.01) -> 0.099 ETH per Dai.
  • If the buy order listed above (30 DAI @ 0.099 ETH) gets partially filled (15 DAI are purchased), then we will have (15 DAI remaining in the order). However, this amount is below the band's minAmount (20 DAI), therefore, another whole order of 15 DAI will be placed on the exchange at the same price of 0.099 ETH.
  • In addition to buy orders, when the Market Maker Keeper starts up, two sell orders will also be placed.

Using Band 2

  • For ease of explanation, let's assume we are selling ETH that is priced at 100.00 DAI (5 ETH @ 101 DAI and 6 ETH @ 102.5 DAI).
  • Now imagine a situation where the price of ETH suddenly drops to 97.50 DAI, pushing the bands downward. In this scenario, the second band will then start working and will become responsible for both of the sell orders, as they fit in between the second band's minMargin and maxMargin amounts.

The Market Maker Keeper will now reset it's bands by performing the following actions:

  1. Creating an order in Band 1 (5 ETH @ 98.475 DAI) using avgMargin and avgAmount.
  2. Cancelling the second order (5 ETH @ 102.5 DAI) (which is now in Band 2) becuase maxMargin has been breached (when price + (price * maxMargin) = orderPrice -> 97.5 + (97.5 * 0.05) -> 102.375 > 102.5).
  3. Keep the first order (5 ETH @ 101 DAI), which is now in Band 2 because it is within minMargin and maxMargin of Band 2.
  4. Creating an order in Band 2 (1 ETH @ 99.937 DAI) using avgMargin and avgAmount.

This results in a total of 3 placed orders:

  • Band 1 -> (5 ETH @ 98.475 DAI)
  • Band 2 -> (5 ETH @ 101 DAI)
  • Band 2 → ((1 ETH @ 99.837 DAI)

6. Order Rate Limitation

Next, we want to add the Rate Limitation to the configurations file. This will make sure that we don't constantly churn out old orders as well as help manage our gas consumption. We do this because we want the period and the amount to be set to a low amount when we start out. This is done because we don't want new users' Market Maker Keeper bots to be frantically trading all of the time. The goal here is that we want to set up our initial states such that it is only placing an order every 5 min or so (or whatever time amount you decide on).

There are two (optional) limit sections (buyLimits and sendLimits) that can be used for limiting the maximum rate of orders created by Market Maker Keepers. They both use the same format as displayed below.

Example of order rate limits:

    "buyLimits": [ { "period": "1h", "amount": 50.0 }, { "period": "1d", "amount": 200.0 } ]
  • The period defines the amount of time that the limit should be applied over.
  • The amount is the maximum amount of orders that should be placed during the set period amount.

In the example above, the periods are set to 1-hour and 1-day and the amounts are set to 50.0 orders and 200.0 orders. This means that over the course of 1-hour, only 50.0 orders can be placed and over the course of 1-day, only 200.0 orders can be placed. The amounts will be expressed in terms of either the buy or the sell token (this will depend on the section). Note that the above code snippet imposes a limit of 50.0 buy token within each 60-minute window. Additionally, a maximum of 200.0 buy tokens within each 24-hour window.

Note: The supported time units are s, m, h, d, and w.

7. Gas price

Providers

Four different providers can be configured in the startup script:

  • To use Etherscan, pass --ethgasstation-api-key followed by your API key.
  • Pass --etherchain-gas-price to use EtherChain.
  • Pass --poanetwork-gas-price to use POA Network.
  • To specify your own starting price, pass --fixed-gas-price followed by your desired price in Gwei.

Tuning parameters

By default, speedup transactions will be submitted after 42 seconds (roughly three blocks). This can be adjusted with --gas-replace-after. Take care not to set this below the time it takes for the node to provide block updates. 30 seconds is probably an acceptable minimum value. The provider's price can be scaled by a coefficient controlled by --gas-initial-multiplier. Aggressiveness of speedup replacements can be tuned with --gas-reactive-multiplier.
The maximum amount of gas for a single transaction, in Gwei, is limited by --gas-maximum.

8. Data Templating Language

The Jsonnet data templating language can be used for the configuration file.

In the case of the data templating language, think of this like a pre-processing language for parsing the file. The whole purpose of the jsonnet is to set up a configuration file such that you can have it increment based on a price. Therefore, in addition to the price feed, you can also base percentages away from the market price. As you can see below, there is a hardcoded amount/price as well as the amounts below it which are dependent on the price.

{
  "_price": 10,

  "_buyToken": "DAI",
  "buyBands": [
    {
      "minMargin": 0.020,
      "avgMargin": 0.050,
      "maxMargin": 0.075,
      "minAmount": 0.05 * $._price,
      "avgAmount": 0.25 * $._price,
      "maxAmount": 0.35 * $._price,
      "dustCutoff": 0.1
    }
  ],

  "_sellToken": "ETH",
  "sellBands": []
}

Notes:

  • If you are working with a token that's price does not fluctuate wildly, you do not need to incorporate relative qualities for your amounts. This is typically used for users that want their Market Maker Keepers open for months at a time and don't want to worry about having to change any of their configurations.
  • Another thing to note about these files is that the Market Maker Keeper reloads the configuration files automatically when it detects a change in them. This makes it easier as you don't have to constantly restart your Keeper bot when you change your band configurations. In short, this works by periodically taking a hash of the configuration file and comparing that hash with the current version. This means that when it sees a change in the hash of the file, it will reload the configuration file and cancel orders as necessary to maintain the newly updated bands.

9. Price Feed Configuration

The price feed is one of the most important determining factors of success in a Market Maker Keeper. If you have the bands set up the way you want, the price feed will help make sure your bands are set at meaningful levels relative to the inside market. If you have wide bands and your strategy is to add liquidity to handle market imbalances, then the price feed is not as important. However, as you tighten up the spreads, the price feed is a crucial component to ensure that you are going to profit in the market.

Below, we list some of the existing public feeds. You can also use web sockets if you have your own price feed that you want to use. In short, it works by each Market Maker Keeper taking in a --price-feed command-line argument which then determines the price used for market-making.

As of today, these are the possible values of this argument ( existing public feeds) that we list:

  • fixed:<amount> - uses a fixed price (fixed:200in this example). See below for a more in-depth example. Note that when you are on mainnet, you typically won't use a fixed amount but it is an ideal example for this walkthrough as there aren't price feeds for Kovan.
  • eth_dai - uses the price from the GDAX (Coinbase) WebSocket ETH/USD price feed.
  • eth_dai-setzer - uses the average of Kraken and Gemini ETH/USD prices.
  • eth_dai-tub - uses the price feed from Tub (only works for keepers being able to access an Ethereum node).
  • eth_dai-pair - uses the price from the GDAX (Coinbase) WebSocket ETH/DAI price feed.
  • eth_dai-pair-midpoint - uses the midpoint orderbook price from the GDAX (Coinbase) WebSocket ETH/DAI pair.
  • dai_eth - inverse of the eth_dai price feed.
  • dai_eth-setzer - inverse of the eth_dai-setzer price feed.
  • dai_eth-tub - inverse of the eth_dai-tub price feed.
  • btc_dai - uses the price from the GDAX (Coinbase) WebSocket BTC/USD price feed.
  • dai_btc - inverse of the btc_dai price feed.
  • ws://... or wss://... - uses a price feed advertised over a WebSocket connection (custom protocol).

Note: The --price-feed command line argument can also contain a comma-separated list of several different price feeds. In this case, if one of them becomes unavailable, the next one in the list will be used instead. All listed price feeds will be constantly running in the background where the second one listed and following ones ready to take over when the first one (or prior one) becomes unavailable. In the example below (in the Running Keepers section), you can see an example of how to use a fixed price amount.

10. Running Market Maker Keepers

Each Market Maker Keeper is a command-line tool which takes in generic command-line arguments (such as --config, --price-feed, --price-feed-expiry, --debug, etc.) as well as some arguments which are specific to that particular Keeper. For example, Ethereum node parameters, addresses, exchange API keys, etc. All accepted command-line arguments are listed in the example section below. They can also be discovered by trying to start a Market Maker Keeper with the --help argument.

Example (Oasis Market Maker Keeper)

In order to run oasis-market-maker-keeper, you will need to go through the following process:

  1. Firstly, you would deploy an Ethereum node (we recommend Parity).
  2. Generate an account in it.
  3. Permanently unlock that account.
  4. Transfer some tokens to it.
  5. Run the Market Maker Keeper (as seen below).

The below file should be copy and pasted into a new file within the root directory of the repository (market-maker-keeper). This should be placed within the same folder where you put the bands.json file.

#!/bin/bash

    bin/oasis-market-maker-keeper \
        --rpc-host 127.0.0.1 \
        --rpc-port <port number> \
        --rpc-timeout 10 \
        --eth-from [address of your generated Ethereum account] \
    		--eth-key "key_file=./keystore.json,pass_file=keystore.pass" \
        --tub-address 0x448a5065aebb8e423f0896e6c5d525c040f59af3 \
        --oasis-address 0x14fbca95be7e99c15cc2996c6c9d841e54b79425 \
        --price-feed fixed:200 \
        --buy-token-address [address of the quote token, could be DAI] \
        --sell-token-address [address of the base token, could be WETH] \
        --config [path to the json bands configuration file, e.g. bands.json] \
        --smart-gas-price \
        --ethgasstation-api-key API_KEY_HERE \
        --min-eth-balance 0.001


  • Ensure that you retrieve and paste your correct contract addresses when using the above snippet.
  • --eth-key ${ACCOUNT_KEY} - includes both the .json file (account.json) of your account and a .pass file (ex: account.pass) that contains your password in plaintext.
  • If you do not have an account, you can use MyEtherWallet on Kovan and export the account details (by means of the Keystore file method). Make sure that you download the .json file to your local machine as this is what you will need to set up the account.

List of required Kovan Addresses for the above :

V2_OASIS_SERVER1_ADDRESS= <account address on Kovan>
V2_OASIS_SERVER1_KEY="key_file=/home/ed/Projects/member-account.json,pass_file=/home/ed/Projects/member-account.pass"
TUB_ADDRESS=0xa71937147b55deb8a530c7229c442fd3f31b7db2 # tub-address
SAI_ADDRESS=0xc4375b7de8af5a38a93548eb8453a498222c4ff2 # buy-token-address
WETH_ADDRESS=0xd0a1e359811322d97991e03f863a0c30c2cf029c # sell-token-address
OASIS_ADDRESS_NEW=0x4a6bc4e803c62081ffebcc8d227b5a87a58f1f8f # oasis-address

General Notes:

  • The OASIS_SERVER1_KEY is simply your Kovan account private key (point this to your ETH accounts key file) and password file. If you do not have this, please set up a file with your password (in plain text).
  • ETH From is the address location where the market-maker-keeper is going to get the tokens that it uses to participate and place orders.
    • Example: Since OasisDEX is a decentralized exchange, it is on-chain, so you need to provide all of the relevant addresses to the dex. Most DEX's are like this because when you are configuring with a dex you need to pass many addresses in, whereas, with a centralized exchange you are generally giving an API key, and username and password (see below for an example of how the process differs for centralized exchanges differ versus decentralized exchanges).

Once completed, you can now run your Market Maker Keeper! Follow the next steps to get it running:

  1. Open up your terminal
  2. Run: chmod +x <the file name of your version of the above bin/oasis-market-maker-keeper snippet>
  3. Run ./<the file name of your version of the above bin/oasis-market-maker-keeper snippet>
  4. That's it, your Keeper should now be running!

Keepers on Centralized Exchanges vs. Decentralized Exchanges

In the situation where you want to use a centralized exchange vs. a decentralized exchange, the process differs a little:

  1. You would need to have an existing account or create an account (on the exchange itself).
  2. Get the set of API keys with trading permissions (will usually need to be generated as well).
  3. Deposit tokens in your account on the exchange (as the keepers do not handle deposits and withdrawals themselves).
  4. Run the Market Maker Keeper.

11. Known limitations

The gate.io API sometimes does not acknowledge order creation, returning following error message: Oops... reloading...<font color=white> 29.148 </font> <script> function r(){window.location.reload();}setTimeout('r()',3000);</script>. This error seems to depend on the API address of the caller. Despite these errors, orders get properly created and registered in the backend, the keeper will find out about it the next time it queries the open orders list (which happens every few seconds).

12. Install and run using Docker and Docker Compose

  1. Install Docker Community Edition for your OS:
https://docs.docker.com/install/
  1. Install Docker Compose for your OS:
https://docs.docker.com/compose/install/
  1. Clone the market-maker-keeper repository and switch into its directory:
git clone https://github.com/makerdao/market-maker-keeper.git
cd market-maker-keeper
  1. Initializing the git submodules that will bring in both the pymaker and the pyexchange library.
git submodule update --init --recursive
  1. Build keeper Docker image.
docker-compose build keeper
  1. Create configuration directory.
    This directory will hold all configuration files (bands) for keepers
mkdir docker-config
  1. Create environment variables.
    Create a file named .env and add all environment variables required to run keeper. docker-compose.yml file contains a configuration example for Coinbase ETH-DAI keeper that can be started with an .env file similar with the one below
COINBASE_API_KEY=API_KEY_HERE
COINBASE_SECRET_KEY=SECRET_KEY_HERE
COINBASE_PASSWORD=PASSWORD_HERE
ETH_DAI_PRICE_FEED=ws://user:readonly@localhost:7777/price/ETH-DAI/socket
COINBASE_ETH_DAI_BANDS=coinbase-ethdai-bands.json

where:
COINBASE_API_KEY, COINBASE_SECRET_KEY and COINBASE_PASSWORD are specific to your Coinbase account
ETH_DAI_PRICE_FEED is the price feed that keeper will use to place orders on Coinbase (see https://github.com/makerdao/uniswap-price-feed for how you can start a Uniswap price feed)
COINBASE_ETH_DAI_BANDS is the name of bands file to be used by keeper (file should be placed in docker-config directory)

  1. Run keeper.
docker-compose up coinbase-ethdai-keeper

13. Token Configuration

An Example token configuration files can be seen below. This configuration should be placed into a seperate .json file, with the path to that file passed as an argument to any keeper requiring pymaker.model.Token objects which simplify decimal normalization within pairs, while also enhancing type checking.

MAKE SURE TO CHECK ALL TOKEN ADDRESSES AND DECIMALS IN CONFIGURATION PRIOR TO RUNNING KEEPERS WITH REAL FUNDS

{
  "tokens": {
    "ETH": {
      "tokenAddress": "0x0000000000000000000000000000000000000000"
    },
    "WETH": {
      "tokenAddress": "<WETH TOKEN ADDRESS>"
    },
    "DAI": {
      "tokenAddress": "<DAI TOKEN ADDRESS>"
    },
    "MKR": {
      "tokenAddress": "<MKR TOKEN ADDRESS>"
    },
    "BAT": {
      "tokenAddress": "<BAT TOKEN ADDRESS>"
    },
    "WBTC": {
      "tokenAddress": "<WBTC TOKEN ADDRESS>",
      "tokenDecimals": 8
    },
    "USDC": {
      "tokenAddress": "<USDC TOKEN ADDRESS>",
      "tokenDecimals": 6
    }
  }
}

14. Support

We are here to help! We welcome any questions about the market making in the #keeper channel in the Maker Chat.

15. License

See COPYING file.

Disclaimer

YOU (MEANING ANY INDIVIDUAL OR ENTITY ACCESSING, USING OR BOTH THE SOFTWARE INCLUDED IN THIS GITHUB REPOSITORY) EXPRESSLY UNDERSTAND AND AGREE THAT YOUR USE OF THE SOFTWARE IS AT YOUR SOLE RISK. THE SOFTWARE IN THIS GITHUB REPOSITORY IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. YOU RELEASE AUTHORS OR COPYRIGHT HOLDERS FROM ALL LIABILITY FOR YOU HAVING ACQUIRED OR NOT ACQUIRED CONTENT IN THIS GITHUB REPOSITORY. THE AUTHORS OR COPYRIGHT HOLDERS MAKE NO REPRESENTATIONS CONCERNING ANY CONTENT CONTAINED IN OR ACCESSED THROUGH THE SERVICE, AND THE AUTHORS OR COPYRIGHT HOLDERS WILL NOT BE RESPONSIBLE OR LIABLE FOR THE ACCURACY, COPYRIGHT COMPLIANCE, LEGALITY OR DECENCY OF MATERIAL CONTAINED IN OR ACCESSED THROUGH THIS GITHUB REPOSITORY.

market-maker-keeper's People

Contributors

bargst avatar cpstl avatar dittaeva avatar dizzy avatar ednoepel avatar grandizzy avatar ith-harvey avatar livnev avatar mbrock avatar mikehathaway avatar mitakash avatar petertheone avatar rainbreak avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

market-maker-keeper's Issues

Moving walls on Oasis in a single transaction (via `TxManager`)

It might be possible to kill and recreate orders in a single Ethereum transaction (via a TxManager for example) for the oasis-market-maker-keeper. Currently there is a significant delay between when the order gets killed and when its replacement gets created, and during that time this specific band is not covered by the keeper in the orderbook.

This might not be trivial to implement as the killed order may be partially or completely taken by some other actor before the "combo" transaction gets mined, which means we may not be able to know what the pay_amount of the new order should be. The advantage of doing it in individual transactions (as it is now) is the fact that we can always be sure of that amount.

Should serve the closest bands if low on tokens

If any of the market maker keepers is running out of SAI or WETH tokens, it should in the first order try to serve the bands closest to the reference price i.e. the ones with the lowest margin (spread).

I guess that currently this order is undetermined, it will probably depend on the ordering in which bands are specified in the config file.

Do not cancel EtherDelta orders if they move away from the centre

If price moves in one direction, we need to cancel the orders that get too close to the centre, but in the case of EtherDelta there is really no need to cancel the ones on the other side that move further from the centre. They will expire anyway, if somebody takes takes them before that we will earn more than we originally planned to, and - what is most important in case of EtherDelta - existing orders to not "eat up" from the deposited balance so they do not block us from placing new orders. I.e. same deposited money can be used for placing multiple orders.

This strategy could potentially save us around 50% gas.

okex tick

get okex market data throught restAPI? Not Websocket?

now problems with the ./install.sh and ./test.sh

ERROR: Could not find a version that satisfies the requirement web3==4.8.2 (from versions: 0.1.7, 0.1.8, 0.1.9, 1.0.0, 1.0.1, 1.1.0, 1.2.0, 1.3.0, 1.4.0, 1.5.0, 1.6.0, 1.7.0, 1.7.1, 1.8.0, 1.9.0, 2.0.0, 2.1.0, 2.1.1, 2.2.0, 2.3.0, 2.4.0, 2.5.0, 2.6.0, 2.7.0, 2.7.1, 2.8.0, 2.8.1, 2.9.0, 3.0.0, 3.0.1, 3.0.2, 3.1.0, 3.1.1, 3.2.0, 3.3.0, 3.4.0, 3.4.1, 3.4.2, 3.4.3, 3.4.4, 3.5.0, 3.5.1, 3.5.2, 3.5.3, 3.6.0, 3.6.1, 3.6.2, 3.7.0, 3.7.1, 3.7.2, 3.8.0, 3.8.1, 3.9.0, 3.10.0, 3.11.0, 3.11.1, 3.13.0, 3.13.1, 3.13.2, 3.13.3, 3.13.4, 3.13.5, 3.14.0, 3.14.1, 3.14.2, 3.15.0, 3.16.0, 3.16.1, 3.16.2, 3.16.3, 3.16.4, 3.16.5, 4.0.0b1, 4.0.0b2, 4.0.0b3, 4.0.0b4, 4.0.0b5, 4.0.0b6, 4.0.0b7, 4.0.0b9, 4.0.0b10, 4.0.0b11, 4.0.0b12, 4.0.0b13, 4.0.0, 4.1.0, 4.2.0, 4.2.1, 4.3.0, 4.4.0, 4.4.1, 4.5.0, 4.6.0, 4.7.1, 4.7.2, 4.8.0, 4.8.1)
ERROR: No matching distribution found for web3==4.8.2

and

~/market-maker-keeper $ ./test.sh
./test.sh: 2: ./test.sh: source: not found
Traceback (most recent call last):
File "/home/pi/.local/bin/py.test", line 6, in
from pytest import main
File "/home/pi/.local/lib/python3.7/site-packages/pytest.py", line 13, in
from _pytest.fixtures import fixture, yield_fixture
File "/home/pi/.local/lib/python3.7/site-packages/_pytest/fixtures.py", line 839, in
class FixtureFunctionMarker(object):
File "/home/pi/.local/lib/python3.7/site-packages/_pytest/fixtures.py", line 841, in FixtureFunctionMarker
params = attr.ib(convert=attr.converters.optional(tuple))
TypeError: attrib() got an unexpected keyword argument 'convert'

now that i think of it... this second part may because the first problem but i am still puting it here just incase

Do price rounding better in `sai_maker_otc.py`

Due to precision issues, the price rounding mechanism currently used in sai_maker_otc.py doesn't guarantee that a price will always be displayed with the required number of decimal places (e.g. in the Oasis frontend). It may be preferable to round the quantities as well (but to a different number of digits) in order to achieve a consistent number of decimal places.

Authored by @livnev, moved from: makerdao/pymaker#35.

kucoin-python==2.0.5

ERROR: Could not find a version that satisfies the requirement kucoin-python==2.0.5 (from versions: 0.1.1)
ERROR: No matching distribution found for kucoin-python==2.0.5

I keep getting the above mentioned error in keeper and uniswap price feed installation.
Is there any way to fix it?

Thank you

`dustCutoff` property in bands configuration file is misleading

The dustCutoff property is specified either in SAI or (W)ETH, depending on whether it’s a buy or a sell band. Would be nice to rename this property to dustSaiCutoff/dustWEthCutoff or dustCutoffSai/dustCutoffWEth. Having said that, the keeper should still be able to read old band configuration files - it should just warn that a deprecated property is being used.

Start

This is really dumb comment but... in the instructions... there is no indication of how to start the bot... Is it make start or...?

./install.sh not working (Using Python 3.6.6)

After installing Python 3.6.6 and creating virtenv ; I run ./install and I'm getting that python 2.7 has reached end of life. I never installed 2.7 only 3.6.6

~/market-maker-keeper# ./install.sh
created virtual environment CPython2.7.17.final.0-64 in 206ms
  creator CPython2Posix(dest=/root/market-maker-keeper/_virtualenv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, wheel=bundle, setuptools=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv)
    added seed packages: pip==20.3.1, setuptools==44.1.1, wheel==0.36.1
  activators PythonActivator,CShellActivator,FishActivator,PowerShellActivator,BashActivator
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Collecting cachetools==3.1.1
  Using cached cachetools-3.1.1-py2.py3-none-any.whl (11 kB)
Collecting dydx-python==0.11.3
  Using cached dydx-python-0.11.3.tar.gz (33 kB)
ERROR: Could not find a version that satisfies the requirement eth-account==0.5.2 (from versions: 0.1.0a1, 0.1.0a2, 0.2.0a0, 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.3.0)
ERROR: No matching distribution found for eth-account==0.5.2

Also Pip is already upgraded:

~/web3.py# pip show pip
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Name: pip
Version: 20.3.1
Summary: The PyPA recommended tool for installing Python packages.
Home-page: https://pip.pypa.io/
Author: The pip developers
Author-email: [email protected]
License: MIT
Location: /usr/local/lib/python2.7/dist-packages
Requires: 
Required-by: 

`oasis-market-maker-keeper` still places orders after shutdown

Looks like order placement still happens in background when we terminate the keeper:

2018-05-04 00:14:31,976 INFO     Waiting 60 seconds in order to perform the final check...
2018-05-04 00:15:23,173 INFO     Sent transaction MatchingMarket('0x14fbca95be7e99c15cc2996c6c9d841e54b79425').offer([200000000000000000, '0x9f8f72aa9304c8b593d555f12ef6589cc3a579a2', 297903314442911380, '0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2', 51565]) with nonce=7251, gas=342110, gas_price=14400000000 (tx_hash=0x50e95c9481258c14e1e56fab9b5cae488182e89d1288b5053b664cc4a07649e7)
2018-05-04 00:15:23,204 INFO     Sent transaction MatchingMarket('0x14fbca95be7e99c15cc2996c6c9d841e54b79425').offer([200000000000000000, '0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2', 136984312794213819, '0x9f8f72aa9304c8b593d555f12ef6589cc3a579a2', 0]) with nonce=7252, gas=330943, gas_price=14400000000 (tx_hash=0x2ff8d2c31cba03965f4c54693f57444a29b789b2ea36c3725f3d79d90bfbfc28)
2018-05-04 00:15:32,036 INFO     Waiting for the order book to refresh...
2018-05-04 00:15:49,700 INFO     Still no open orders after the final check.
2018-05-04 00:15:49,700 INFO     Shutdown logic finished
2018-05-04 00:15:49,700 INFO     Keeper terminated
2018-05-04 00:16:05,917 INFO     Transaction MatchingMarket('0x14fbca95be7e99c15cc2996c6c9d841e54b79425').offer([200000000000000000, '0x9f8f72aa9304c8b593d555f12ef6589cc3a579a2', 297903314442911380, '0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2', 51565]) was successful (tx_hash=0x2d10b2eda2d487c3989474ce2f2cea3f18368e61a99c9c14fb2e5e44b4522cdf)
2018-05-04 00:16:26,085 INFO     Transaction MatchingMarket('0x14fbca95be7e99c15cc2996c6c9d841e54b79425').offer([200000000000000000, '0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2', 136984312794213819, '0x9f8f72aa9304c8b593d555f12ef6589cc3a579a2', 0]) was successful (tx_hash=0x2ff8d2c31cba03965f4c54693f57444a29b789b2ea36c3725f3d79d90bfbfc28)

typo in readme.md

In section "ethfinex-market-maker-keeper" description speaks about Idex exchange: "This keeper supports market-making on the IDEX exchange". Must be corrected by ethfinex, link is ok.

ReadMe seems to be missing details about the incentives paid to keepers?

The word incentive is mentioned twice:
The DAI Stablecoin System incentivizes external agents, called keepers, to automate certain operations around the Ethereum blockchain.
and
A big part of the DAI Stablecoin System (DSS) is the incentivization of external agents, called Keepers (which can be human but are typically automated bots).
but appears missing reference to any incentivisation. Point 9 Running Market Maker Keepers explains the command line arguments, but nothing about how the payouts are calculated. I guess that is coming from another part of the program (Maker DAO say), but for noobs like me would be good to have a link to say... the page on the component where this does reside. (I'll see about adding it and adding a pull request once I find it?!)

Running test.sh on Windows 10

I am trying to run the test.sh script on Windows 10 after installation. I noticed that virtualenv on Windows 10 downloads the activate/deactivate files into the "Scripts" folder (vs bin on Linux). After changing the test.sh file to:
source _virtualenv/Scripts/activate
I am able to launch the virtualenv for testing purposes as intended. However, it does not seem that the PYTHONPATH line is working on my system. I receive the error (after doing the above and running test.sh):

File "C:\Users\XYZ\workspace\market-maker-keeper\tests\conftest.py", line 20, in <module>
    from pymaker.deployment import Deployment
ModuleNotFoundError: No module named 'pymaker'

This leads me to believe that there is an issue with test.sh changing the pythonpath for the tests. Any guidance on how I can get my tests performed on Windows 10 would be greatly appreciated. Love the project - Thanks!

EtherDelta keeper does not try to withdraw ETH if balance is low

If undeposited ETH balance is getting low, the EtherDelta keeper does not attempt to withdraw some of the ETH even if the deposited amount turns out to be much higher than it actually needs for maintaining the bands its configured for.

Of course this task is not as trivial as it looks as there might be a quandary if the deposited amount is too low to cover configured bands but we are getting low on ETH at the same time.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.