Giter VIP home page Giter VIP logo

conductor's Introduction

conductor - Steem Witness Toolkit

conductor is a user friendly, KISS utility for creating, updating and management of your witness.

Requirements

This tool has been tested on Linux, and requires Python 3.5 or higher.

On Ubuntu, you also need to install libssl-dev for cryptographic utilities.

sudo apt install libffi-dev libssl-dev python3 python3-dev python3-pip

We also need the Python library for Steem.

pip3 install -U git+git://github.com/Netherdrake/steem-python

Install

pip3 install -U git+https://github.com/Netherdrake/conductor

First Steps

conductor is built on top of steem-python, and leverages it's BIP38 encrypted wallet to store our witness Active key. This key is required for price feed updates.

To create the wallet, and add our key to it, simply run:

steempy addkey

(Optional) If you wish not to have to type your BIP38 wallet password every time you use conductor, set the UNLOCK environment variable.

export UNLOCK=your-wallet-pw

(Optional) You may also want to set backup RPC nodes to add resilience to conductor.

steempy set nodes https://steemd.steemit.com,https://rpc.steemliberator.com,https://gtg.steem.house:8090,https://steemd.privex.io

Creating or importing your witness

conductor init

conductor will ask you for your witness (Steem account) name. If the witness already exists, it will import its current settings from the blockchain.

Otherwise, it will guide you trough the setup wizard, and create the witness for you.

Updating your witness

If you wish to change one or more of your witness properties, such as witness url, interest rate, block size, etc., you can do so using conductor update command.

~ % conductor update
What should be your witness URL? [https://steemdb.com/@furion/witness]:
How much do you want the account creation fee to be (STEEM)? [0.500 STEEM]:
What should be the maximum block size? [65536]:
What should be the SBD interest rate? [0]: 10

Configuration:
---------------
{'props': {'account_creation_fee': '0.500 STEEM',
           'maximum_block_size': 65536,
           'sbd_interest_rate': 10},
 'witness': {'name': 'furion', 'url': 'https://steemdb.com/@furion/witness'}}

Do you want to commit the updated values? [y/N]: n
Aborted!
~ %

Generating block signing key-pairs

Each node deployment should have its own signing key, to avoid double-signing. We can generate new random (/dev/urandom based) key-pairs with a simple command:

conductor keygen

Enabling your witness

Enabling your witness is as simple as setting a public signing key. This command can also be used for key rotation (for example, if you're falling back to a backup witness node).

To set a public signing key on your witness run:

conductor enable <PUBLIC_SIGNING_KEY>

Disabling a witness

Sometimes you might want to disable your witness yourself. For example, if you're upgrading your witness server, and don't want to miss any blocks.

To disable your witness run:

conductor disable

Kill Switch

Kill Switch is a witness monitoring utility, that tracks block misses. If your witness server bugs out, and stops producing blocks, this tool will automatically disable your witness to avoid further misses.

To run a killswitch, simply run:

conductor kill-switch

Optionally, we can provide number of blocks number of blocks we are allowed to miss before kill-switch disables our witness. We can achieve this by providing -n argument, like so: conductor kill-switch -n 25. By default -n is 10.

Automatic Failover

We can use the Kill Switch to automatically failover as well. Instead of disabling our witness, the kill-switch can change our signing key to secondary key (backup node), and then monitor that. If all keys provided trough -k flags miss blocks as well, the witness is finally disabled.

Example

conductor kill-switch -n 2 -k <BACKUP_NODE_PUBLIC_SIGNING_KEY> -k <BACKUP_NODE_2> ...

See conductor kill-switch -h for more options.

Price Feeds

Price feeds are a vital component of STEEM ecosystem, as they power SBD->STEEM conversions, as well as rewards estimates. Witnesses act as an oracle between the blockchain and real-world, by providing honest input on what the implied price of STEEM is. Furthermore, the prices may contain bias to loosely support the SBD stablecoin's peg to USD.

conductor ships with markets module from SteemData. This module interfaces with 3rd party exchanges to fetch VWAP (volume weighted average prices) mean (average of VWAP's from all exchanges) prices.

Exchanges Used:
  • Bitstamp, Bitfinex, Kraken, OKCoin for BTC/USD
  • Poloniex, Bittrex for STEEM/BTC and SBD/BTC

To run pricefeeds, simply run:

conductor feed

Peg Support: Price feeds support SBD peg signalling. You can explicitly define whether or not you'd like to introduce bias to your price to support the loose $1 USD == $1 SBD peg. If no option is provided, pegging is disabled by default.

conductor feed --sbd-peg
conductor feed --no-sbd-peg

Usage

~ % conductor
Usage: conductor [OPTIONS] COMMAND [ARGS]...

  Steem Witness Toolkit.

Options:
  -h, --help  Show this message and exit.

Commands:
disable      Disable a witness.
enable       Enable a witness, or change key.
feed         Update Price Feeds.
init         Add your witness account.
keygen       Generate a random signing key-pair.
kill-switch  Monitor for misses w/ disable.
status       Print basic witness info.
tickers      Print Tickers.
update       Update witness properties.

There are two additional, read only commands we haven't covered yet. status and tickers. They simply print some info for us.

Status

~ % conductor status
Status:
-------
+---------+--------+-------------------------------------------------------+
| Enabled | Misses | Key                                                   |
+---------+--------+-------------------------------------------------------+
| True    | 105    | STM7WDG2QpThdkRa3G2PYXM7gH9UksoGm4xqoFBrNet6GH7ToNUYx |
+---------+--------+-------------------------------------------------------+

Configuration:
--------------
{'props': {'account_creation_fee': '0.500 STEEM',
           'maximum_block_size': 65536,
           'sbd_interest_rate': 0},
 'witness': {'name': 'furion', 'url': 'https://steemdb.com/@furion/witness'}}

~ %

Tickers

~ % conductor tickers
| Symbol    |   Price |
|-----------+---------|
| BTC/USD   | 2444.31 |
| SBD/USD   |   1.804 |
| STEEM/USD |    1.19 |
~ %

License

MIT.

Pull requests are welcome.

conductor's People

Contributors

netherdrake avatar someguy123 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

conductor's Issues

Invalid cast from string_type to Array

Conductor version 0.3.3

  File "/usr/local/lib/python3.5/dist-packages/steembase/http_client.py", line 192, in exec
    return_with_args=return_with_args)
  File "/usr/local/lib/python3.5/dist-packages/steembase/http_client.py", line 213, in _return
    raise RPCError(error_message)
steembase.exceptions.RPCError: Bad Cast:Invalid cast from string_type to Array

This happens when trying to do conductor enable STMxxxxxx.

I think something changed with the STEEM API recently (maybe appbase related), it's pretty frustrating as it isn't smart enough to fail over to another RPC node that is compatible.

TypeError: 'NoneType' object is not subscriptable

I am running conductor 0.3.1 and I get the following message:

Monitoring the witness, current misses: 1
Traceback (most recent call last):
  File "/home/witness/anaconda3/lib/python3.6/site-packages/conductor/watchdog.py", line 83, in watchdog
    diff = total_missed() - misses
  File "/home/witness/anaconda3/lib/python3.6/site-packages/conductor/watchdog.py", line 20, in total_missed
    return get_witness(witness('name'))['total_missed']
TypeError: 'NoneType' object is not subscriptable

It starts fine, and runs for a little while (5 to 30 minutes, maybe more), but then throws the error. I am not sure if it is still running, I just have a blinking cursor.

Improve key change predicates

A a few checks are needed prior to witness key change:
1.) Check current node's health
2.) If not failing over, check witness position in mining schedule, and avoid changing keys at critical times.

Using `conductor enable` results in old data being used

When running conductor enable the program will not look up the current values but rather use the cached values.

I've previously set the parameters from another machine and when switching signing node conductor enable <key> doesn't use the latest values.

not able to use conductor feed

Hi furion,

Any guesses I might e missing here? Works well with the rest.

I am using Ubuntu 16.04.3 LTS

Thu Nov 30 14:14:43 2017
Old Price: 1.167
New Price: 0.994

Current STEEM price: 0.994 USD
Current SBD price: 1.245 USD
Quote: 1.000 STEEM

Spread between prices: 17.404%
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/conductor/feeds.py", line 60, in run_price_feeds
    refresh_price_feeds(witness('name'), **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/conductor/feeds.py", line 50, in refresh_price_feeds
    steem.commit.witness_feed_publish(steem_usd, quote=quote, account=witness_name)
  File "/home/user/.local/lib/python3.5/site-packages/steem/commit.py", line 858, in witness_feed_publish
    return self.finalizeOp(op, account, "active")
  File "/home/user/.local/lib/python3.5/site-packages/steem/commit.py", line 126, in finalizeOp
    tx.sign()
  File "/home/user/.local/lib/python3.5/site-packages/steem/transactionbuilder.py", line 108, in sign
    raise MissingKeyError
steembase.exceptions.MissingKeyError

When using "conductor update", does not cast the creation fee correctly

During the question

How much should be account creation fee?

What happens: Entering 0.2 results in an error

ValueError: not enough values to unpack (expected 2, got 1)

What should happen: 0.2 should be automatically padded to 3 DP, with "STEEM" placed in front

e.g. 0.200 STEEM

Temporary fix, enter the full data 0.200 STEEM

Conductor not compatible with funcy update

On new conductor installs I run into this issue:

└─[$] <git:(master)> conductor status
Traceback (most recent call last):
  File "/usr/local/bin/conductor", line 7, in <module>
    from conductor.cli import conductor
  File "/usr/local/lib/python3.6/dist-packages/conductor/cli.py", line 6, in <module>
    from funcy import silent
ImportError: cannot import name 'silent'

Either you need to upgrade conductor to be compatible with the new funcy, or you need to lock in the older version 1.10 into requirements.txt

e.g. I did this and conductor worked again:

pip3 install -U funcy==1.10

Feed Price Drift

When conductor feed runs for a longer period of time, the mean price starts drifting.

To be investigated.

steembase.storage.WrongMasterPasswordException when using correct password calling feed

Hello,

I am currently using conductor to aid in setting up a witness node. I have set UNLOCK in my environment to my BIP38 wallet password. This works great when calling conductor update, but I get a stack trace when trying to call conductor feed. I'm not entirely sure why the password is working in one scenario and considered bad in the other. Any ideas? Here's the stack trace:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/steembase/storage.py", line 409, in decryptEncryptedMaster
    decrypted_master = aes.decrypt(encrypted_master)
  File "/usr/local/lib/python3.5/dist-packages/steem/aes.py", line 40, in decrypt
    return self._unpad(cipher.decrypt(enc[AES.block_size:])).decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfb in position 2: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/conductor/feeds.py", line 70, in run_price_feeds
    refresh_price_feeds(witness('name'))
  File "/usr/local/lib/python3.5/dist-packages/conductor/feeds.py", line 51, in refresh_price_feeds
    steem.commit.witness_feed_publish(steem_usd, quote=quote, account=witness_name)
  File "/usr/local/lib/python3.5/dist-packages/steem/commit.py", line 858, in witness_feed_publish
    return self.finalizeOp(op, account, "active")
  File "/usr/local/lib/python3.5/dist-packages/steem/commit.py", line 125, in finalizeOp
    tx.appendSigner(account, permission)
  File "/usr/local/lib/python3.5/dist-packages/steem/transactionbuilder.py", line 67, in appendSigner
    keys = fetchkeys(account)
  File "/usr/local/lib/python3.5/dist-packages/steem/transactionbuilder.py", line 55, in fetchkeys
    wif = self.wallet.getPrivateKeyForPublicKey(authority[0])
  File "/usr/local/lib/python3.5/dist-packages/steem/wallet.py", line 232, in getPrivateKeyForPublicKey
    return self.decrypt_wif(self.keyStorage.getPrivateKeyForPublicKey(pub))
  File "/usr/local/lib/python3.5/dist-packages/steem/wallet.py", line 163, in decrypt_wif
    self.unlock()
  File "/usr/local/lib/python3.5/dist-packages/steem/wallet.py", line 100, in unlock
    masterpwd = self.MasterPassword(pwd)
  File "/usr/local/lib/python3.5/dist-packages/steembase/storage.py", line 401, in __init__
    self.decryptEncryptedMaster()
  File "/usr/local/lib/python3.5/dist-packages/steembase/storage.py", line 411, in decryptEncryptedMaster
    raise WrongMasterPasswordException
steembase.storage.WrongMasterPasswordException

Broken english in update dialog

The message

How much should be account creation fee? [0.2]

Is broken english.

It should be more like:

How much do you want the account creation fee to be (STEEM)? [0.2]

Or at least simplified to:

Account creation fee (STEEM)? [0.2]

updates needed for HF20

@gitschwifty wrote

Witness init and update now ask for account_subsidy_budget and account_subsidy_decay, kill-switch now requires environment unlock because of KEK upgrade of steempy, unlock wallet unused because of updates as well.

PR is here : #19

ImportError: cannot import name 'MasterPassword

A refactoring in steem-python is having an impact on conductor.utils.py resulting into a runtime error:

https://github.com/steemit/steem-python/commit/e4f908aef6035cfc957a4713b2ce9c88baeead2d

Traceback (most recent call last):
  File "/usr/local/bin/conductor", line 11, in <module>
    load_entry_point('conductor==0.3.4', 'console_scripts', 'conductor')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 480, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2693, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2324, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2330, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/local/lib/python3.6/dist-packages/conductor-0.3.4-py3.6.egg/conductor/cli.py", line 12, in <module>
  File "/usr/local/lib/python3.6/dist-packages/conductor-0.3.4-py3.6.egg/conductor/feeds.py", line 8, in <module>
  File "/usr/local/lib/python3.6/dist-packages/conductor-0.3.4-py3.6.egg/conductor/utils.py", line 4, in <module>
ImportError: cannot import name 'MasterPassword'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.