agicommies / communex Goto Github PK

View Code? Open in Web Editor NEW

24.0 3.0 15.0 919 KB

alternative commune cli/sdk

License: MIT No Attribution

Python 97.62% Makefile 0.30% Nix 2.08% Shell 0.01%

blockchain library rpc-client rpc-framework sdk sdk-python substrate communeai

communex's Introduction

CommuneX

Why CommuneX
Installation with pip
Installation with Nix
Features
- Planned
CLI Usage
- Examples
- Completions
Contributing
Commune compatibility

Why CommuneX

CommuneX serves as an alternative library/SDK to the original Commune Ai codebase, offering a streamlined and user-friendly experience. It is designed for simplicity and scalable development. To learn more visit docs

Installation with `pip`

Requirements: Python 3.10+

Install the communex Python package directly with pip:

pip install communex

Or add it to your Poetry project with:

poetry add communex

Installation with Nix

To install communex the communex cli with Nix

nix profile install .

Features

Planned

Module API extraction and documentation generator

CLI Usage

The CLI commands are structured as follows:

comx [OPTIONS] COMMAND [ARGS]

There are six top-level subcommands:

balance: transfer, stake, unstake and showing balance operations
key: creating, saving (AKA regenerating), listing and showing balance operations
module: info, list, register, serve, update
network: block, parameters, proposals / proposing, voting operations
subnet: info, list, update
misc: apr, circulating supply

comx subcommand [OPTIONS] COMMAND [ARGS]...

Examples

Retrieving Balance

# Show staked, free and total balance.
comx balance show 5FgfC2DY4yreEWEughz46RZYQ8oBhHVqD9fVq6gV89E6z4Ea

Creating a Key

comx key create key_name

Retrieving Key Info

comx key show key_name

# Add the `--show-private` flag to show sentitive fields like private key.
comx key show key_name --show-private

Listing Keys

# Lists the names and addresses of keys stored on disk.
comx key list

List Keys With Balances

# Lists keys stored on disk with their balance (free, staked and total).
comx key balances

Retrieving Module Information

# Note that the module has to be registered on the network.
comx module info vali::calc [--balance]

Retrieving Global Parameters

comx network params

Retrieving Subnet Parameters

comx subnet list

Retrieving Circulating Supply

# Gets all tokens then were ever emitted minus burned tokens.
comx misc circulating-supply

Completions

You can enable completions for your shell by running:

# On bash
comx --install-completion bash
# On zsh
comx --install-completion zsh

Contributing

Bug reports and pull requests and other forms of contribution are welcomed and encouraged! :)

To report a bug or request a feature, please open an issue on GitHub.

If you have any questions, feel free to ask on the CommuneX Discord channel or post on our GitHub discussions page.

To contribute to the codebase, using Poetry you can install the development dependencies with:

poetry install --with dev

it can require some enviroment-specific binaries to be installed

Commune compatibility

Yes, communex is compatible with the commune library/CLI. However, there are important considerations to note. communex verifies the integrity of your keys, which means that mixing certain types of keys is not permissible. Specifically, if you possess node keys or other similar types that are not designed to receive tokens, you to relocate them outside of the key directory.

communex's People

Contributors

Stargazers

Watchers

Forkers

agent-artificial bakobiibizo ddayan comsci-ai 0xdeadbad 0xcryptofire rjmillerpy callmewhy commune-ai lvivona gold-coding silver-coding-blockchain ashikshafi08

communex's Issues

show version on `--help` and `--version`

We should be able to see the current version of comx with:

comx --help
comx --version

https://typer.tiangolo.com/tutorial/options/version/

comx --testnet balance run-faucet key - error: ValueError: Pallet 'FaucetModule' not found

i try to run faucet:
comx --testnet balance run-faucet <my_key>

error message:
ValueError: Pallet 'FaucetModule' not found

`_ss58_address` In key format

Commune was using _ss58_address for an extended period of time in the past. Which is in our repository considered invalid. Currently we are working on a warning instead of an Error / Exception

Invalid key format (Stored node keys)

Example of a node key structure

{"data": "{"crypto_type": 1, "seed_hex": "ac99589a883cd29f92703cedf776782ced3242a233761fb4af2cf910b6e64d9d", "derive_path": null, "path": "subspace.node.main.alice.aura", "ss58_format": 42, "public_key": "bcc8573a4efc3f4f131765d5d161d506af161eb69e1f907b9763de1e3f6f9d1e", "ss58_address": "5GLENbQdPWymoSMBonewqR2kisZd7ALfnGFkgByNof2HCWLf", "private_key": "af34dda12dc5cbbda38c4404e391152abdcc057c74fa2a52ab8f5af0e76f1208616e6c181bd3c3e7613b3f60ea5723b39ed00d1d9c027e0f65156232da8dbe68", "mnemonic": "quote romance syrup borrow skull resemble connect prepare hunt forward share view"}", "encrypted": false, "timestamp": 1701111511}

Work on ignoring this key format is currently in progress.

`-y` flag to bypass confirmations

It should be a global flag implemented in the main callback, that will inject the flag value in the context in a property that's used in commands like comx balance transfer to auto-accept confirmations.

communex/src/communex/cli/root.py

Line 45 in e27c6f2

ctx.obj = ExtraCtxData(output_json=json, use_testnet=testnet)

communex/src/communex/cli/balance.py

Line 126 in e27c6f2

 if not context.confirm(f"Are you sure you want to transfer {amount} tokens to {dest}?"): 

Improving handling node connections

Try to connect to other nodes with exponential fallback when node connection fails.

eth-typing warnings

Describe the bug

Every comx command results with several etc-typing invalid ChainId warnings.

To Reproduce

Running comx --help

Expected behavior

Shows help menu without warnings.

Guide markdown docs

Type-check and tests automation on GitHub

communex/Makefile

Lines 11 to 25 in f886e72

 check: lint type_check 

 lint: 

 ruff check ./src ./tests 

 type_check: 

 pyright ./src ./tests 

 test_all: test test_slow 

 test: 

 pytest -k "not slow" 

 test_slow: 

 pytest -k "slow"

refac: `netuid` from Optional to Required

We are past the point of operating only on S0, take out the optional netuid flag -> make it required everywhere.

[Errno -3] Temporary failure in name resolution

User has reported this bug when running the command

comx key balances

Error message:

│
│   643 │   │   **options,                                                     │
│   644 │   )                                                                  │
│   645 │   websock.settimeout(timeout if timeout is not None else getdefaultt │
│ ❱ 646 │   websock.connect(url, **options)                                    │
│   647 │   return websock                                                     │
│   648                                                                        │
│                                                                              │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │               class_ = <class 'websocket._core.WebSocket'>               │ │
│ │   enable_multithread = True                                              │ │
│ │      fire_cont_frame = False                                             │ │
│ │              options = {                                                 │ │
│ │                        │   'max_size': 4294967296,                       │ │
│ │                        │   'read_limit': 4294967296,                     │ │
│ │                        │   'write_limit': 4294967296                     │ │
│ │                        }                                                 │ │
│ │ skip_utf8_validation = False                                             │ │
│ │              sockopt = []                                                │ │
│ │               sslopt = {}                                                │ │
│ │              timeout = None                                              │ │
│ │                  url = 'wss://commune.api.onfinality.io/public-ws'       │ │
│ │              websock = <websocket._core.WebSocket object at              │ │
│ │                        0xffff924035b0>                                   │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│                                                                              │
│ /usr/local/lib/python3.10/dist-packages/websocket/_core.py:256 in connect    │
│                                                                              │
│   253 │   │   │   Pre-initialized stream socket.                             │
│   254 │   │   """                                                            │
│   255 │   │   self.sock_opt.timeout = options.get("timeout", self.sock_opt.t │
│ ❱ 256 │   │   self.sock, addrs = connect(                                    │
│   257 │   │   │   url, self.sock_opt, proxy_info(**options), options.pop("so │
│   258 │   │   )                                                              │
│   259                                                                        │
│                                                                              │
│ ╭──────────────────────────── locals ────────────────────────────╮           │
│ │ options = {                                                    │           │
│ │           │   'max_size': 4294967296,                          │           │
│ │           │   'read_limit': 4294967296,                        │           │
│ │           │   'write_limit': 4294967296                        │           │
│ │           }                                                    │           │
│ │    self = <websocket._core.WebSocket object at 0xffff924035b0> │           │
│ │     url = 'wss://commune.api.onfinality.io/public-ws'          │           │
│ ╰────────────────────────────────────────────────────────────────╯           │
│                                                                              │
│ /usr/local/lib/python3.10/dist-packages/websocket/_http.py:133 in connect    │
│                                                                              │
│   130 │   if socket:                                                         │
│   131 │   │   return socket, (hostname, port_from_url, resource)             │
│   132 │                                                                      │
│

❱ 133 │   addrinfo_list, need_tunnel, auth = _get_addrinfo_list(             │
│   134 │   │   hostname, port_from_url, is_secure, proxy                      │
│   135 │   )                                                                  │
│   136 │   if not addrinfo_list:                                              │
│                                                                              │
│ ╭─────────────────────────────── locals ────────────────────────────────╮    │
│ │      hostname = 'commune.api.onfinality.io'                           │    │
│ │     is_secure = True                                                  │    │
│ │       options = <websocket._socket.sock_opt object at 0xffff924035e0> │    │
│ │ port_from_url = 443                                                   │    │
│ │         proxy = <websocket._http.proxy_info object at 0xffff92403700> │    │
│ │      resource = '/public-ws'                                          │    │
│ │        socket = None                                                  │    │
│ │           url = 'wss://commune.api.onfinality.io/public-ws'           │    │
│ ╰───────────────────────────────────────────────────────────────────────╯    │
│                                                                              │
│ /usr/local/lib/python3.10/dist-packages/websocket/_http.py:187 in            │
│ _get_addrinfo_list                                                           │
│                                                                              │
│   184 │   │   │   )                                                          │
│   185 │   │   │   return addrinfo_list, True, pauth                          │
│   186 │   except socket.gaierror as e:                                       │
│ ❱ 187 │   │   raise WebSocketAddressException(e)                             │
│   188                                                                        │
│   189                                                                        │
│   190 def _open_socket(addrinfo_list, sockopt, timeout):                     │
│                                                                              │
│ ╭───────────────────────────── locals ──────────────────────────────╮        │
│ │  hostname = 'commune.api.onfinality.io'                           │        │
│ │ is_secure = True                                                  │        │
│ │     pauth = None                                                  │        │
│ │     phost = None                                                  │        │
│ │      port = 443                                                   │        │
│ │     pport = 0                                                     │        │
│ │     proxy = <websocket._http.proxy_info object at 0xffff92403700> │        │
│ ╰───────────────────────────────────────────────────────────────────╯        │
╰──────────────────────────────────────────────────────────────────────────────╯
WebSocketAddressException: [Errno -3] Temporary failure in name resolution```

transfer all

Enhanced transferring features

Background

We've received several requests for a feature that enables users to unstake all keys on a disk and transfer their balances to a specified address. This feature aims to simplify the management of multiple wallet addresses.

Example that Huck sent (applied to the commune repository logic):

import argparse
import commune as c

# Set up command-line argument parsing
parser = argparse.ArgumentParser(description='Transfer all balance to a specific address.')
parser.add_argument('--address', type=str, help='The address to transfer to.')

parser.add_argument('--min', type=float, default=0.5, help='Min stake amount to left. Default is 0.5.')

# Parse command-line arguments
args = parser.parse_args()

wallet_addresses = c.keys()
print(wallet_addresses)

for address in wallet_addresses:
    try:
        # Unstake all balance from the wallet
        staked = c.get_stake(address)
        print(f"Staked amount is {staked}")
        if (staked > 0):
            c.unstake(staked, address)
        
        # Transfer all balance to the specific address
        balance = c.get_balance(address)
        print(f"Total balance is {balance}")

        if (balance > args.min):
            c.transfer(args.address, balance - args.min, address)

    except Exception as e:
        # Code that will run if the exception is raised
        print(f"An error occurred: {e}")```

Discussion and Feedback:

We are looking to the community for feedback on this feature. Please share your thoughts, potential use cases, or suggestions to improve this feature. Your input is valuable in shaping this enhancement to better suit user needs.

CI and git hook checks

For ruff, pyright, nix etc

Permission denied when creating keys

If you run into [Errno 13] Permission denied: '/home/$USER/.commune/key'
when creating or manipulating your keys.
Make sure to run the following:

sudo chown -R $USER:$USER ~/.commune

feat module: rate limiter

TASKS

Cli improvement: make process automatically run on the background (pm2 managed), user can specify a flag to run on foreground. Make sure that --help specifies very explicitly how to pass the path of the module. I.e format folder.file.Class
Rate limiter that @gabrielcipriano made. So we avoid DDOS attacks.
Make sure that the client that is calling the api is registered on the network.
Ability to blacklist certain calling keys.
Ability to whitelist certain calling keys. (when used, other keys by default blacklisted)

Better handle existential deposits in transfers

Currently, we are subtracting the "existential deposit" from the transfer amount before doing transfers.

This should be handled in another way.

   def transfer(
            self,
            key: Keypair,
            amount: int,
            dest: Ss58Address,
    ) -> ExtrinsicReceipt:
    # ...
    amount = amount - self.get_existential_deposit()

nix flake

We should have at least a Nix flake that provides some binaries for development, like poetry and ruff (#15).

Eventually, we can add poetry2nix so we can build the actual Python package with Nix based on the Poetry project.

There are a bunch of potential overrides that we can merge upstream to poetry2nix for packages as substrate-interface and its dependencies like py-bip39-bindings, py-ed25519-zebra-bindings, py-bip39-bindings.

Hello

Hello.
Nice to meet you
I saw your post in Discord channel.
I just tell you I have experience with crawler, scrapy and Selenium and join your team.
Pls consider me.

Module definition and server

pagination and/or export to file for listing commands

Is your feature request related to a problem? Please describe

comx module list --netuid 0

Writes all the modules into the terminal which makes it impossible to scroll when there's too many modules on a subnet.

Describe the solution you'd like

Paging like in less and an option to write output to a file.

Describe alternatives you've considered

comx module list --netuid 0 | less
comx module list --netuid 0 >> module_list

Additional context

Can use a package like:
https://github.com/zaneb/autopage

Module client

Client class to interact with served commune modules.

Expand Nix definitions

Moving shared definitions (poetry2nix overrides) to the repo agicommies/nix-com
p2n.mkPoetryEnv definition
Docker image definition

Refactor CLI handling

use make_custom_context in all CLI command functions, like in save
all output and input goes through the CustomCtx interface
all informative and error outputs go to stderr, such that that relevant output can be parsed (specially on JSON output mode)
disable colors and input on non-interactive shells/scripts/etc
JSON output option for all commands

add stake metric to comx subnet info

Mining / Validating stats

comx misc my-stats

Add a feature like comx misc my-stats [--netuid] that will display your mining and validating stats.

How it works

Take all keys addresses user has on disk.
Take all models on the network (see comx module list, for implementation)

Loop through the keys, loop through the modules, if the key matches the module on the network. You display a well formatted table use Rich.

Tables

The tables will be split into:
Miners x Validators x Inactive

miner: has incentive > 0
validator: has dividends > 0
inactive: incentive == 0 && dividends == 0
If it's a validator you display the Apy, stake, emission, dividends, registration block, last update etc
Miner, you display same stats as on validator, except apy
Inactive, same as on miner

come key list and come key balance stopped working for me.

It was working fine, then it stopped minutes later.
Here is the error I receive.

│

│ /usr/lib/python3.10/json/init.py:339 in loads │
│ │
│ 336 │ │ │ │ │ │ │ │ s, 0) │
│ 337 │ else: │
│ 338 │ │ if not isinstance(s, (bytes, bytearray)): │
│ ❱ 339 │ │ │ raise TypeError(f'the JSON object must be str, bytes or bytearray, ' │
│ 340 │ │ │ │ │ │ │ f'not {s.class.name}') │
│ 341 │ │ s = s.decode(detect_encoding(s), 'surrogatepass') │
│ 342

TypeError: the JSON object must be str, bytes or bytearray, not dict

Key encryption

I'm not sure if people are using the encryption feature on the commune lib. If so, we should implement a compatible format, otherwise we can come up with something else.

Environment variable to set node URL

Setting the environment variable COMMUNE_NODE_URLS like this

export COMMUNE_NODE_URLS="wss://testnet-commune-api-node-0.communeai.net"

should make comx connect to one of the specified nodes.

The value is a list, with the purpose of load balancing / failover (which doesn't need to be implemented right now). Probably comma separated, but could be something else if more convenient to implement.

key `Account` not found

Background

Some people have reported bug when querying for the comx key balances throwing the following error message

│ │   substrate = <substrateinterface.base.SubstrateInterface object at      │ │
│ │               0xffffa240f370>                                            │ │
│

╰──────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────╯
KeyError: 'Account'

We are investigating the cause of this problem, currently seems like the node might be causing an issue.
Feedback is appreciated !

	check: lint type_check

	lint:
	ruff check ./src ./tests

	type_check:
	pyright ./src ./tests

	test_all: test test_slow

	test:
	pytest -k "not slow"

	test_slow:
	pytest -k "slow"

agicommies / communex Goto Github PK

communex's Introduction

CommuneX

Why CommuneX

Installation with pip

Installation with Nix

Features

Planned

CLI Usage

Examples

Retrieving Balance

Creating a Key

Retrieving Key Info

Listing Keys

List Keys With Balances

Retrieving Module Information

Retrieving Global Parameters

Retrieving Subnet Parameters

Retrieving Circulating Supply

Completions

Contributing

Commune compatibility

communex's People

Contributors

Stargazers

Watchers

Forkers

communex's Issues

Example of a node key structure

Describe the bug

To Reproduce

Expected behavior

Enhanced transferring features

Background

Example that Huck sent (applied to the commune repository logic):

Discussion and Feedback:

TASKS

Is your feature request related to a problem? Please describe

Describe the solution you'd like

Describe alternatives you've considered

Additional context

comx misc my-stats

How it works

Tables

Background

Recommend Projects

Recommend Topics

Recommend Org

Installation with `pip`