singnet / snet-cli Goto Github PK

• Create a high level interaction with Registry to delete a service.

Currently CLI supports publishing service models as a service developer, but doesn't support pulling models as a consumer.

This part of code runs only if "dest_dir" doesn't exist.

• Create snet organization [arguments]
• User will be able to interact with the Organization's functions in a high level.
• List of available low level functions (high level [arguments]):
  • createOrganization (create)
  • changeOrganizationOwner (change-owner)
  • addOrganizationMembers (add-members)
  • removeOrganizationMembers (rem-members)
  • deleteOrganization (delete)
  • listOrganizations (list)
  • getOrganizationByName (info)
  • listServicesForOrganization (list-services)
  • listTypeRepositoriesForOrganization (list-repo)

I would propose to support standard autocompletion pattern with tab.

For example:

• complete "snet co" after pressing tab to "snet contract"
• complete "snet contract M" after pressing tab to "snet contract MultiPartyEscrow"
• etc...

It should be a way to do it very easily since we are using standard argparse library. However I coudn't find a fast way to do it....

As mentioned on Slack, the folder name model/ could cause confusion with trained NN models of a Service.

This is a request to add -y option which answers yes by default on all questions without waiting from a user prompt

We need the following functionality

1. "simple" on-chain calls to MultiPartyEscrow contract (MPE)

• We need the possibility to call the following function: deposit, withdraw, openChannel, channelExtendAndAddFunds, channelClaimTimeout (in general, all public functions, except channelClaim, because it is a special case)

2. off-chain calls by behalf of the client.

• The function to call the server using existing channel.

3. on-chain call by behalf of the "treasurer-server"
   We need to implement the protocol of close/reopen which will be called by behalf of treasurer-server.

• Send to the daemons request to stop using old channel and switch to the new one (change the nonce of the channel)
• Wait from the daemons the last state of the old channel
• Make on-chain call to close/reopen the channel (using the last state obtained from the daemons): MPE.claimChannel

see also:
https://github.com/singnet/wiki/blob/master/multiPartyEscrowContract/multiPartyEscrowApiChanges.md
with general development plan.

Hello I have tried building the snet binary from pip as well the source code but both time after installation i get the following error stack
root@aayush:~/singnet/src/github.com/singnet/snet-cli# snet Traceback (most recent call last): File "/usr/local/bin/snet", line 11, in <module> load_entry_point('snet-cli', 'console_scripts', 'snet')() File "/root/singnet/src/github.com/singnet/snet-cli/snet_cli/__init__.py", line 11, in main parser = arguments.get_root_parser(conf) File "/root/singnet/src/github.com/singnet/snet-cli/snet_cli/arguments.py", line 42, in get_root_parser add_root_options(parser, config) File "/root/singnet/src/github.com/singnet/snet-cli/snet_cli/arguments.py", line 80, in add_root_options add_contract_options(contract_p) File "/root/singnet/src/github.com/singnet/snet-cli/snet_cli/arguments.py", line 235, in add_contract_options add_contract_function_options(contract_p, contract_name) File "/root/singnet/src/github.com/singnet/snet-cli/snet_cli/arguments.py", line 415, in add_contract_function_options for fn in filter(lambda e: e["type"] == "function", contract_def["abi"]): File "/root/singnet/src/github.com/singnet/snet-cli/snet_cli/arguments.py", line 415, in <lambda> for fn in filter(lambda e: e["type"] == "function", contract_def["abi"]):
python version is 3.6.3. I am trying to setup on ubuntu 16, 64 bit.
Thanks for the help

I run the command where 'snet mpe-client call_server' where some arguments are not correct.
Something like:
snet mpe-client call_server 0x5c7a4290f6f8ff64c69eeffdfafc8644a4ec3a4e 0 10 localhost:8080 "\"Addition"\" add '{"a":10,"b":32}'

snet prints the error from the service but does not fail the process:

unspent_amount before call (None means that we cannot get it now):420000
Error: Critial Error. ""Addition"" is not an identifier
If you want to see full Traceback then run:
snet --print-traceback [parameters]

See for example CircleCI output: https://circleci.com/gh/stellarspot/platform-pipeline/91

To reproduce:

$ snet set current_agent_at 0x1e89d9ed5bcc22f934af631cda771019081e57b2
set current_agent_at 0x1e89d9ed5bcc22f934af631cda771019081e57b2

Then running snet client call [method] [json], I got this:

...
File "/usr/local/lib/python3.6/dist-packages/web3/utils/validation.py", line 123, in validate_address
    value,
web3.exceptions.InvalidAddress: ('Web3.py only accepts checksum addresses. The software that gave you this non-checksum address should be considered unsafe, please file it as a bug on their platform. Try using an ENS name instead. Or, if you must accept lower safety, use Web3.toChecksumAddress(lower_case_address).', '0x1e89d9ed5bcc22f934af631cda771019081e57b2')

But getting the address from etherscan (0x1E89D9ed5bCC22F934AF631CdA771019081E57B2), it works.

Question is...should we use this lower safety function or just warn user to change address format?

Sometimes the transaction returns this receipt:

Proceed? (y/n): y
Submitting transaction...

    event_summaries:
    -   args:
            agent: '0xFD55Dc4276E31C7F2022997A2e8F296043C26D7b'
        event: AgentCreated
    receipt_summary:
        blockHash: null
        blockNumber: null
        cumulativeGasUsed: 2878039
        gasUsed: 1788285
        transactionHash: '0x22b8c631a1e9be622d6da1187538e940b983595f15ce966331f59e57ef6f9826'

Checking:
identity.py:58-61
identity.py:86-89
identity.py:128-131
identity.py:179-182
identity.py:291-294
I saw that it just wait for something in receipt but it doesn't check the content of the fields insite it.

Suggestion:

# Wait for transaction to be mined
receipt = dict()
while not receipt:
    time.sleep(1)
    receipt = self.w3.eth.getTransactionReceipt(txn_hash)
    if receipt and "blockHash" in receipt and receipt["blockHash"] is None:
        receipt = dict()

As I understand, one can use the same identity on multiple networks, but from the view of good security practice I wouldn't want to be using my mainnet identity on kovan. It would be too easy to accidentally use real ETH instead of KETH.

Would it make sense to allow the user to specify a network when creating an identity? Then when that identity is selected, the correct network could automatically be updated for the current session.

You could still allow no network to be specified. In that case, when an identity with no network is selected, a warning could be shown. This would remind the user that they may want to change the network manually.

We had the following discussion on slack (https://snet.slack.com/messages/CCDKDJNUU/)
sergey:
(proposition 3: defining algorithm for sending transaction on "real" blockchain)
When we send transaction on real Ethereum network, even setting reasonable gas-price (from gasstation), we cannot be sure that transaction will be accepted in given amount of time, or will be accepted at all. Maybe there is an ICO now, or cryptokitties again. Actually I had such expirience few days ago playing with my ethereum account. So our automatic procedure for sending transaction might be as following:

• send transaction with reasonable gas price (probably check gasstation before)
• If transaction has not been accepted in, for example, 5 minutes, increase gas price and send it again (maybe recheck gasstation).
• Continue increasing gas price till given maximal value.
• If transaction with maximal gas price has not been accepted in 20 minutes send a warning to the owner with proposition manually increase maximal gas price.
• Wait and maybe do some predefined actions for suspending the system (on the server side)..
  Maybe there is a standard procedure already?
  cassio:
  On the final proposition: yeah, we should research possible existing solutions, but if we don’t find any we should write our own along the lines of your heuristic.

When we publish a service, we are given an agent address.

The address is used to configure the daemon.

So the daemon won't be running until after the service is published.

Thus hitting the endpoint during the publish command seems unnecessary (unless there is already a valid agent address, and publish is just updating the endpoint).

Prompt currently claims it expects the price in AGI, but it appears to actually be in 10 ** -8 AGI.

When installing the snet-cli it requires libusb and libudev as dependencies. I would be best if that is mentioned in the README file .

Valid agent address but nonexistent:

$ snet set current_agent_at 0x1234567890123456789012345678901234567890
set current_agent_at 0x1234567890123456789012345678901234567890

$ snet client call some_method '{"model": "ResNet153", "img_path": "ping"}'
Traceback (most recent call last):
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/web3-4.2.1-py3.6.egg/web3/contract.py", line 1269, in call_contract_function
    output_data = decode_abi(output_types, return_data)
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/eth_abi/abi.py", line 96, in decode_abi
    return decoder(stream)
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/eth_abi/decoding.py", line 118, in __call__
    return self.decode(stream)
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/eth_utils/functional.py", line 22, in inner
    return callback(fn(*args, **kwargs))
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/eth_abi/decoding.py", line 164, in decode
    yield decoder(stream)
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/eth_abi/decoding.py", line 118, in __call__
    return self.decode(stream)
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/eth_abi/decoding.py", line 133, in decode
    start_pos = decode_uint_256(stream)
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/eth_abi/decoding.py", line 118, in __call__
    return self.decode(stream)
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/eth_abi/decoding.py", line 186, in decode
    raw_data = self.read_data_from_stream(stream)
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/eth_abi/decoding.py", line 279, in read_data_from_stream
    len(data),
eth_abi.exceptions.InsufficientDataBytes: Tried to read 32 bytes.  Only got 0 bytes

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/bin/snet", line 11, in <module>
    load_entry_point('snet-cli==0.1.5', 'console_scripts', 'snet')()
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/snet_cli-0.1.5-py3.6.egg/snet_cli/__init__.py", line 16, in main
    getattr(args.cmd(conf, args), args.fn)()
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/snet_cli-0.1.5-py3.6.egg/snet_cli/commands.py", line 516, in call
    ident=self.ident).get_spec()
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/snet_cli-0.1.5-py3.6.egg/snet_cli/commands.py", line 442, in get_spec
    ident=self.ident).call()
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/snet_cli-0.1.5-py3.6.egg/snet_cli/commands.py", line 716, in call
    result = contract.call(self.args.contract_function, *positional_inputs, **named_inputs)
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/snet_cli-0.1.5-py3.6.egg/snet_cli/contract.py", line 8, in call
    return getattr(self.contract.functions, function_name)(*positional_inputs, **named_inputs).call()
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/web3-4.2.1-py3.6.egg/web3/contract.py", line 1031, in call
    **self.kwargs)
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/web3-4.2.1-py3.6.egg/web3/contract.py", line 1291, in call_contract_function
    raise BadFunctionCallOutput(msg) from e
web3.exceptions.BadFunctionCallOutput: Could not transact with/call contract function, is contract deployed correctly and chain synced?

Invalid address (>40 hexa):

$ snet set current_agent_at 0x123456789012345678901234567890123456789012
set current_agent_at 0x123456789012345678901234567890123456789012

$ snet client call some_method '{"model": "ResNet153", "img_path": "ping"}'
Traceback (most recent call last):
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/bin/snet", line 11, in <module>
    load_entry_point('snet-cli==0.1.5', 'console_scripts', 'snet')()
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/snet_cli-0.1.5-py3.6.egg/snet_cli/__init__.py", line 16, in main
    getattr(args.cmd(conf, args), args.fn)()
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/snet_cli-0.1.5-py3.6.egg/snet_cli/commands.py", line 516, in call
    ident=self.ident).get_spec()
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/snet_cli-0.1.5-py3.6.egg/snet_cli/commands.py", line 442, in get_spec
    ident=self.ident).call()
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/snet_cli-0.1.5-py3.6.egg/snet_cli/commands.py", line 708, in call
    contract = Contract(self.w3, contract_address, abi)
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/snet_cli-0.1.5-py3.6.egg/snet_cli/contract.py", line 4, in __init__
    self.contract = self.w3.eth.contract(address=self.w3.toChecksumAddress(address), abi=abi)
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/eth_utils/address.py", line 113, in to_checksum_address
    norm_address = to_normalized_address(address)
  File "/home/artur/workspace/artur_git/MyENVs/snet_015/lib/python3.6/site-packages/eth_utils/address.py", line 68, in to_normalized_address
    "Unknown format {}, attempted to normalize to {}".format(address, hex_address)
ValueError: Unknown format 0x123456789012345678901234567890123456789012, attempted to normalize to 0x123456789012345678901234567890123456789012

Valid address but invalid method:

$ snet set current_agent_at 0x96089B497151f01cdd193f2e787B94Fe125B9e5A
set current_agent_at 0x96089B497151f01cdd193f2e787B94Fe125B9e5A

$ snet client call some_method '{"model": "ResNet153", "img_path": "ping"}'
Retrieving service spec of 0x96089B497151f01cdd193f2e787B94Fe125B9e5A
    destination: /home/artur/.snet/service_spec/QmbjCY5X1DwHppbhEjgDzHkvK5RJqFKmBWrbfqQnRMg2BT

ERROR: failed to load service spec

I have called ./scripts/blockchain install after switching on new branch and found that npm install is not called because of precondition:

The config file is created with open read permission to anyone.

This is also where private keys are stored.

$ ls ~/.snet/config
-rw-rw-r-- 1 joel joel 483 Aug 14 16:40 /home/joel/.snet/config

Fix is to create the file with correct permissions (read permissions only available to owner), then write the ConfigParser to it, inside the persist function.

singnet/alpha-dapp#8

This doesn't make sense and there currently is this entry in the registry:

NovelSoftwareSystems/CellCouner | 0xa5c9...e9f1 | 1e-8 AGI | http://192.168.1.199:5000

Local network addresses should be limited to private ethereum nets, not published on kovan or mainnet.

The IP blocks that should be tested for are here: https://en.wikipedia.org/wiki/Private_network#Private_IPv4_addresses

Checking on one's balance is probably a common enough use case that it'd be nice to make it easy for people via a single command.

Currently the two pieces of information can be obtained by the contract command:

snet contract SingularityNetToken balanceOf MY_ADDRESS

And the python code:

from web3 import Web3, HTTPProvider
w3 = Web3(HTTPProvider("https://kovan.infura.io"))
w3.eth.getBalance(MY_ADDRESS)

Both of these return large integers, without a decimal point delimiting the whole currency amount. ETH and AGI also have different decimal point positions.

To avoid making the user think, they should by default be converted to AGI and ETH. (e.g. instead of showing 2657719035000000000, it should show 2.6577 ETH or some other number of significant figures)

I think it's very handful for an user to be able to check the installed version of snet-cli from command line.

Like this (avoiding use -v for use it with --verbose in the future):

$ snet
...
SingularityNET CLI

optional arguments:
  -h, --help         show this help message and exit
  -V, --version      show version and exit
...

CLI should have the ability to generate client stubs for calling SingularityNET AI Services in a variety of supported programming languages

Depends on #28

Steps to reproduce:

• Go to the example-service project
• Run in command line:
  snet service init
• For the choice 'Choose the path to your service's spec directory: (default: "service_spec/")' type . (current directory, this allows to resolve the provided path)
• Use snet to publish the local config
  -Run example-service with snet-daemon
• Execute snet client call classify

The error 'ERROR: failed to retrieve service spec' is not printed.

Trying the below command with snet-cli doesn't work but the dApp handles it well. The error message is posted below. The service used is network_analysis_service.

snet client call --agent-at 0x4205dA8c725AADFBaB2573015Ce2D803f321f8d5 projected_graph '{"bipartite_graph": {"bipartite_0": ["Pam", "Goeff", "Philip", "Sam", "Fred", "Jane", "Sue","Charlie"],"bipartite_1": ["American Diner", "Sushi", "Italian", "Indian", "Chinese","Tapas", "Thai","French", "Hungarian", "Lebanese", "Greek"],"edges": [["Pam", "French"], ["Pam", "Hungarian"], ["Pam", "Sushi"],["Goeff", "American Diner"],["Goeff", "Indian"], ["Goeff", "Chinese"], ["Philip", "Lebanese"],["Philip", "Italian"], ["Philip", "Tapas"], ["Sam", "American Diner"], ["Sam", "Sushi"],["Sam", "Italian"], ["Fred", "Italian"], ["Fred", "Tapas"], ["Fred", "Thai"], ["Jane", "French"], ["Jane", "Hungarian"], ["Jane", "Sushi"], ["Sue", "Greek"], ["Sue", "Tapas"], ["Sue", "Thai"], ["Charlie", "American Diner"], ["Charlie", "Indian"],["Charlie", "Chinese"]]}, "nodes": ["Pam", "Charlie", "Goeff", "Fred", "Sam", "Sue", "Philip", "Jane"], "weight": "Newman"}'

Error message below

Traceback (most recent call last):
File "/usr/local/bin/snet", line 11, in
sys.exit(main())
File "/usr/local/lib/python3.6/dist-packages/snet_cli/init.py", line 16, in main
getattr(args.cmd(conf, args), args.fn)()
File "/usr/local/lib/python3.6/dist-packages/snet_cli/commands.py", line 423, in call
params_source, params = self._get_call_params(self.args)
File "/usr/local/lib/python3.6/dist-packages/snet_cli/commands.py", line 353, in _get_call_params
elif Path(params_string).is_file():
File "/usr/lib/python3.6/pathlib.py", line 1360, in is_file
return S_ISREG(self.stat().st_mode)
File "/usr/lib/python3.6/pathlib.py", line 1156, in stat
return self._accessor.stat(self)
File "/usr/lib/python3.6/pathlib.py", line 387, in wrapped
return strfunc(str(pathobj), *args)
OSError: [Errno 36] File name too long: '{"bipartite_graph": {"bipartite_0": ["Pam", "Goeff", "Philip", "Sam", "Fred", "Jane", "Sue","Charlie"],"bipartite_1": ["American Diner", "Sushi", "Italian", "Indian", "Chinese","Tapas", "Thai","French", "Hungarian", "Lebanese", "Greek"],"edges": [["Pam", "French"], ["Pam", "Hungarian"], ["Pam", "Sushi"],["Goeff", "American Diner"],["Goeff", "Indian"], ["Goeff", "Chinese"], ["Philip", "Lebanese"],["Philip", "Italian"], ["Philip", "Tapas"], ["Sam", "American Diner"], ["Sam", "Sushi"],["Sam", "Italian"], ["Fred", "Italian"], ["Fred", "Tapas"], ["Fred", "Thai"], ["Jane", "French"], ["Jane", "Hungarian"], ["Jane", "Sushi"], ["Sue", "Greek"], ["Sue", "Tapas"], ["Sue", "Thai"], ["Charlie", "American Diner"], ["Charlie", "Indian"],["Charlie", "Chinese"]]}, "nodes": ["Pam", "Charlie", "Goeff", "Fred", "Sam", "Sue", "Philip", "Jane"], "weight": "Newman"}'

• Service with plain text instead of protobuf (.proto) file.
  • Service with ~1Mb file:
    -rw-rw-r-- 1 artur artur 1013076 Set 17 17:04 NOT_PROTO_FILE_1MB.proto
  • But not with > 1.5Mb file:
    -rw-rw-r-- 1 artur artur 1475526 Set 17 16:59 NOT_PROTO_FILE_1_5MB.proto

We should replace all "ugly" ContractCommand calls with call_contract_command or transact_contract_command functions which will be introduced with #77.
For example

agent_contract_def = get_contract_def("Agent")
        price = ContractCommand(
            config=self.config,
            args=self.get_contract_argser(
                contract_address=agent_address,
                contract_function="currentPrice",
                contract_def=agent_contract_def)(),
            out_f=None,
            err_f=None,
            w3=self.w3,
            ident=self.ident).call()

Can be replaced with

self.call_contract_command("Agent", agent_address, "currentPrice", [])

There are several design choices for how service authors interact with the CLI that I think could be improved. Publishing a service requires individual steps and keeping track of a global state which results in having to track the agent address manually. See discussion on singnet/wiki#16

This is a proposal for a substantial refactor of the interface, but to only be worked on once the core SNet behaviour has been built and stabilized. Discussion is welcome!

Local state

Instead of tracking config related to snet-cli globally, and dumping different json files in different locations, we should keep all SNet related files together.

e.g. inside a service directory, we could have a structure like:

• .snet\service.json
• .snet\daemon.json
• .snet\[other?] - should compiled protobuf files go here too?

The daemon and CLI could then read the information they need from this local context if it exists.

Motivation:

• the daemon wouldn't require a user to specify where the config file is, and the user wouldn't have to separately create it. All this information could be generated as part of creating a service.
• the current_agent_at and other current_ state variables are not a useful pattern to service authors. It's hard to remember what the current identity, network, and agent address all are set to. All of these could be stored in service.json meaning the user doesn't have to remember anything... they just have to be in the right directory.

In addition, like git, if there is no .snet directory, our tools should walk to parent directories until a .snet directory is found and use that.

Commands

Ideally, once a user has configured the CLI (with identity, network), then there should only be two core
commands. We could call these different things, but I think following git subcommand names will mean less cognitive demand of new users:

• init - create a new service in the current directory. This should also generate template proto files for the interface definition, unless they already exist. This doesn't push anything to blockchain or IPFS.
• push - this command should automagically figure out if it needs to create/modify contracts or publish changes to IPFS. So the first time you run it it will create a new agent - possibly with a warning/confirmation. Later it will only do something if you've changed the config somehow.

Beyond that, in the future I could see these to commands being useful:

• run/develop
  • First regenerate proto/grpc related code if proto files have changed.
  • Then run a local server (the command to run the user's service should be in the local config) and the SNet daemon. This removes the need for service authors to create their own scripts that launch and monitor these two processes.
  • run would provide a functional server that could be production ready (for simple services that don't have complex architecture)
  • develop would have additional debugging abilities, including hosting a webserver that would show a simple version of what the dapp provides. This would allow users to easily debug and test their UI layout locally.
• deploy - when we have SNet infrastructure, this would deploy the current service to our (or another) cloud service.

Documentation and Metadata

There is a field in metadata for a description of the service. This is nice, but
it'd be great to make it really easy to publish markdown descriptions. These could then be rendered nicely in a dapp).

Rather than putting this documentation in a json value (urgh!!), we could allow the user to place it as a markdown file inside the services .snet directory. This would automatically be bundled and published to IPFS. Down the road we might want to allow more complex documentation structures e.g. an entire directory of documentation that can be fetched by service clients and browsed locally. If we want a decentralised developer network, then this is a better option than relying on github or other external websites for decent documentation.

Summary

I know there is a lot of work before we get to this point, but I wanted to describe it here because I think aiming for a developer experience like this would make SNet really pleasant to use.

But please let me know if you think otherwise!

GitHub warned me about this vulnerability when I created a Repo for testing snet-cli.

We found a potential security vulnerability in a repository for which you have been granted security alert access.
Known high severity security vulnerability detected in pycryptodome < 3.6.6 defined in requirements.txt.
requirements.txt update suggested: pycryptodome ~> 3.6.6.
Always verify the validity and compatibility of suggestions with your codebase.

See discussion: #55 (comment)

Current snet-cli implementation allows to print both log messages and errors into std err or nothing.
It would be useful to add a debug level so it allows to print only error messages or both error and log messages.

Command line length is limited in different ways depending on one's OS.

base64 encoding of images or other binary data results in a potentially very large input string which makes it impossible to send reasonable size images on a default Windows set up (but even linux/osx have limits!).

Making a client call should allow the parameters to be read from a file, or ideally read from stdin.

I'm happy to implement this tomorrow unless people object to this approach, or have a strong desire to do it themselves :-) (I need it for testing face-services)

setup.py contains dependency 'bip32utils==0.3.post3'
Now it does not work:

> sudo pip3 install bip32utils==0.3.post3
Collecting bip32utils==0.3.post3
  Could not find a version that satisfies the requirement bip32utils==0.3.post3 (from versions: )
No matching distribution found for bip32utils==0.3.post3

> sudo pip3 install bip32utils
Collecting bip32utils
  Could not find a version that satisfies the requirement bip32utils (from versions: )
No matching distribution found for bip32utils

It seems that the git repository that contains Python api for BIP32 HD key nodes is removed:
https://github.com/prusnak/bip32utils

Integration test fails with exception:
https://circleci.com/gh/stellarspot/platform-pipeline/45
pkg_resources.ContextualVersionConflict: (eth-typing 2.0.0 (/usr/local/lib/python3.6/dist-packages), Requirement.parse('eth-typing<2.0.0,>=1.3.0'), {'eth-utils'})

dependency tree:

  - web3 [required: ==4.2.1, installed: 4.2.1]
    - eth-abi [required: >=1.1.0,<2, installed: 1.2.0]
      - eth-typing [required: <=2, installed: 2.0.0]

11 days ago eth-typing v2.0.0 was released.
There is the fix for the eth-abi dependency to use 'eth-typing<2' which has not been released yet:
ethereum/eth-abi@04f1373

To create a personal org, I need to do:

snet organization create joel MY_ADDRESS

But snet already has my identity, so I wonder if it should be sufficient to just do:

snet organization create joel

I'm also unclear who the owner of the organisation is if I specify a different address. Since I'm making the transaction, am I automatically the owner? Or is that role granted upon the initial members?

If I'm automatically the owner, then I think having an empty member list should be possible.

There are python libraries to work with dynamic loading of modules, e.g. https://stackoverflow.com/a/19393328/272238

Here we claim we can't use this approach, but it is unclear why.

Given python has a lot of tools for runtime introspection, and the risks with exec/eval I'd like to understand what the limitation is that forces us to use them.

I'm trying to publish one of the face-services.

All the face-services rely on a shared protobuf definition file face_common.proto.

When I try to publish the face detection service, I get this error:

$ snet service publish
Error: Import path must be a valid file: /home/joel/work/services/face-services/services/grpc/services/grpc/face_common.proto

This error is correct, there is no file at that location. It is duplicating the services/grpc part of the path.

However, whatever I try, I can't get it to use the correct path for the import.

I have tried:

• changing the import statement in face_detect.proto to use a relative path to the file being processed
• a relative path to the current working directory
• ensuring face_common.proto in the same directory as face_detect.proto and using no path prefix: import "face_common.proto";

What ever I try, it always adds the prefix services/grpc twice instead of once.

The reason sharing a common definition is important is that then APIs will rely on consistent primitive message types. Without this, every API is essentially independent and application developers will have to spend all their time converting between types when relying on multiple SNet services.

We should check hash of files from IPFS. Otherwise we can be attacked if ipfs client is compromised.
Function which read IPFS is here:

Depends on issue singnet/snet-daemon#99

At the moment state_service.proto is hardcoded to the snet-cli sources (see commit 01b83dbcf587d7c7ab7f7c01ba52d4b8a22dce8c). snet-cli should get this file from snet-daemon release artifact.

• Endpoints:
  • test:?/><%$endpoint
  • http://test?&^*)(.com

@astroseger as discussed yesterday, creating issues for open items

Input a list of tags for your service, space separated: (default: [])

Error: 'list' object has no attribute'read'

I assume the the default value is being provided as a list, and the parsing code is attempting to read this.

If you publish a service using a service.json and then publish it again with different name a new service will be published at the same Contract Address.

(snet_015) artur@singNET:~/Myservice$ cat service.json
{
    "name": "Service1",
    "service_spec": "service_spec/",
    "organization": "SNET_BH",
    "path": "",
    "price": 2,
    "endpoint": "http://54.203.198.53:7777",
    "tags": [
        "Tag_1,",
        "Tag_2,",
        "Tag_3"
    ],
    "metadata": {
        "description": "Testing..."
    },
    "networks": {
        "42": {
            "agentAddress": "0xc3d631F11A3cf5628d8bb52EC12EDB654e579F3c"
        }
    }
}
(snet_015) artur@singNET:~/Myservice$ snet service publish
Creating transaction to update agent contract's metadataURI from ipfs:/ipfs/QmZWchjSCqbLnbg8rMajdTsGudRP9VC4dhvUoSMSqd3nZo to ipfs:/ipfs/QmaFu326ZyEsYoe3NzghUrX9f7Ycq2fbX4cW6nFWTkKdeV...

    transaction:
        chainId: 42
        data: '...'
        from: '0xFF2a327ed1Ca40CE93F116C5d6646b56991c0ddE'
        gas: 42081
        gasPrice: 1000000000
        nonce: 2650
        to: '0xc3d631F11A3cf5628d8bb52EC12EDB654e579F3c'
        value: 0

Proceed? (y/n): y
Submitting transaction...

    event_summaries: []
    receipt_summary:
        blockHash: '0x398dcba46ec7fefd24b522283e9552994e6ff2d94864cc0cb1719d5ab547e622'
        blockNumber: 8804199
        cumulativeGasUsed: 173237
        gasUsed: 42081
        transactionHash: '0xe3e10bb674bdda400f23a43bb9498f7253b813aed2ca240a64ab1b3a32a73b75'

Creating transaction to update agent contract's endpoint from http://54.203.198.53:7000 to http://54.203.198.53:7777...

    transaction:
        chainId: 42
        data: '...'
        from: '0xFF2a327ed1Ca40CE93F116C5d6646b56991c0ddE'
        gas: 34930
        gasPrice: 1000000000
        nonce: 2651
        to: '0xc3d631F11A3cf5628d8bb52EC12EDB654e579F3c'
        value: 0

Proceed? (y/n): y
Submitting transaction...

    event_summaries: []
    receipt_summary:
        blockHash: '0xc3641b76ead393efc24e2128033cdcfaea7b60b07e55a58e998a3dfe01718ec2'
        blockNumber: 8804200
        cumulativeGasUsed: 432877
        gasUsed: 34930
        transactionHash: '0xf6aef25fc9a5a0f6c84310a96fd74a95209b3ee72ddc4963e5228deb3f140ddb'

Creating transaction to create service registration...

    transaction:
        chainId: 42
        data: '...'
        from: '0xFF2a327ed1Ca40CE93F116C5d6646b56991c0ddE'
        gas: 804713
        gasPrice: 1000000000
        nonce: 2652
        to: '0x440cF8424fcD7Fc2D2fF3a5668c919E93A3d2aAb'
        value: 0

Proceed? (y/n): y
Submitting transaction...

    event_summaries:
    -   args:
            orgName: 534e45545f424800000000000000000000000000000000000000000000000000
            orgNameIndexed: 534e45545f424800000000000000000000000000000000000000000000000000
            serviceName: '5365727669636531000000000000000000000000000000000000000000000000'
            serviceNameIndexed: '5365727669636531000000000000000000000000000000000000000000000000'
        event: ServiceCreated
    receipt_summary:
        blockHash: null
        blockNumber: null
        cumulativeGasUsed: 888134
        gasUsed: 804713
        transactionHash: '0xad0aefdaa23cf21e6c72ed4bd0b35e3d08b5073985ea3ec034d02406cfb3bd3f'

Adding contract address to session...

set current_agent_at 0xc3d631F11A3cf5628d8bb52EC12EDB654e579F3c

Service published!
(snet_015) artur@singNET:~/Myservice$ cat service.json
{
    "name": "Service2", <====== NEW NAME
    "service_spec": "service_spec/",
    "organization": "SNET_BH",
    "path": "",
    "price": 2,
    "endpoint": "http://54.203.198.53:7777",
    "tags": [
        "Tag_1,",
        "Tag_2,",
        "Tag_3"
    ],
    "metadata": {
        "description": "Testing..."
    },
    "networks": {
        "42": {
            "agentAddress": "0xc3d631F11A3cf5628d8bb52EC12EDB654e579F3c" <=== KEEP THIS
        }
    }
}
(snet_015) artur@singNET:~/Myservice$ snet service publish
Creating transaction to create service registration...

    transaction:
        chainId: 42
        data: '...'
        from: '0xFF2a327ed1Ca40CE93F116C5d6646b56991c0ddE'
        gas: 590231
        gasPrice: 1000000000
        nonce: 2653
        to: '0x440cF8424fcD7Fc2D2fF3a5668c919E93A3d2aAb'
        value: 0

Proceed? (y/n): y
Submitting transaction...

    event_summaries:
    -   args:
            orgName: 534e45545f424800000000000000000000000000000000000000000000000000
            orgNameIndexed: 534e45545f424800000000000000000000000000000000000000000000000000
            serviceName: '5365727669636532000000000000000000000000000000000000000000000000'
            serviceNameIndexed: '5365727669636532000000000000000000000000000000000000000000000000'
        event: ServiceCreated
    receipt_summary:
        blockHash: null
        blockNumber: null
        cumulativeGasUsed: 629199
        gasUsed: 590231
        transactionHash: '0x21ddd2162ebc74347fdaf23bb8d0f134099646b24bfc4cab9fba3eae95ce364b'

Adding contract address to session...

set current_agent_at 0xc3d631F11A3cf5628d8bb52EC12EDB654e579F3c

Service published!
(snet_015) artur@singNET:~/Myservice$ snet organization info SNET_BH

Owner:
 - 0xff2a327ed1ca40ce93f116c5d6646b56991c0dde

Members:
 - 0xff2a327ed1ca40ce93f116c5d6646b56991c0dde

Services:
 - BasicService
 - CNN_ImageRecon
 - CNN_ObjectDetection
 - CNN_ObjectDetection_gRPC
 - MetaService
 - Service1
 - Service2

We should alert users (with a warning message), avoiding wrong publications.

• More logs on success and error.
• Catching transactions exceptions.
• Fix and check if all possibles params are being updated.

From what I can tell, the node and npm dependency is only needed to copy the contract json files from the singularitynet-alpha-blockchain package.

Is there some tooling from node/npm that we want to use, or would a cleaner alternative be to use a python script to download the contract files from a tagged release on github?

My understanding is also that npm is only needed for snet-cli development as the contract json files get bundled in the python package. Thus most users won't need node installed and it could be removed from the documented requirements?