cessproject / cess Goto Github PK
View Code? Open in Web Editor NEWAn implementation of the distributed storage network supported by CESS LAB.
Home Page: https://www.cess.cloud/
License: Apache License 2.0
An implementation of the distributed storage network supported by CESS LAB.
Home Page: https://www.cess.cloud/
License: Apache License 2.0
Hi there, I am a cess fan and watched the last week's CESS technology presentation. The speaker has shared a lot of information about CESS how to develop on the substrate at that moment. The best point that makes me interesting is CESS created a pallet of sminer in the substrate.
Hi, while trying to start the consensus miner I'm getting the below error
:~$ sudo cess start
[+] Running 1/0
✘ Network kaleido Error 0.0s
failed to create network kaleido: Error response from daemon: Pool overlaps with other one on this address space
This is CESS version I'm using
:~$ docker network ls
Node mode: authority
Profile: testnet
nodeadm version: v0.4.4
Docker images:
config-gen: config-gen 1.4.6 20227e510d85
chain: cess-node 0.7.4-unknown 85c1d88b3bf8
kld-agent: kaleido-rotator 0.2.6-testnet af22e54a4e68
kld-sgx: kaleido 0.2.5-testnet c9f0ee9f5934
And the docker networks
:~$ sudo docker network ls
NETWORK ID NAME DRIVER SCOPE
522fcb73380e bridge bridge local
0777207ffbef host host local
07a35af1dd56 none null local
My config file
{
"node": {
"mode": "authority",
"externalIp": "45.77.47.184",
"domainName": "",
"chainWsUrl": "ws://127.0.0.1:9944",
"profile": "testnet",
"backupChainWsUrls": [
"wss://testnet-rpc0.cess.cloud/ws/",
"wss://testnet-rpc1.cess.cloud/ws/"
],
"kldSgxImageId": ""
},
"chain": {
"name": "Ginger-718599",
"port": 30336
},
"kaleido": {
"agentPort": 10010,
"kaleidoPort": 4001,
"bootPeerIds": "12D3KooWAdyc4qPWFHsxMtXvSrm7CXNFhUmKPQdoXuKQXki69qBo,12D3KooWLtpEaPbJhTyC8qoRp8PrjHy4ou7TCSY3XEtzNsh4Jatb,12D3KooWHY6BRu2MtG9SempACgYCcGHRSEai2ZkWY3E4VKDYrqh9",
"stashAccount": "cXkMzNVe1TC8ju1bUuxazHPM2jyuz8Ei2xQdXk5zWWANV7mz5",
"controllerPhrase": "december ... marine",
"allowLogCollection": true,
"bootDnsaddr": "/dnsaddr/boot-kldr-testnet.cess.cloud"
}
}
Hi, Some people are facing issues with the storage node, getting error "rpc error: code = Internal desc = the miner not exists"
Below are the logs form sudo docker logs bucket
++ 2024-04-28 20:19:38 Listening on port: 6000
++ 2024-04-28 20:19:38 RPC addresses: [wss://[testnet-rpc2.cess.cloud/ws/](http://testnet-rpc2.cess.cloud/ws/) wss://[testnet-rpc0.cess.cloud/ws/](http://testnet-rpc0.cess.cloud/ws/) wss://[testnet-rpc1.cess.cloud/ws/](http://testnet-rpc1.cess.cloud/ws/)]
++ 2024-04-28 20:19:38 Bootnodes: [_[dnsaddr.boot-bucket-testnet.cess.cloud](http://dnsaddr.boot-bucket-testnet.cess.cloud/)]
++ 2024-04-28 20:24:03 Local peer id: 12D3KooWRYSwQ38wiV7RttSj9Q6pigGbGBNhXE958StVYLi7FzZW
++ 2024-04-28 20:24:03 Chain network: cess-testnet
++ 2024-04-28 20:24:03 P2P network: cess-testnet
++ 2024-04-28 20:24:03 Number of cpu cores used: 4
++ 2024-04-28 20:24:03 RPC address used: wss://[testnet-rpc2.cess.cloud/ws/](http://testnet-rpc2.cess.cloud/ws/)
OK 2024-04-28 20:24:03 Synchronization main chain completed: 1257180
XX 2024-04-28 20:24:06 invalid miner state: not ready
++ 2024-04-28 20:24:06 Connected to 12D3KooWDWeiiqbpNGAqA5QbDTdKgTtwX8LCShWkTpcyxpRf2jA9 successfully
++ 2024-04-28 20:24:06 Connected to 12D3KooWS8a18xoBzwkmUsgGBctNo6QCr6XCpUDR946mTBBUTe83 successfully
++ 2024-04-28 20:24:06 Connected to 12D3KooWNcTWWuUWKhjTVDF1xZ38yCoHXoF4aDjnbjsNpeVwj33U successfully
++ 2024-04-28 20:24:06 Will request miner init param to [45.195.74.41:19999](http://45.195.74.41:19999/)
XX 2024-04-28 20:24:20 [RequestMinerGetNewKey] rpc error: code = Internal desc = the miner not exists
++ 2024-04-28 20:24:20 Will request miner init param to [64.176.51.140:19999](http://64.176.51.140:19999/)
XX 2024-04-28 20:24:20 [RequestMinerGetNewKey] rpc error: code = Internal desc = the miner not exists
++ 2024-04-28 20:24:20 Will request miner init param to [45.195.74.37:19999](http://45.195.74.37:19999/)
XX 2024-04-28 20:24:25 [RequestMinerGetNewKey] rpc error: code = Internal desc = the miner not exists
++ 2024-04-28 20:24:25 Will request miner init param to [211.217.153.201:19999](http://211.217.153.201:19999/)
XX 2024-04-28 20:24:25 [RequestMinerGetNewKey] rpc error: code = Internal desc = the miner not exists
++ 2024-04-28 20:24:25 Will request miner init param to [45.195.74.38:19999](http://45.195.74.38:19999/)
XX 2024-04-28 20:24:26 [RequestMinerGetNewKey] rpc error: code = Internal desc = the miner not exists
++ 2024-04-28 20:24:26 Will request miner init param to [64.176.38.24:19999](http://64.176.38.24:19999/)
XX 2024-04-28 20:24:26 [RequestMinerGetNewKey] rpc error: code = Unimplemented desc =
++ 2024-04-28 20:24:26 Will request miner init param to [45.195.74.44:19999](http://45.195.74.44:19999/)
XX 2024-04-28 20:24:32 [RequestMinerGetNewKey] rpc error: code = Internal desc = the miner not exists
++ 2024-04-28 20:24:32 Will request miner init param to [45.77.201.235:19999](http://45.77.201.235:19999/)
XX 2024-04-28 20:24:52 [RequestMinerGetNewKey] rpc error: code = Unavailable desc = connection error: desc = "error reading server preface: read tcp [161.35.96.89:48594](http://161.35.96.89:48594/)->[45.77.201.235:19999](http://45.77.201.235:19999/): use of closed network connection"
++ 2024-04-28 20:24:52 Will request miner init param to [45.195.74.35:19999](http://45.195.74.35:19999/)
XX 2024-04-28 20:24:52 [RequestMinerGetNewKey] rpc error: code = Internal desc = the miner not exists
++ 2024-04-28 20:24:52 Will request miner init param to [117.0.192.156:19999](http://117.0.192.156:19999/)
XX 2024-04-28 20:24:57 [RequestMinerGetNewKey] rpc error: code = Internal desc = the miner not exists
++ 2024-04-28 20:24:57 Will request miner init param to [108.61.179.233:19999](http://108.61.179.233:19999/)
XX 2024-04-28 20:25:17 [RequestMinerGetNewKey] rpc error: code = Unavailable desc = connection error: desc = "error reading server preface: read tcp [161.35.96.89:41454](http://161.35.96.89:41454/)->[108.61.179.233:19999](http://108.61.179.233:19999/): read: connection reset by peer"
++ 2024-04-28 20:25:17 Will request miner init param to [114.203.200.187:19999](http://114.203.200.187:19999/)
XX 2024-04-28 20:25:17 [RequestMinerGetNewKey] rpc error: code = Internal desc = the miner not exists
++ 2024-04-28 20:25:17 Will request miner init param to [148.113.1.150:19999](http://148.113.1.150:19999/)
XX 2024-04-28 20:25:18 [RequestMinerGetNewKey] rpc error: code = Internal desc = the miner not exists
++ 2024-04-28 20:25:18 Will request miner init param to [135.125.2.37:19999](http://135.125.2.37:19999/)
XX 2024-04-28 20:25:18 [RequestMinerGetNewKey] rpc error: code = Internal desc = the miner not exists
++ 2024-04-28 20:25:18 Will request miner init param to [45.195.74.42:19999](http://45.195.74.42:19999/)
XX 2024-04-28 20:25:19 [RequestMinerGetNewKey] rpc error: code = Internal desc = the miner not exists
++ 2024-04-28 20:25:19 Will request miner init param to [45.195.74.40:19999](http://45.195.74.40:19999/)
XX 2024-04-28 20:25:28 [RequestMinerGetNewKey] rpc error: code = Internal desc = the miner not exists
++ 2024-04-28 20:25:28 Will request miner init param to [45.195.74.39:19999](http://45.195.74.39:19999/)
XX 2024-04-28 20:25:30 [RequestMinerGetNewKey] rpc error: code = Internal desc = the miner not exists
++ 2024-04-28 20:25:30 Will request miner init param to [148.113.8.196:19999](http://148.113.8.196:19999/)
XX 2024-04-28 20:25:31 [RequestMinerGetNewKey] rpc error: code = Internal desc = the miner not exists
++ 2024-04-28 20:25:31 Will request miner init param to [64.176.44.135:4002](http://64.176.44.135:4002/)
XX 2024-04-28 20:25:31 [RequestMinerGetNewKey] rpc error: code = Unimplemented desc =
++ 2024-04-28 20:25:31 Will request miner init param to [45.195.74.46:19999](http://45.195.74.46:19999/)
XX 2024-04-28 20:25:32 [RequestMinerGetNewKey] rpc error: code = Internal desc = the miner not exists
++ 2024-04-28 20:25:32 Will request miner init param to [45.195.74.47:19999](http://45.195.74.47:19999/)
XX 2024-04-28 20:25:33 [RequestMinerGetNewKey] rpc error: code = Internal desc = the miner not exists
++ 2024-04-28 20:25:33 Will request miner init param to [104.238.140.193:1999](http://104.238.140.193:1999/)
I noticed that you are using Twox64Concat
by default for all your StorageMap keys. Be aware of the potential issues of choosing the incorrect hasher.
From: https://docs.substrate.io/v3/advanced/cryptography/#xxhash
Because xxHash is not cryptographically secure, it is possible that the output of the hash algorithm can be reasonably controlled by modifying the input. This can allow a user to attack this algorithm by creating key collisions, hash collisions, and imbalanced storage tries.
xxHash is used in places where outside parties cannot manipulate the input of the hash function. For example, it is used to generate the key for runtime storage values, whose inputs are controlled by the runtime developer.
For those keys that are under the user's control, Blake2 should be used instead.
E.g: File name is under user's control
https://github.com/CESSProject/cess/blob/main/c-pallets/file-bank/src/lib.rs#L258
Currently, 1M of native CESS tokens (on Substrate side) = 1 EVM CESS tokens
It is a good practice to give indexes to your runtime pallets.
More info here: https://substrate.stackexchange.com/questions/1163/whats-the-logic-behind-polkadots-choice-in-pallet-indexing
It happens that, the verification life cycle of random challenges are smaller than the computational proof cycle.
Possible solution: Adding reputation score for consensus after completing the verification task.
Action: On-going
my cess bucket report and restarting:
++ 2023-10-07 04:23:11 Rpc addresses: [ws://127.0.0.1:9944 wss://testnet-rpc0.cess.cloud/ws/ wss://testnet-rpc1.cess.cloud/ws/]
++ 2023-10-07 04:23:11 Bootnodes: [_dnsaddr.boot-kldr-testnet.cess.cloud]
XX 2023-10-07 04:23:11 [cess.New] json: cannot unmarshal number into Go struct field SysProperties.Ss58Format of type types.Bytes
I tried the following but still no solution:
cess stop
cess down
cess purge
cess profile testnet
cess config set
############################
cess status
NAMES STATUS
chain Up 47 minutes
watchtower Up 52 minutes (healthy)
bucket Restarting (1) 38 seconds ago
############################
cess bucket stat
XX 2023-10-07 04:32:30 json: cannot unmarshal number into Go struct field SysProperties.Ss58Format of type types.Bytes
############################
cess config show
{
"node": {
"mode": "storage",
"externalIp": "",
"domainName": "",
"chainWsUrl": "ws://127.0.0.1:9944",
"profile": "testnet",
"backupChainWsUrls": [
"wss://testnet-rpc0.cess.cloud/ws/",
"wss://testnet-rpc1.cess.cloud/ws/"
]
},
"chain": {
"name": "cess",
"port": 30336
},
"bucket": {
"port": 15001,
"space": 2000,
"incomeAccount": "cXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
"signPhrase": "XXXX foam edit XXXXX stumble XXXXX void XXXXX orphan XXXXX embrace XXXX",
"diskPath": "/data4/cess",
"bootAddr": "_dnsaddr.boot-kldr-testnet.cess.cloud"
}
}
############################
I have disabled ipv6,network is ok
$ ping testnet-rpc0.cess.cloud
PING testnet-rpc0.cess.cloud (104.21.59.6) 56(84) bytes of data.
64 bytes from 104.21.59.6 (104.21.59.6): icmp_seq=1 ttl=60 time=0.767 ms
64 bytes from 104.21.59.6 (104.21.59.6): icmp_seq=2 ttl=60 time=0.684 ms
^C
--- testnet-rpc0.cess.cloud ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 0.684/0.725/0.767/0.041 ms
$ ping testnet-rpc1.cess.cloud
PING testnet-rpc1.cess.cloud (172.67.167.97) 56(84) bytes of data.
64 bytes from 172.67.167.97 (172.67.167.97): icmp_seq=1 ttl=60 time=0.708 ms
64 bytes from 172.67.167.97 (172.67.167.97): icmp_seq=2 ttl=60 time=0.718 ms
############################
system info:
Ubuntu 22.04.3 LTS
5.15.0-86-generic
docker info:
Client: Docker Engine - Community
Version: 24.0.6
API version: 1.43
Go version: go1.20.7
Git commit: ed223bc
Built: Mon Sep 4 12:31:44 2023
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 24.0.6
API version: 1.43 (minimum version 1.12)
Go version: go1.20.7
Git commit: 1a79695
Built: Mon Sep 4 12:31:44 2023
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.24
GitCommit: 61f9fd88f79f081d64d6fa3bb1a0dc71ec870523
runc:
Version: 1.1.9
GitCommit: v1.1.9-0-gccaecfc
docker-init:
Version: 0.19.0
GitCommit: de40ad0
cess info:
Node mode: storage
Profile: testnet
nodeadm version: v0.4.4
Docker images:
config-gen: config-gen 1.4.6 20227e510d85
chain: cess-node 0.7.0-unknown d7bc4ab3739a
bucket: bucket v0.7.1 c2160c94f3b7
UnitPrice
value should be able to be set from genesis or have a default value. If for some reason OCW http request fails, UnitPrice
would default to 0.offchain_fetch_price()
will not be run until the next day. Some conditions should be added to automatically check if the retrieved price is under some limits or otherwise to fallback to some default value or perform the http request again. Make sure the results the OCWs generate are validated on-chain (and not assumed to be valid).update_price()
is an unsigned dispatchable. Anyone could call to this method and modify the price: https://github.com/CESSProject/cess/blob/main/c-pallets/file-bank/src/lib.rs#L603-L607I have a server, because I didn't have enough TCESS to stake 8000G of storage a few days ago, I initially staked 2000G with 8000 TCESS.
But now I've received enough TCESS, and I want to reconfigure and increase bucket.space to 8000G.
However, when I try
cess stop
cess config set
cess start
I encounter an issue.
Even though cess config show
shows that bucket.space has been increased to 8000, but the funds in my wallet haven't changed at all, 'reserved' is still at 8000
Improve benchmarks for file-bank
and replace the fixed weights
When I study the challenge process of cess, I noticed that when challenging data, data blocks are formed by dividing the fragments into smaller chunks. Why data is chllenged in forms of chunk rather than data fragments, what's the point of this design?
Avoid panics at almost any cost and add "proofs" for expect() calls on why they cannot fail. The only reason to panic should be in case a block should not be built when that code path is triggered or when it indicates a bug.
In most cases you should handle the Error.
e.g: A potential place to handle the error: https://github.com/CESSProject/cess/blob/main/c-pallets/file-bank/src/lib.rs#L537
when I run the storage mining tool to submit the proofs, only submit one proof per transaction, which will easily cause congestion and extra fees.
https://github.com/CESSProject/sbp-code-docs/blob/main/m1/cess-dscm.md
Your off-chain worker is not signing transactions anymore, it is using unsigned transactions instead.
Hello, can the team come up with a tutorial on creating an ubuntu virtual machine with hyper_v on windows and installing docker and cess programs? Because most people have limited technology, and windows is the dominant system. With this tutorial, I believe it will attract more people to participate in the cess project and make it more well-known and bigger
I have a question when I want put my database to the hard disk space, I have no idea what is the size of the hard disk space is appropriate for my database?
In Polkadot Blockchain Academy CESS sharing, I met with students with following feedback:
After deploy a flipper ink! contract in my CESS v0.6.0 local dev node. When calling a read op from testnet.cess.cloud:
A write op will be as follow:
Calling the same contract in https://cloudflare-ipfs.com/ipns/dotapps.io. Read op:
A write op:
Upgrade the Polkadot.js Apps hosted at https://testnet.cess.cloud/
Hi, I have a question about the account, how do I add my own account when I join a node in the cess chain? what is the exactly way?
I have observed that some of the pallets in the c-pallets directory are missing readme, which has caused difficulties for developers to understand the code.
If someone in the team can fill in these documents, it will be of great help to me.
Hi, recently I've noticed that when I upload small files all the time 3 miners are assigned to store the file segments. However, in case of larger files like 1GiB, only 2 miners are assigned. Why is this so?
Below is the FID of the file with 2 assigned miners.
FID: 2b15b2919269b6ee2685e492a907f6aea76fddd232966c577dac6bc959d72d8c
Without using any external solutions, the following demonstrate the default multi-sig UX in Polkadot. Is that enough for our use case?
We need to integrate pallet_multisig
and pallet_proxy
in CESS runtime, as shown in the Substrate codebase to use multi-sig feature.
Is this enough for our use case?
If not, I also found two nice solutions on creating multi-sig account in Substrate chain:
To use the above two solutions, there is probably some integration work need to be done.
Review potential overflow situations where operators +
, -
, *
and /
are used.
E.g:https://github.com/CESSProject/cess/blob/main/c-pallets/file-bank/src/lib.rs#L550
If operations are safe it is a good practice to add a comment about why.
Use checked_add
, checked_sub
, checked_mul
, checked_div
, instead.
We should integrate a EVM blockchain explorer if we really want to push developer to use our Solidity smart contract platform, so it can see the EVM contract transaction details and events emitted. Currently support of seeing EVM txs and events in Polkadot-js apps is minimal.
Ref:
The verification results of random challenges will not be cleared, in special circumstances, causing an impact.
Possible Solution: When miners are in a debt situation, the additional pledge from the debt portion will be added to the bonus poo
Status: on-going
Why is it required to open more than ten ports? It looks very unprofessional。
Isn't it possible to handle one port by adding a proxy mini service within the program? Opening so many ports is not complete
We recommend you to run
cargo +nightly fmt --all -- --check
Refer to Substrate example:
https://github.com/substrate-developer-hub/substrate-node-template/blob/main/rust-toolchain.toml
But using this rust-toolchain file causes you to download a new nightly version and recompile your code everyday. So you can fix the compiler nightly version. Example here.
The use of off-chain worker for fetching the storage price should improved taking into consideration how they work and when they are executed.
Some context about things to pay attention to when using off-chain workers:
I noticed that you reduced MILLISECS_PER_BLOCK
to 3000
. Unless there is a very strong reason to reduce this value and necessary calculations have been done to ensure network stability, I would recommend to keep the default value of 6 seconds.
More info here: https://substrate.stackexchange.com/questions/1618/what-happens-when-i-decrease-the-millisecs-per-block-parameter-in-runtime-src
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.