algorand / go-algorand Goto Github PK

go not c.

This is a quite radical change, that was mentioned by Nikolai and Sergey in the past; instead of using the account public key to identify the account, we should use the index of the account in our database.

Given that the index is much smaller ( 4-6 bytes vs 32 byte ), the messaging sizes are expected to go down. This has trickling consequences, and would affect many aspects of our platform. ( all for good )

(originally reported by @tsachiherman)

Summary: There's a lot of upgrade code in agreement that can be removed since we are resetting mainnet. The reason why this needs to be relatively highly prioritized is because this simplifies consensus code and improves correctness.

In cmd/goal/messages.go I noticed that the error formatting strings don't follow a naming convention. Some of them prefix err, error, or suffix Error.

I'm not sure if there's the desired naming convention, which helps using hints while coding (e.g: since all errors start with err, and ide can hint existing errors). Possibly, this may apply to more places.

If that's the case, this could be tagged as an easy issue for someone to get involved in the code?

Right now there's no convenient way to dump and examine a raw block. It'd be nice if there were a command like goal ledger dumpblock 400 to dump block 400 as base64 encoded msgpack to stdout. Perhaps it could have a --json flag to translate it to json, msgpacktool-style.

This would be useful for debugging (what's going on in this block header?) and also just a fun way to see what's going on in the blocks.

(placeholder)

A fresh build currently spams the following:

2019/07/11 11:42:06 See warnings below:
2019/07/11 11:42:06 - WARNING: definition "#/definitions/APIV1RequestEnvelope" is not used anywhere
2019/07/11 11:42:06 - WARNING: definition "#/definitions/VersionsRequest" is not used anywhere
2019/07/11 11:42:06 - WARNING: definition "#/definitions/ListWalletsRequest" is not used anywhere
2019/07/11 11:42:06 - WARNING: definition "#/definitions/APIV1Request" is not used anywhere```

The warnings are benign but should be addressed/removed.

At the moment libsodium-fork is just checked into the go-algorand tree directly. This is a maintenance burden and hurts auditability. We should switch to including libsodium-fork as a git submodule.
To avoid a significant fraction of submodule terribleness, this should probably look like the following:

1. Add a go-algorand/submodules/ directory and put github.com/algorand/libsodium in go-algorand/submodules/libsodium-fork as a submodule.
2. Make go-algorand/crypto/libsodium-fork a symlink to ../submodules/libsodium-fork. This will let us switch between branches without submodules and branches with submodules without horribly confusing git.
3. Add git submodule update --init to the makefile, to be done before libsodium build

@zeldovich commented this earlier: On closer examination, not an actual problem (because we call VotingStake only after doing a Lookup, which does WithUpdatedRewards), just confusing code and comments that we should fix after v1.

Do we still want to do something about this issue? Maybe this is less of a bug and more of a clean-up task?

The agreement fuzzer currently tests pre-FPR agreement behavior and breaks when deprecating v10 (and possibly also v8?). Fix the agreement fuzzer to properly deal with FPR-enabled agreement and then deprecate the dead v8 and v10 upgrade code.

Summary: EnsureDigests’s behavior has been a source of stall bugs. At least two issues are caused by EnsureDigest (@Vervious, if you have any more details on these types of issues, please add).

EnsureDigest should only be used as a hint for faster catchup. The code when rewritten needs to confirm the right block correctly after EnsureDigest is rewritten.

(Copied)
Or some mechanism for reporting this so we can tell which of possibly many part keys are actually being used.

A single node has seen 34 of these in last 24 hrs (nobody else has). We should figure out why.

Telemetry for node around the time of the first PANIC: https://6a48e17ae50d413885db0bb7cd79265c.us-east-1.aws.found.io:9243/goto/93d8d6713bbdfe113c414744d49b4f39

(originally reported by @Karmastic)

In the Set up auction parameters @ https://github.com/algorand/go-algorand/blob/master/auction/README.md, it's mentioned Create the initial parameters file, initparams.json, from the provided template. However, the template file is missing?

(please feel free to move this issue to another repo)

The website:

https://www.algorand.com/

does not contain any links to this repository.

So, that explains possibly why I read somewhere "Algorand is NOT Open Source" - just a few days ago.

Solution

• Mention "Open Source" in a prominent position on the site.
• add a github "corner-banner"
• add a github symbol in the "follow-us" footer

Seen on travis:

node-0:
 demuxCoserviceType: 1
 pseudonodeCoserviceType: 1
 clockCoserviceType: 0
--- FAIL: TestAgreementSynchronous1 (0.23s)
panic: unexpected number of zeroes: 3 != 4 [recovered]
	panic: unexpected number of zeroes: 3 != 4 [recovered]
	panic: unexpected number of zeroes: 3 != 4
goroutine 35561 [running]:
testing.tRunner.func1(0xc001f2ae00)
	/home/travis/.gimme/versions/go1.12.linux.amd64/src/testing/testing.go:830 +0x68a
panic(0x1232640, 0xc002a83070)
	/home/travis/.gimme/versions/go1.12.linux.amd64/src/runtime/panic.go:522 +0x1b5
github.com/algorand/go-algorand/agreement.setupAgreement.func1()
	/home/travis/gopath/src/github.com/algorand/go-algorand/agreement/service_test.go:765 +0x2eb
panic(0x1232640, 0xc002a83070)
	/home/travis/.gimme/versions/go1.12.linux.amd64/src/runtime/panic.go:522 +0x1b5
github.com/algorand/go-algorand/agreement.expectNewPeriod(0xc0023a6a50, 0x1, 0x1, 0x4, 0xc0020a17c0)
	/home/travis/gopath/src/github.com/algorand/go-algorand/agreement/service_test.go:804 +0x1f8
github.com/algorand/go-algorand/agreement.runRound(0xc0023a6a50, 0x1, 0x1, 0xc0020a17c0, 0x3, 0x3)
	/home/travis/gopath/src/github.com/algorand/go-algorand/agreement/service_test.go:833 +0xa3
github.com/algorand/go-algorand/agreement.simulateAgreementWithLedgerFactory(0xc001f2ae00, 0x1, 0x5, 0x0, 0x13ff1a8)
	/home/travis/gopath/src/github.com/algorand/go-algorand/agreement/service_test.go:870 +0x28f
github.com/algorand/go-algorand/agreement.simulateAgreement(...)
	/home/travis/gopath/src/github.com/algorand/go-algorand/agreement/service_test.go:854
github.com/algorand/go-algorand/agreement.TestAgreementSynchronous1(0xc001f2ae00)
	/home/travis/gopath/src/github.com/algorand/go-algorand/agreement/service_test.go:885 +0x61
testing.tRunner(0xc001f2ae00, 0x13fed30)
	/home/travis/.gimme/versions/go1.12.linux.amd64/src/testing/testing.go:865 +0x164
created by testing.(*T).Run
	/home/travis/.gimme/versions/go1.12.linux.amd64/src/testing/testing.go:916 +0x69a
FAIL	github.com/algorand/go-algorand/agreement	100.978s
Makefile:194: recipe for target 'short_test_target_github.com/algorand/go-algorand/agreement' failed

Note that the system is inherently nondeterministic, and travis is a particularly finicky execution environment. Nonetheless we should investigate and put more effort into more deterministic tests. I cannot replicate this locally running -test.count 10000 times.

(Note: travis artifacts are currently broken. As is, we have no information to debug. We should revisit when we see another failure and have corresponding cadaver and log files to debug.)

Summary: The spec does something that does actually happen in the code. We need to fix that. Pretty simple, title says it all.

On Mac at least we need to brew install libtool, autoconf, and automake. These should be added to configure_dev.sh to simplify dev machine setup.

Is your feature request related to a problem?

No.

Describe the solution you'd like

Add ability to generate Bash terminal completions for Algorand utilities or provide generated files.

Additional context

It would be a nice time saver to not have to rely on help or developer.algorand.org as much and it would increase the usability and professionalism of the package.

How to do it

I've done this locally for goal by adding a new command to Cobra. In cmd/goal/commands.go:

rootCmd.AddCommand(completionCmd)

var completionCmd = &cobra.Command{
       Use:   "completion",
      Short: "Generates bash completion scripts for goal",
       Long: `To generate completions run 'goal completion > $HOME/scripts/goal_completion'

To configure your Bash shell to load completions for each session:
# ~/.bashrc or ~/.profile
if [ -f "$HOME/scripts/goal_completion" ]; then
    . "$HOME/scripts/goal_completion"
fi
`,
      Run: func(cmd *cobra.Command, args []string) {
              rootCmd.GenBashCompletion(os.Stdout);
      },
}

I've made the modifications to my local version and I can confirm it works. However, as I'm not sure how you'd want to provide this capability to other users I'll leave that up to you.

Summary:
Now that the pseudonode verification is asynchronous, the pseudonode should generatevotePresent/payloadPresent events, reusing the cryptoverifier logic path, instead of using another verifier.

Solution notes:
This story may require prioritizing verification of locally generated messages.

Otherwise, should comprise of piping some channels into different places and removing dead code. And comprehensive testing, since this affects consensus. An upgrade is not required.

Motivation:
remove redundancy in consensus, simplify code whenever possible

Right now goal account list shows only accounts to which you have private keys. But a user might care about participation-only accounts or accounts whose keys are in cold storage. It would be nice if there were a way to tell goal which accounts to watch. Addresses for which we've registered participation keys or have the private keys could be added to the watchlist automatically. Other addresses (such as cold storage accounts) could be added manually with some command like goal account watch <addr>

This is also motivated by scalability concerns: At some point it will no longer be practical for nodes to maintain the balances of every account in the system -- they'll need to maintain balances (and balance proofs) for the specific accounts they care about. So it may make sense to go ahead and put in place the infrastructure for letting algod keep track of which accounts to watch (rather than having goal maintain the list). This could be the AccountManager or something similar to it.

data/ledger.go
{{// The Ledger object in this (data) package provides a wrapper around the
// Ledger from the ledger package. The reason for this is compatibility
// with the existing callers of the previous ledger API, without increasing
// the complexity of the ledger.Ledger code. This Ledger object also
// implements various wrappers that return subsets of data exposed by
// ledger.Ledger, or return it in different forms, or return it for the
// latest round (as opposed to arbitrary rounds).}}

Is there some reunification/refactor possible between data/ledger.go and ledger/*.go ?

Hi,

Near the end of function toByteArray in crypto/passphrase/passphrase.go, it incorrectly checks if the current buffer value is not zero instead of checking if the number of bits (numberOfBits) is different than zero.

Regards,
Mauro.

Noticed these warnings in the logs at a rate of about 3/second. We probably don't need to keep making requests when the status is 400

{"file":"fetcher.go","function":"github.com/algorand/go-algorand/rpcs.(*NetworkFetcher).FetchBlock","level":"info","line":221,"msg":"networkFetcher.FetchBlock: asking client xxxx:4161 for block 237179","time":"2019-06-13T15:28:08.423330Z"}
{"file":"httpFetcher.go","function":"github.com/algorand/go-algorand/rpcs.(*HTTPFetcher).GetBlockBytes","level":"warning","line":88,"msg":"http block fetcher response status code : 400","time":"2019-06-13T15:28:08.451490Z"}

Subject of the issue

Some goal commands accept addr while others accept address as long-form arguments.

Your environment

• Software version: algod -v: 1.0.39516.dev [master] (commit #db5c6d56+)

• Operating System: Any

Steps to reproduce

goal account balance shows -a, --address string Account address to retrieve balance (required)
while
goal account delete shows -a, --addr string Address of account to delete

Expected behaviour

The long-form options should either both be --address or --addr.

Actual behaviour

One of the options is --address and the other is --addr.

Right now basics.Address's String and MarshalText don't include the checksum and there's a separate type basics.ChecksumAddress that includes the checksum. This is unnecessary since we never represent an address as text without including the checksum, so basics.Address should just compute the checksum in its String and MarshalText methods (and verify the checksum in its UnmarshalText method). This will also render unnecessary the duplicate code in cmd/goal/inspect.go

(copied)
It would be nice to make ConsensusParams available by API so that client libraries don't have to maintain a parallel copy to what is maintained in Go code.

/v1/consensusparams could return current version
/v1/consensusparams/{escaped version} a specific version

OR

The precise need that prompted this was wanting to see MinTxnFee in APIs related to transaction fees where suggested fee was 1 which was well below the actual MinTxnFee. So maybe we just need that.

Certain goal commands accept filenames as input and output parameters. However, they do not accept stdin or stdout, which are convenient for use in shell pipelines. For instance, goal clerk sign accepts a -i input file and a -o output file but not stdin/stdout.

Adding support to goal for accepting stdin/stdout will simplify scripting.

Summary: We need to consolidate two distinct pathways for checking txn validity

Reason for prioritization: fewer bugs in transaction verification, easier to reason about system

(GOAL2-431)

https://developer.algorand.org/docs/kmd-rest-models#SignTransactionRequest

API V1 POST TransactionSignRequest is the request for POST /v1/transaction/sign
transaction (optional) byte[] format: binary
wallet_handle_token (optional) String
wallet_password (optional) String

Do you have more detailed descriptions of parameter names and types? all optional ? thx.

I think you should expose some API that allows for generation of Merkle proofs for transactions inside a block.

Is your feature request related to a problem?

The Algorand node behaves differently depending on who the user is or how the utilities are called. In addition, upgrades revert some customizations but not others. Permissions are not consistent with other system components. Some environment setups cause unexpected behavior (e.g. inability to generate multisig transactions correctly).

Describe the solution you'd like

Follow FHS 3.0 as closely as possible:

Both algorand user and group should be created as nogroup shouldn't own any files. Running the algod daemon with its own user and group tends to be best practice nowadays for secure deployments.

Debian package example changes:

adduser --system --group --home /var/lib/algorand --no-create-home algorand >/dev/null
chown -R algorand:algorand /var/lib/algorand

Consequently, if doing so, algorand.service must change to run as Group=algorand.

In addition, for Debian package specifically, add algorand.service and system.json to conffiles as these should not be replaced during an upgrade if they are modified.

The following can be added to postinst for the reasons specified here to improve default KMD install security:

setcap cap_ipc_lock+ep /usr/bin/kmd

For non-DEB/RPM installs consider using /usr/local/algorand/bin and adding that to PATH or also /usr/local/bin to better follow FHS 3.0.

Additional context

Other important areas for discussion, some of which depend or should match with the intent of those who designed this algorand node implementation, are:

• Environment variables and their recommended use, i.e. ALGORAND_DATA and ALGORAND_KMD at the very least.
• How to ensure data consistency. For example, a single user can create a state where goal commands return different data depending on how its invoked due to .algorand location resolution (it would be helpful if this is documented better and explicitly).
• In DEB/RPM installs several goal commands have side-effects that require elevated user privileges such as those dealing with participation keys--these should be called out explicitly. This is really due to the fact that binary utilities such as goal are powerful and don't need to go through algod API-like to perform some operations. In my mind this is good, but requires deep thought as to how the underlying data is owned, accessed, and modified.

The end goal to strive for predictable and reasonable behavior such that no end user is required to refer to the source code in this repo to diagnose issues in their production systems.

Right now mergeConfigFromFile in config/config.go forces the Archival flag to true -- regardless of what's in the config file -- if the node has a listening address (i.e., acts as a relay). This makes it impossible to configure a non-archival node to listen for incoming connections.

We should instead have relays default to being archival but allow overriding this in the config file. (Alternatively, make the options completely independent)

Summary: Now that genesis hash is required, we can remove genesis id. We can perform a protocol upgrade that removes the genesis ID. This change affects what’s in transactions and reduces complexity of new transactions.

Note: There's a protocol upgrade associated with this task so we want to account for testing time

I ran a carpenter -d ~/.algorand command and got a full screen of status code of RoundInterruped. Is this a normal status code ? It was only interrupted by an "Attest" or something else similar once in a while.

goal node status

Last committed block: 0
Time since last block: 0.0s
Sync Time: 254.4s
Last consensus protocol: https://github.com/algorand/spec/tree/a26ed78ed8f834e2b9ccb6eb7d3ee9f629a6e622
Next consensus protocol: https://github.com/algorand/spec/tree/a26ed78ed8f834e2b9ccb6eb7d3ee9f629a6e622
Round for next consensus protocol: 1
Next consensus protocol supported: true
Genesis ID: mainnet-v1.0
Genesis hash:

is chang "network": "devnet", to "network": "mainnet",? thx.

-------log--------
{"file":"fetcher.go","function":"github.com/algorand/go-algorand/rpcs.(*NetworkFetcher).FetchBlock","level":"info","line":221,"msg":"networkFetcher.FetchBlock: asking client r-lu.algorand-mainnet.network:4160 for block 1","time":"2019-06-20T11:11:47.131498+08:00"}
{"Context":"sync","file":"service.go","function":"github.com/algorand/go-algorand/catchup.(*Service).fetchAndWrite","level":"warning","line":198,"msg":"fetchAndWrite(2): cert did not authenticate block (attempt 1010): unauthenticatedBundle.verify: vote {_struct:{} R:{_struct:{} Sender:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA Round:0 Period:0 Step:0 Proposal:{_struct:{} OriginalPeriod:0 OriginalProposer:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA BlockDigest:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA EncodingDigest:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA}} Cred:{_struct:{} Weight:0 VrfOut:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA DomainSeparationEnabled:false Hashable:{_struct:{} RawOut:[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] Member:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA Iter:0} UnauthenticatedCredential:{_struct:{} Proof:[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]}} Sig:{Sig:[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] PK:[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] PKSigOld:[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] PK2:[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] PK1Sig:[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] PK2Sig:[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]}} (index 1) was invalid in bundle: unauthenticatedVote.verify: could not verify FS signature on vote by EYOZMULFFZZ5QDDMWQ64HKIMUPPNEL3WJMNGAFD43L52ZXTPESBA given [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]: {_struct:{} R:{_struct:{} Sender:EYOZMULFFZZ5QDDMWQ64HKIMUPPNEL3WJMNGAFD43L52ZXTPESBA Round:2 Period:0 Step:2 Proposal:{_struct:{} OriginalPeriod:0 OriginalProposer:HHZQOGQKMQDLBEL3HXMDX7AGTNORYVZ4JFDWVSL5QLWMD3EXOIAA BlockDigest:XFQ7KBE2JDSBNE3JWLR2VA3EAAZ5T7CW7H42YHCNQML7Q5MAOBVA EncodingDigest:LYMX344RIQMQ6FEUVOPTUB2H45PIB6YMQFXDNUJQCBHS7AQFQV4A}} Cred:{_struct:{} Proof:[193 255 24 1 125 88 232 52 116 237 224 168 45 29 158 227 30 18 35 197 163 249 167 69 163 164 249 37 143 167 214 195 196 146 85 175 243 225 40 138 23 205 106 210 56 155 169 209 69 204 143 136 213 96 5 238 101 88 28 232 56 89 167 83 120 197 165 158 107 125 145 147 198 36 63 35 248 14 123 6]} Sig:{Sig:[229 223 7 250 55 218 88 247 254 31 198 152 201 238 134 19 116 231 251 0 164 254 22 201 202 113 130 202 26 204 112 203 35 102 176 212 204 230 115 197 156 165 166 117 90 108 208 155 35 146 2 111 40 35 48 94 184 74 91 196 89 81 252 2] PK:[81 95 248 132 247 164 242 180 87 108 45 149 51 57 164 249 235 31 241 233 58 61 245 166 208 129 90 216 139 38 189 236] PKSigOld:[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] PK2:[41 60 53 140 207 49 172 202 201 254 49 183 30 86 174 236 70 98 134 172 79 155 187 102 33 9 148 32 164 172 181 82] PK1Sig:[154 112 34 72 181 70 65 20 107 35 86 186 163 135 141 63 39 211 152 182 101 158 248 124 182 251 157 208 8 253 180 248 99 228 225 43 34 153 49 142 130 151 140 172 238 112 92 108 241 5 93 201 23 122 214 68 26 209 51 222 193 146 18 8] PK2Sig:[36 181 232 31 107 203 246 107 143 104 233 32 83 183 198 39 248 27 237 245 150 192 118 90 198 165 221 174 198 224 81 79 178 104 109 245 76 217 230 205 73 240 102 10 237 87 144 119 181 230 250 229 188 239 160 124 88 26 76 137 43 53 29 3]}}","name":"","time":"2019-06-20T11:11:47.382353+08:00"}
{"file":"fetcher.go","function":"github.com/algorand/go-algorand/rpcs.(*NetworkFetcher).FetchBlock","level":"info","line":221,"msg":"networkFetcher.FetchBlock: asking client r-he.algorand-mainnet.network:4160 for block 2","time":"2019-06-20T11:11:47.382611+08:00"}

Subject of the issue

We unintentionally filter votes from the pipelined round if the current period in the current round is large (>1, it seems).

Steps to reproduce

Enter recovery, period 8. Check if votes from pipelined rounds are delivered. They are not, but they should be.

Priority

Low priority, does not affect safety, nor complete loss of liveness (just a little slower on recovery)

Summary: In votes, we should sign the round, period, step, and proposal value, but not the sender. This way all votes in a bundle will include signatures of the same message. This is necessary for signature aggregation in bundles, which is a post-v1 feature, so it doesn't need to happen before launch. This could be delayed until we do signature aggregation. The change is reasonably small.

Description

The calculation precision of the sortition (function Select in data/committee/sortition/sortition.go) is not enough.

For example: when binomialP = 1e-6, vrfOutput > "0xfffffffffffffcff000000000000000000000000000000000000000000000000" , then the function Select will nearly always return a value equals the binomialN, that's the problem.

If the sortition is under an exactly precision, with the above value and when binomialN = 1e6 , then I think the correct Select result should be 18 rather than 1000000.

Consequence

With a bad precision, I think it will weakness the effects of anti-sybil-attack.

I make a test in data/committee/credential_test.go, after run several times, I got several result as follow (a line with red background means the sybils will get more votes than being honest)

The test code is as follow:

func TestSybilAttackRisk(t *testing.T) {
	n := 51
	// generate accounts
	genesis := make(map[basics.Address]*basics.AccountData)
	gen := rand.New(rand.NewSource(time.Now().Unix()))
	addrs := make([]basics.Address, n)
	secrets := make([]*crypto.SignatureSecrets, n)
	vrfSecrets := make([]*crypto.VrfPrivkey, n)
	otSecrets := make([]*crypto.OneTimeSignatureSecrets, n)
	proto := config.Consensus[protocol.ConsensusCurrentVersion]
	lookback := basics.Round(2*proto.SeedRefreshInterval + proto.SeedLookback + 1)
	total := basics.MicroAlgos{Raw:1e9}
	myStake := uint64(5e7)
	perSybilStake := myStake/uint64(n-1)
	for i := 0; i < n; i++ {
		addr, sigSec, vrfSec, otSec := newAccount(t, gen, lookback, 1000)
		addrs[i] = addr
		secrets[i] = sigSec
		vrfSecrets[i] = vrfSec
		otSecrets[i] = otSec

		short := addr
		genesis[short] = &basics.AccountData{
			Status:      basics.Online,
			MicroAlgos:  basics.MicroAlgos{Raw: perSybilStake},
			SelectionID: vrfSec.Pubkey(),
			VoteID:      otSec.OneTimeSignatureVerifier,
		}
	}
	genesis[addrs[0]].MicroAlgos.Raw = myStake

	rand.Seed(time.Now().Unix())
	var seed Seed
	rand.Read(seed[:])

	sel := AgreementSelector{
		Seed:   seed,
		Round:  basics.Round(1),
		Period: Period(1),
		Step:   Soft,
	}

	cnt, risk, rt, advantage, at := 1000, 0, 0, 0, 0
	for i := 0; i < cnt; i++ {
		addr0 := addrs[0]
		data0 := genesis[addr0]
		m := Membership{
			Record:     basics.BalanceRecord{Addr:addr0, AccountData:*data0},
			Selector:   sel,
			TotalMoney: total,
		}
		res,err := MakeCredential(vrfSecrets[0],sel).Verify(proto,m)
		var myWeight uint64
		if err == nil {
			myWeight = res.Weight
		}
		var sybilTotalWeight uint64
		for j := 1; j < n; j++ {
			addrj := addrs[j]
			dataj := genesis[addrj]
			m.Record = basics.BalanceRecord{Addr:addrj,AccountData:*dataj}
			res,err = MakeCredential(vrfSecrets[j],sel).Verify(proto,m)
			if err == nil {
				sybilTotalWeight += res.Weight
			}
		}
		at += int(myWeight)
		rt += int(sybilTotalWeight)
		if myWeight < sybilTotalWeight {
			risk++
			fmt.Printf("sybil  win: 0x%x  %d  %d\n", seed[:], myWeight, sybilTotalWeight)
		} else if myWeight > sybilTotalWeight {
			advantage++
			fmt.Printf("honest win: 0x%x  %d  %d\n", seed[:], myWeight, sybilTotalWeight)
		}
		rand.Read(seed[:])
		sel.Seed = seed
	}
	fmt.Printf("total: %d , honest win: %d, votes: %d, sybils win %d, votes: %d \n", cnt, advantage, at, risk, rt)
	if risk > advantage+advantage/20 {
		t.Errorf("potential sybil attack risk")
	}
}

Subject of the issue

If running with “shared_server": true then goal addpartkey generates a file with ownership $USER:$USER and permissions 644. Consequently node fails to restart with a 'cannot load readonly database' error.

Message in node.log:

{"file":"accountManager.go","function":"github.com/algorand/go-algorand/data.(*AccountManager).DeleteOldKeys","level":"warning","line":130,"msg":"AccountManager.DeleteOldKeys(205159): key for <address> (0-3000000): Participation.DeleteOldKeys: failed to update account: attempt to write a readonly database","time":"2019-06-22T16:58:30.314799Z"}

Partial output of ls -alF:

/var/lib/algorand/mainnet-v1.0:
drwxrwx--- 2 algorand algorand     4096 Jun 23 07:48 ./
drwxrwx--- 7 algorand algorand     4096 Jun 23 06:00 ../
-rw-r--r-- 1 <user>     <user>     3563520 Jun 23 07:48 <address>.0.3000000.partkey

Your environment

• Software version: 4294967323 1.0.27.stable [rel/stable] (commit #ee904b7b)
• Node status: Not applicable
• Operating System: Linux <hostname> 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Steps to reproduce

As user:

$ goal account addpartkey ...
$ goal account listpartkeys ...
$ algod -d $ALGORAND_DATA
$ less -n +F $ALGORAND_DATA/node.log

Expected behaviour

Key should be added, ownership changed to algorand user and permissions set to 664 or 660. With 644 permissions and ownership algorand, goal account listpartkeys fails to list the key.

Actual behaviour

See above, but in summary algod can't read key and fails to start.

Mainnet launched with v17, so all previous upgrade code should be removed. This is the continuation of #59 , which was too big to do all at once.

from comments on #102
delete the now-useless code from cmd/goal/inspect.go? In particular, you should be able to delete inspectTransaction, inspectTxnHeader, inspectPaymentTxnFields, txnToInspect, and txnFromInspect.
(See cmd/goal/inspect.go line 72)

Subject of the issue

Describe your issue here.

Your environment

• Software version: algod -v
• Node status if applicable: goal node status
• Operating System details.
  ubuntu 16.04
• In many cases log files and cadaver files are also useful to include. Since these files may be large, an Algorand developer may request them later. These files may include public addresses that you're participating with. If that is a concern please be sure to scrub that data.

Steps to reproduce

sudo add-apt-repository "deb https://releases.algorand.com/deb/ stable main"
curl -o /tmp/algorand.pub https://releases.algorand.com/key.pub
sudo apt-key add /tmp/algorand.pub
sudo apt install algorand #only this step report error all other steps run ok

Expected behaviour

Install successfully

Actual behaviour

Error reported as below:

$ sudo apt install algorand
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package algorand

AlgodClient.TransactionByID(hash)->HTTP 500 Internal Server Error: indexer isn't running, this call is disabled
AlgodClient.Block(height)->HTTP 500 Internal Server Error: indexer isn't running, this call is disabled

Summary: Next bundles which don’t have blocks can negatively impact liveness. Agreement code needs to be fixed for these bundles to contain these blocks. Since this affects liveness, resolving this will reduce stall risk.

• First task is to implement this solution (sending blocks) ~ 3 points
• Second is to QA and deploy ~ 2 points