populus raises BytecodeMismatch about populus HOT 9 CLOSED

So, bytecode comparison is not absolutely required. I think the minimal check we do need is to ensure that the bytecode isn't empty. Given the uncertainty of what comes next for the library, if you need this to get things working 👍

from populus.

r'73[0-9a-f]{40}3014' is EVM bytecode for

PUSH20 <address>
ADDRESS
EQ

I.e., a comparison of a 20-byte constant I've labeled <address> (likely indeed an address, determined by the "constructor" at deployment time) with the actual ADDRESS in the current execution context.

I think this relates to this change introduced in solc v0.4.20:

Code Generator: Prevent non-view functions in libraries from being called
directly (as opposed to via delegatecall).

The commit in solc that introduced this change is ethereum/solidity@6807010, part of PR ethereum/solidity#3203.

from populus.

r'73[0-9a-f]{40}3014' is EVM bytecode for

Great find @veox! From my quick read through the commit that you mentioned, it makes total sense.

or it could ignore the bytes from 1 to 21 in the comparison, or something of this nature.

I was planning to use re.sub, i.e substitute with a placeholder if the pattern matches and then compare the bytecode. This way, the compare_bytecode method will be compatible with older versions of solidity too.

something similar to the exisiting swarm hash header check:

from populus.

cc @pipermerriam

from populus.

My assumption is that this is related to Swarm headers and metadata. Does Web 4.0 offer compile and related tools, so we could backport some of the utilities from there?

from populus.

@miohtama From reading bytecode-verifier's code I can see that different solidity versions have different ways of matching bytecode_runtime with the code deployed on the blockchain. Have a look at this line

My assumption is that this is related to Swarm headers and metadata

You are correct. I can verify from this code. However, looks like the newer versions have some changes which do not conform with the existing checks.

Does Web 4.0 offer compile and related tools, so we could backport some of the utilities from there?

Do you mean web3 v4.0? If you're talking about web3 than web3 doesn't have any utilities for compiling. web3 works with the assumption that abi and bytecode will be provided by the user.

from populus.

@pipermerriam I got my code working by bypassing compare_bytecode. What is the design rationale behind having a check for comparing bytecode?

from populus.

I decided to find out what was going wrong in compare_bytecode. I compiled a simple smart contract against different versions of solidity. I noticed some differences in bytecodes for solidty version < 0.4.20 and >=0.4.20. Here are some results:

for solidity version 0.4.19

In [6]: con['bin-runtime'] # expected_bytecode
Out[6]: '6060604052600080fd00a165627a7a72305820fe198797cf091375d4191836566e817de267342d7cfea15c50d3dc6ed958e91e0029'
# fast forward some steps of contract deployment
In [17]: w3.eth.getCode('0xbfBc3f7D8C0cF90E553Fc93ec95beE2d7104e7cd').hex() # deployed bytecode
Out[17]: '0x6060604052600080fd00a165627a7a72305820fe198797cf091375d4191836566e817de267342d7cfea15c50d3dc6ed958e91e0029'

for solidity version 0.4.20

In [30]: con['bin-runtime']
Out[30]: '73000000000000000000000000000000000000000030146060604052600080fd00a165627a7a72305820917938062ffe211194022a5ffe160a7e99ba4e03a77b32685732efe661b16da90029'

In [36]: w3.eth.getCode('0x6dC65c405cF4053d9Adf018FD0eb7Ad52491bDb8').hex()
Out[36]: '0x736dc65c405cf4053d9adf018fd0eb7ad52491bdb830146060604052600080fd00a165627a7a72305820917938062ffe211194022a5ffe160a7e99ba4e03a77b32685732efe661b16da90029'

if you notice carefully v0.4.20 has 46 extra hexadecimal digits compared to v0.4.19.
In fact, the first 46 digits in v0.4.20 have the following format r'73[0-9a-zA-Z]{40}3014' and
40 digits between 73 and 3014 happen to be the address at which they are deployed.

Also, the last 86 digits in both versions consist of the swarm hash. So, there's nothing wrong with the swarm hash.

To be double sure about my debugging, I added v0.4.20 and 0.4.21 to CI.
Here's the build: https://travis-ci.org/voith/populus/builds/405702207.
You can see the build pass for all versions of solidity except v0.4.20 and v0.4.22.

PS: I've checked this regex: r'73[0-9a-zA-Z]{40}3014' for several contracts, but I couldn't find much info about it in the solidity docs. However, the release notes of 0.4.20 say that significant changes have been made to optimize generated bytecode.

from populus.

This is likely to stay part of solc, so Populus' compare_bytecode could substitute the actual deployed address into the original runtime bytecode provided by solc; or it could ignore the bytes from 1 to 21 in the comparison; or something of this nature.

from populus.