Structs return as lists

[fubuloubu]

• Version: 4.3.0
• Python: 3.6
• OS: linux

What was wrong?

I have a contract with a mapping and a struct with public accessors:

pragma solidity ^0.4.24;
contract A {
  struct MyStruct {
    uint a;
    uint b;
  }
  MyStruct public myStruct;
  mapping (address => MyStruct) public myMapping;
}

However, in web3py, it returns lists instead of structs:

from web3 import Web3, EthereumTesterProvider
w3 = Web3(EthereumTesterProvider())
contract_data = {
  'bytecode' : '0x608060405234801561001057600080fd5b50610143806100206000396000f30060806040526004361061004c576000357c0100000000000000000000000000000000000000000000000000000000900463ffffffff1680637c07d4f1146100515780638bc9fe6c146100af575b600080fd5b34801561005d57600080fd5b50610092600480360381019080803573ffffffffffffffffffffffffffffffffffffffff1690602001909291905050506100e1565b604051808381526020018281526020019250505060405180910390f35b3480156100bb57600080fd5b506100c4610105565b604051808381526020018281526020019250505060405180910390f35b60026020528060005260406000206000915090508060000154908060010154905082565b600080600001549080600101549050825600a165627a7a72305820c9c5dcef2ed03794dc04e36476afbb4cfb28b05e167806d4fdfb32e0e66a44540029',
  'bytecode_runtime' : '0x60806040526004361061004c576000357c0100000000000000000000000000000000000000000000000000000000900463ffffffff1680637c07d4f1146100515780638bc9fe6c146100af575b600080fd5b34801561005d57600080fd5b50610092600480360381019080803573ffffffffffffffffffffffffffffffffffffffff1690602001909291905050506100e1565b604051808381526020018281526020019250505060405180910390f35b3480156100bb57600080fd5b506100c4610105565b604051808381526020018281526020019250505060405180910390f35b60026020528060005260406000206000915090508060000154908060010154905082565b600080600001549080600101549050825600a165627a7a72305820c9c5dcef2ed03794dc04e36476afbb4cfb28b05e167806d4fdfb32e0e66a44540029',
  'abi' : [{"constant":True,"inputs":[{"name":"","type":"address"}],"name":"myMapping","outputs":[{"name":"a","type":"uint256"},{"name":"b","type":"uint256"}],"payable":False,"stateMutability":"view","type":"function"},{"constant":True,"inputs":[],"name":"myStruct","outputs":[{"name":"a","type":"uint256"},{"name":"b","type":"uint256"}],"payable":False,"stateMutability":"view","type":"function"}]
}
tx_hash = w3.eth.contract(**contract_data).constructor().transact()
tx_receipt = w3.eth.waitForTransactionReceipt(tx_hash)
contract = w3.eth.contract(address=tx_receipt.contractAddress, abi=contract_data['abi'])
contract.functions.myMapping("0x0").call()
[0, 0]
contract.functions.myStruct().call()
[0, 0]

It would be nicer if this returned a MyStruct struct object, or some dict with those keys, or something more explicit than a list.

How can it be fixed?

There is no direct way to access MyStruct's layout from the ABI, but typically it apppears similar to a tuple and always has the members annotated in the output args of the getter. There should be some way to cast that to a struct object or dict so it's a little more intuitive for developers.

[pipermerriam]

I think that collections.namedtuple is what we need to use for this.

• maintains field ordering (required for using output of one call as the input of another)
• allows attribute access.

We could technically used ordered dicts, but I think that would be a mistake as it would be very easy for an ordered dict to get downcast to a normal dict and then for ambiguous field ordering to cause some sort of bug.

[fubuloubu]

Awesome, that definitely looks like the right way to handle it.

How are we going to create the names for the tuples?

>>> from collections import namedtuple
>>> namedtuple('MyStruct', ['a', 'b'])
__main__.MyStruct

Struct names aren't available in the ABI, so will probably have to compose random names for them.

[pipermerriam]

Solidity provides names in the ABI of structs (though they aren't necessarily required).

[fubuloubu]

Names of the members, not names of the struct itself. You need to provide both (or come up with your own) to use namedtuple. My point above is that a and b are given in both outputs, but MyStruct is not available in the ABI

[fubuloubu]

Another subtlety is that the way Solidity works, it's not really possible to tell the difference between a struct being returned, and an annotated tuple.

This contract:

pragma solidity ^0.4.24;
contract A {
  struct MyStruct {
    uint a;
    uint b;
  }
  MyStruct public myStruct;
  mapping (address => MyStruct) public myMapping;
  
  function myFunction() public pure returns (uint a, uint b) {
      return (0, 1);
  }
}

has the ABI:

[{"constant":true,"inputs":[{"name":"","type":"address"}],"name":"myMapping","outputs":[{"name":"a","type":"uint256"},{"name":"b","type":"uint256"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":true,"inputs":[],"name":"myStruct","outputs":[{"name":"a","type":"uint256"},{"name":"b","type":"uint256"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":true,"inputs":[],"name":"myFunction","outputs":[{"name":"a","type":"uint256"},{"name":"b","type":"uint256"}],"payable":false,"stateMutability":"pure","type":"function"}]

which all three of myStruct(), myMapping(address) and myFunction() have a return type with outputs [{"name":"a","type":"uint256"},{"name":"b","type":"uint256"}], so this solution will have to be a bit more generic than assuming that return type is always a struct

Vyper doesn't allow naming returns, so a struct will always have named members, and a tuple will be all out:

myStruct: public({
    a: uint256,
    b: uint256
})
myMapping: public({
    a: uint256,
    b: uint256
}[address])

@public
@constant
def myFunction() -> (uint256, uint256):
    return (0, 1)

has ABI:

[{"name": "myFunction", "outputs": [{"type": "uint256", "name": "out"}, {"type": "uint256", "name": "out"}], "inputs": [], "constant": true, "payable": false, "type": "function", "gas": 355}, {"name": "myStruct__a", "outputs": [{"type": "uint256", "name": "out"}], "inputs": [], "constant": true, "payable": false, "type": "function", "gas": 585}, {"name": "myStruct__b", "outputs": [{"type": "uint256", "name": "out"}], "inputs": [], "constant": true, "payable": false, "type": "function", "gas": 621}, {"name": "myMapping__a", "outputs": [{"type": "uint256", "name": "out"}], "inputs": [{"type": "address", "name": "arg0"}], "constant": true, "payable": false, "type": "function", "gas": 817}, {"name": "myMapping__b", "outputs": [{"type": "uint256", "name": "out"}], "inputs": [{"type": "address", "name": "arg0"}], "constant": true, "payable": false, "type": "function", "gas": 853}]

which is even weirder, although part of this is buggy behavior IMO. See vyperlang/vyper#932

[pipermerriam]

Structs map to the ABI tuple type which I would advocate for vyper to introduce support for at some point. We can probably talk with the solidity folks to add the name of the struct as extra semantic information to the tuple type. I think in the absense of this name we can use something generic like Tuple or StructTuple.

[fubuloubu]

I think a better convention may be that structs are ABI tuples with named members, and regular tuples are ABI tuples with numbered members
E.g.

{
"struct_output":[{"a":"uint256"},{"b":"uint256"}],
"tuple_output":[{"0":"uint256"},{"1":"uint256"}]
}