M2 Test output

[JB273]

Running Test M2 on Jupyter notebook 6.0.3 , output is not correct;

Test-M2: Median is correctly computed for even and odd number of available slots
enter the number of blocks you want to test: 10
current and last block numbers 11796 11817
wait until the new epoch begins at block number ... 11818
new epoch first block reached 11818 will begin testing
1iteration will begin ...
current block number 11818

KeyError Traceback (most recent call last)
in
----> 1 M2_test()

in M2_test()
31 if i['metrics'] == None:
32 continue
---> 33 by_shard_metrics = i['metrics']['by-shard']
34 slots = len(by_shard_metrics)
35 delegation = total_delegation / slots

KeyError: 'by-shard'

[ivorytowerdds]

the same issue! see #1

[ivorytowerdds]

I pass twice today. @JB273 maybe you can use the new script to test, if you also pass, then we can close this one!

[sophoah]

@ivorytowerdds @JB273 M2_test in the new script is still not working even after change.

In master repo, test_case.py M2_test uses:

        # calculate the median manually
        slot_winners = getMedianRawStakeSnapshot()['epos-slot-winners']
        stake = []
        for i in slot_winners:
            stake.append((float(i['eposed-stake'])))
        cal_median = float(get_median(stake))

i've changed that to my local copy :

        # calculate the median manually
        validator_infos = getAllValidatorInformation()
        stake = []
        for i in validator_infos:
            total_delegation = i['total-delegation']
            if i['metrics'] == None:
                continue
            by_key_metrics = i['metrics']['by-bls-key']
            slots = len(by_key_metrics)
            delegation = total_delegation / slots
            for i in range(slots):
                stake.append(delegation)
        cal_median = float(get_median(stake))

however it looks like the manual calculation vs the one returned by the network is always different:

=============== Starting M2_test ===============

INFO:economic-test:Test-M2: Median is correctly computed for even and odd number of available slots
INFO:economic-test:current and last block numbers: 34508, 34541
INFO:economic-test:current epoch: 908, begin testing...
WARNING:economic-test:Test-M2: Fail
WARNING:economic-test:calculated median: 2.6519582272405003e+24
WARNING:economic-test:rpc median: 2.5e+24

OR

=============== Starting M2_test ===============

INFO:economic-test:Test-M2: Median is correctly computed for even and odd number of available slots
INFO:economic-test:current and last block numbers: 34489, 34503
INFO:economic-test:current epoch: 907, begin testing...
WARNING:economic-test:Test-M2: Fail
WARNING:economic-test:calculated median: 2.5e+24
WARNING:economic-test:rpc median: 2.6519582272405e+24

Code looks correct after change but the number of total slot taken seems to be the issues between our manual calculation and the one coming from the network.

[ivorytowerdds]

I check the M2 script. It's what we wrote had early last week https://github.com/harmony-one/harmony-log-analysis/blob/master/notebooks/economic-test/old_version_test/M2-test.ipynb. But since it's failed sometimes, Ganesha asked me to write using the current method, use the getMedianRawStakeSnapshot. Does it fail for you? For me, it doesn't fail for about 5 tests.

I will change the script back to the getMedianRawStakeSnapshot temporarily