Add the py-dht repo as a submodule

[cortze]

Motivation

Following up with the roadmap of the das-research repo, the next step is to simulate the propagation and retrieval of BlockParts or BlockSegments into a public DHT performed by Ethereum's BeaconNodes.

Description

This PR aims to integrate the previously created py-dht module into the das-research simulator. This module has the main components needed to simulate a DHT, such as:

• a Network interface that allows nodes to find each other and establish a Connection (delays and error rates are configurable)
• a DHTClient that keeps a local RoutingTable and can look for the XOR closest nodes to any given Hash, provide a BlockSegment or Retrieve any given BlockSegment
• an essential Hash implementation that allows homogenizing the hash space for both NodeIDs and BlockSegments (currently based on pythons 64bits hash() function)

Tasks

• add py-dht as a new submodule
• make sure that the module is callable from the main DAS module

[leobago]

Do we really need a bash script to install dependencies?

[cortze]

Do we really need a bash script to install dependencies?

It is not 100% necessary, but I thought it was clearer to aggregate the submodule init and the dependencies into a single command. I can remove it if you want

[cortze]

I just squashed some of the commits to make the PR cleaner, but with the current version, the code is able to init the DHT Network defined in the smallConf.py file.

INFO : Simulator : Initializing DHTNetwork... with 512 nodes
INFO : Simulator : DHT fast-init (-1 jobs) done in 1.10 secs

[cskiraly]

@cortze example run is failing for me with the following error:
TypeError: '<' not supported between instances of 'int' and 'list'

Any ideas how a 'list' gets there?

(myenv) cskiraly@Csabas-MBP das-research % python3 study.py smallConf.py

INFO : Study : Starting simulations:
INFO : Simulator : Shape: {'run': 0, 'numberNodes': 128, 'blockSize': 64, 'failureModel': 'random', 'failureRate': 40, 'netDegree': 8, 'class1ratio': 0.8, 'chi': 2, 'vpn1': 1, 'vpn2': 500, 'bwUplinkProd': 200, 'bwUplink1': 10, 'bwUplink2': 200, 'randomSeed': 'DAS-bs-64-nn-128-fm-random-fr-40-c1r-0.8-chi-2-vpn1-1-vpn2-500-bwupprod-200-bwup1-10-bwup2-200-nd-8dht-seed-builder-seeding-segments-k-20-alpha-1-r-0', 'dhtSeeding': 'builder-seeding-segments', 'k': 20, 'alpha': 1} ... Block Available: 1 in 9 steps
INFO : Simulator : Shape: {'run': 0, 'numberNodes': 128, 'blockSize': 64, 'failureModel': 'random', 'failureRate': 40, 'netDegree': 8, 'class1ratio': 0.8, 'chi': 2, 'vpn1': 1, 'vpn2': 500, 'bwUplinkProd': 200, 'bwUplink1': 10, 'bwUplink2': 200, 'randomSeed': 'DAS-bs-64-nn-128-fm-random-fr-40-c1r-0.8-chi-2-vpn1-1-vpn2-500-bwupprod-200-bwup1-10-bwup2-200-nd-8dht-seed-builder-seeding-segments-k-20-alpha-1-r-0', 'dhtSeeding': 'builder-seeding-segments', 'k': 20, 'alpha': 1} ... Setting up DHT Network
INFO : Simulator : Initializing DHTNetwork... with 128 nodes
INFO : Simulator : DHT fast-init (-1 jobs) done in 1.39 secs
INFO : Simulator : Seeding DHT with 'builder-seeding-segments' strategy
INFO : Simulator : Node 0 will start providing the block to the DHT!
joblib.externals.loky.process_executor._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/Users/cskiraly/dagger/DAS/das-research/myenv/lib/python3.11/site-packages/joblib/externals/loky/process_executor.py", line 428, in _process_worker
    r = call_item()
        ^^^^^^^^^^^
  File "/Users/cskiraly/dagger/DAS/das-research/myenv/lib/python3.11/site-packages/joblib/externals/loky/process_executor.py", line 275, in __call__
    return self.fn(*self.args, **self.kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/cskiraly/dagger/DAS/das-research/myenv/lib/python3.11/site-packages/joblib/_parallel_backends.py", line 620, in __call__
    return self.func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/cskiraly/dagger/DAS/das-research/myenv/lib/python3.11/site-packages/joblib/parallel.py", line 288, in __call__
    return [func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/cskiraly/dagger/DAS/das-research/myenv/lib/python3.11/site-packages/joblib/parallel.py", line 288, in <listcomp>
    return [func(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/cskiraly/dagger/DAS/das-research/study.py", line 41, in runOnce
    sim.runBlockPublicationToDHT(shape.dhtSeeding)
  File "/Users/cskiraly/dagger/DAS/das-research/DAS/simulator.py", line 336, in runBlockPublicationToDHT
    self.dhtBlockProposerSeedingDHTwithSegments()
  File "/Users/cskiraly/dagger/DAS/das-research/DAS/simulator.py", line 370, in dhtBlockProposerSeedingDHTwithSegments
    nodes, _, summary, aggrDelay = blockProposer.lookup_for_hash(segmentHash)
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/cskiraly/dagger/DAS/das-research/py-dht/dht/dht.py", line 89, in lookup_for_hash
    connection, conndelay = self.network.connect_to_node(self.ID, node, origin_overhead, remote_overhead)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/cskiraly/dagger/DAS/das-research/py-dht/dht/dht.py", line 465, in connect_to_node
    if random.randint(0, 99) < self.slowerrorrate:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: '<' not supported between instances of 'int' and 'list'
"""

[cortze]

@cortze example run is failing for me with the following error: TypeError: '<' not supported between instances of 'int' and 'list'

Any ideas how a 'list' gets there?

That seems to be my bad. There is a mismatch between the py-dht submodule version for the DHT folder and the DAS one. Looks like a broken interface due to an update.

I generally used the DHT module as a standalone DHT simulation, as it doesn't require to be executed right after the DAS simulation.

I can fix the interface and the smallConf.py configuration, or I could directly remove it if you prefer. I remember that @leobago 's take here was to leave it as part of DAS, but I'm happy to do any. of them.