Changes to address issues while integrating with nwaku

[chaitanyaprem]

Started integrating the #15 branch with nwaku and noticed some issues. Also realized that we may have to change the way protocol interfaces are provided for integration. This PR covers all such changes. It is still in progress.
Integration is happening in https://github.com/waku-org/nwaku/tree/feat/mix-poc as part of waku-org/nwaku#3284

• fix imports so that importing mix into another repo doesn't lead to errors. I was getting all sorts of compilation errors as relative paths were not used in importing local files leading to conflicts when i import mix_protocol.nim or any other file.
• any interface changes needed

[AkshayaMani]

There is some overlap between the changes in this PR and this branch currently in progress. This branch also includes additional features such as:

• Destination embedded in the Sphinx Packet.
• Exit connection to destination.
• Code improvements.
• Cleaner entry abstraction (WiP) – will also include libp2p code changes (still under discussion).
  It’s possible that the Waku integration could benefit from these changes, so re-basing onto that branch at some point could be useful.

[chaitanyaprem]

There is some overlap between the changes in this PR and this branch currently in progress. This branch also includes additional features such as:

* Destination embedded in the Sphinx Packet.

* Exit connection to destination.

* Code improvements.

* Cleaner entry abstraction (WiP) – will also include libp2p code changes (still under discussion).
  It’s possible that the Waku integration could benefit from these changes, so re-basing onto that branch at some point could be useful.

awesome, Thanks!
Let me know once this branch is ready and will update waku to use this branch.

[chaitanyaprem]

@AkshayaMani are you planning to abstract out the protocol codec dependency mix has with the protocols that want to use it?
Referring to this?

mix/src/protocol.nim

Lines 9 to 15 in ba1125b

	type ProtocolType* = enum
	GossipSub12 = GossipSubCodec_12
	GossipSub11 = GossipSubCodec_11
	GossipSub10 = GossipSubCodec_10
	NoRespPing = NoRespPingCodec
	WakuLightPushProtocol = WakuLightPushCodec # TODO: need to change this to maybe register protocols to get an index or something like that.
	OtherProtocol = "other" # Placeholder for other protocols

also fixed a bug in readLP in ea38fb1
you may want to take this fix.

[AkshayaMani]

@AkshayaMani are you planning to abstract out the protocol codec dependency mix has with the protocols that want to use it? Referring to this?

mix/src/protocol.nim

Lines 9 to 15 in ba1125b

	type ProtocolType* = enum
	GossipSub12 = GossipSubCodec_12
	GossipSub11 = GossipSubCodec_11
	GossipSub10 = GossipSubCodec_10
	NoRespPing = NoRespPingCodec
	WakuLightPushProtocol = WakuLightPushCodec # TODO: need to change this to maybe register protocols to get an index or something like that.
	OtherProtocol = "other" # Placeholder for other protocols

We can't embed the protocol codec directly in the Sphinx packet, as the embedded codecs must be of fixed size to keep the packet size constant. Additionally, since these codecs are included in every message, they must not impact message size and must be bandwidth efficient. Therefore, for now, the protocol codec dependency must remain as is to meet these constraints.

also fixed a bug in readLP in ea38fb1 you may want to take this fix.

The exit abstraction (that contains the readLP function) has been removed in the other branch. The rationale for this change is that the destination no longer needs to support the mix protocol. Instead, the exit now makes direct connections to the destination protocol. This does introduce a trade-off in anonymity, as the exit can now see the message content.

[chaitanyaprem]

We can't embed the protocol codec directly in the Sphinx packet, as the embedded codecs must be of fixed size to keep the packet size constant. Additionally, since these codecs are included in every message, they must not impact message size and must be bandwidth efficient. Therefore, for now, the protocol codec dependency must remain as is to meet these constraints.

I understand that we can't embed the whole codec directly. can we instead take an approach where mix repo maintains list of protocol ids to codecs and hardcode them rather than creating a dependency like this?
i am looking at https://github.com/multiformats/multibase#multibase-table as an example where a shortcode mapping to each code is pre-defined or listed and any new base needs to raise a PR in the repo to get it included. I am wondering this way, we can avoid mix having to depend on other proto repos.
Or alternatively, we can just define the actual protocolID itself in the protocol file rather than trying to import constants which is causing the dependency.

The exit abstraction (that contains the readLP function) has been removed in the other branch. The rationale for this change is that the destination no longer needs to support the mix protocol. Instead, the exit now makes direct connections to the destination protocol. This does introduce a trade-off in anonymity, as the exit can now see the message content.

ah, got it..will rerun my poc after rebasing with these new changes.

[chaitanyaprem]

@AkshayaMani i had rebased to this branch and with some changes was able to achieve lightpush POC where exit nodes delivers message to a different lightpush node.

cc @jm-clius