Skip to content

updated libwaku API to integrate with logos chat poc for mix integration#3679

Draft
chaitanyaprem wants to merge 4 commits into
masterfrom
poc/logos-testnet-mix
Draft

updated libwaku API to integrate with logos chat poc for mix integration#3679
chaitanyaprem wants to merge 4 commits into
masterfrom
poc/logos-testnet-mix

Conversation

@chaitanyaprem

@chaitanyaprem chaitanyaprem commented Dec 24, 2025

Copy link
Copy Markdown
Contributor

Description

Changes made to get logos-testnet-0.1 AnonComms demo app ready.
This PR should not be merged. Changes which require merging are moved to #3722

  • Made some fixes/changes to the API accordingly to address them.
  • Also changed lightpush to by default use mix if mix is enabled.
  • update relay API to only take content-topic
  • test everything using logos-chat-ui app

@github-actions

github-actions Bot commented Dec 24, 2025

Copy link
Copy Markdown

You can find the image built from this PR at

quay.io/wakuorg/nwaku-pr:3679

Built from 417549e

Comment thread waku/node/kernel_api/lightpush.nim Outdated
@chaitanyaprem
chaitanyaprem force-pushed the poc/logos-testnet-mix branch 2 times, most recently from 8e6850e to 9b3af38 Compare January 21, 2026 06:21
@chaitanyaprem
chaitanyaprem marked this pull request as ready for review February 10, 2026 06:32

@jm-clius jm-clius left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On a high level this makes sense to me so far. We can continue conversation with messaging team as to whether they'd be willing to merge this or prefer us maintaining a branch for the testnet

Comment thread waku/factory/node_factory.nim
Comment thread waku/discovery/waku_ext_kademlia.nim Outdated
Comment thread waku/discovery/waku_kademlia.nim
@chaitanyaprem
chaitanyaprem force-pushed the poc/logos-testnet-mix branch 2 times, most recently from 5669cd5 to b44e51b Compare February 11, 2026 06:13
@github-actions

Copy link
Copy Markdown

This PR may contain changes to database schema of one of the drivers.

If you are introducing any changes to the schema, make sure the upgrade from the latest release to this change passes without any errors/issues.

Please make sure the label release-notes is added to make sure upgrade instructions properly highlight this change.

@chaitanyaprem
chaitanyaprem force-pushed the poc/logos-testnet-mix branch 3 times, most recently from ee9eda2 to 5c814aa Compare February 11, 2026 06:59
@chaitanyaprem

Copy link
Copy Markdown
Contributor Author

This PR may contain changes to database schema of one of the drivers.

If you are introducing any changes to the schema, make sure the upgrade from the latest release to this change passes without any errors/issues.

Please make sure the label release-notes is added to make sure upgrade instructions properly highlight this change.

@Ivansete-status i dont think i have touched any files that affect database schema.

Could it be addition of new config params that is causing this? If so, do i need to change anything else?

@NagyZoltanPeter NagyZoltanPeter left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving it with some comments added for consideration.
Also asking if it's aiming to demo Max+new kademila disco or something more permanent? In the sense we have plans to separate discovery and mix from messaging this seems to be a tight coupling here.

Comment thread library/kernel_api/protocols/lightpush_api.nim Outdated
Comment thread waku/discovery/waku_ext_kademlia.nim Outdated
Comment thread waku/discovery/waku_ext_kademlia.nim Outdated
Comment thread waku/discovery/waku_kademlia.nim
Comment thread waku/discovery/waku_kademlia.nim
Comment thread waku/discovery/waku_ext_kademlia.nim Outdated
Comment thread waku/discovery/waku_ext_kademlia.nim Outdated
Comment thread waku/discovery/waku_kademlia.nim
Comment thread waku/node/waku_node.nim Outdated
Comment thread waku.nimble
@chaitanyaprem

Copy link
Copy Markdown
Contributor Author

Also asking if it's aiming to demo Max+new kademila disco or something more permanent? In the sense we have plans to separate discovery and mix from messaging this seems to be a tight coupling here.

The aim is to demo Mix+new Kad disco. But this would be testable in real network.
Since we do have plans to separate discovery and mix from messaging, the code is little tightly coupled.

@chaitanyaprem
chaitanyaprem force-pushed the poc/logos-testnet-mix branch 2 times, most recently from a8fe3f3 to ab123e4 Compare February 12, 2026 05:25

@NagyZoltanPeter NagyZoltanPeter left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thank you!
I think at offsite we need to discuss mix with the new API....

Comment thread waku/node/kernel_api/lightpush.nim
Comment thread waku/node/waku_node.nim Outdated

@Ivansete-status Ivansete-status left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR!
I'm adding some nitpicks that I hope you find useful.
To me, the most important part is that we should keep the kernel api untouched and leverage on the new LogosDelivery API as much as possible.
Cheers

Comment thread apps/chat2mix/chat2mix.nim Outdated
Comment thread waku/node/peer_manager/waku_peer_store.nim Outdated
Comment thread waku/discovery/waku_kademlia.nim Outdated
Comment thread library/kernel_api/protocols/relay_api.nim
Comment thread library/libwaku.h
Comment thread waku/node/waku_node.nim Outdated
Comment thread waku/node/waku_node.nim
Comment thread waku/node/waku_node.nim Outdated
Comment thread waku/waku_mix/protocol.nim Outdated
Comment thread waku/waku_mix/protocol.nim

@fcecin fcecin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM and approved but please be confident you made both Ivan and Zoltan happy with this before merge.

Some thoughts on Kernel API breakage, FWIW: as we discussed in Budapest, the Kernel API is de jure "unsupported" even if de facto we do care about it and want it to be working and in nice shape whenever that's convenient and possible (until it's actually deleted at some point in the future). So actually changing it and breaking it should be expected by all its users, and although breakage is always "bad," it's also "good" here in that gets recycled back as an incentive to migrate to the spec'd Logos Message Delivery API which will be ready and stable at some point. We discussed the utility of Kernel API for internal demos and development (use cases that this PR wants to support), which is why we didn't just delete the old API. These remaining use cases need to be ready to deal with any breakage caused by changes made to accomodate each of them, on their end.

@NagyZoltanPeter NagyZoltanPeter left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Little bit of change I/we would request.
As discussed through with @Ivansete-status the change on KernelAPI relay_subscribe/unsubscribe, it is not just about the interface change, but started ring in me a warn that as such it can mislead the user of the API as you wrote correctly in the comment that it is possible the by unsubscribing from one content topic can lead to unsubscribe other content_topics that falls under the same shard as relay subscription is all about shard subscribing to gossipsub network and mesh.

As such we should not allow unsubscribe from arbitrary shards as would compromise network participation of a relay node.

WDYT?

@chaitanyaprem

Copy link
Copy Markdown
Contributor Author

I think that's a good approach! For me to understand, how exactly this subscribe/unsubscribe on relay connects to mix feature? Btw, new subscribe API will only need content topic and will manage it properly in both core and edge mode in the background.

This is the PR that i want to merge to master.

@chaitanyaprem

Copy link
Copy Markdown
Contributor Author

For me to understand, how exactly this subscribe/unsubscribe on relay

It doesn connect specifically to mix feature at all. This was more for demo app as the app only knows of conten-topic and deriving pubsubTopic from that and only subscribing to specific content-topic expecting to receive messages in the app not for all content-topics of the pubsubTopic rather only the one it is subscribed to.

I remember we did something similar in go-waku where a user can subcribe to content-topic and would be delivered only msgs from that content-topic eventhough underlying subscription is compeltly there for pubsubTopic.

@NagyZoltanPeter

NagyZoltanPeter commented Feb 13, 2026 via email

Copy link
Copy Markdown
Contributor

@chaitanyaprem

Copy link
Copy Markdown
Contributor Author

Thanks I see now. I think the most problematic part here is the unsubscribe... But what you described will be certainly covered in the new API.

On Fri, Feb 13, 2026, 07:27 Prem Chaitanya Prathi @.> wrote: chaitanyaprem left a comment (logos-messaging/logos-messaging-nim#3679) <#3679 (comment)> For me to understand, how exactly this subscribe/unsubscribe on relay It doesn connect specifically to mix feature at all. This was more for demo app as the app only knows of conten-topic and deriving pubsubTopic from that and only subscribing to specific content-topic expecting to receive messages in the app not for all content-topics of the pubsubTopic rather only the one it is subscribed to. I remember we did something similar in go-waku where a user can subcribe to content-topic and would be delivered only msgs from that content-topic eventhough underlying subscription is compeltly there for pubsubTopic. — Reply to this email directly, view it on GitHub <#3679 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/A3FU54K3WQ46TYZZTSKFIS34LVVG5AVCNFSM6AAAAACP5KSVKGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTQOJVGE2TKNBVGA . You are receiving this because your review was requested.Message ID: @.>

Yep, you will have to build and abstraction that keeps track of all content-topics subscribed and even if all of them are unsubscribed, not sure if you should/shouldn't unsubscribe from pubsubTopic. that is the only tricky decision.

@NagyZoltanPeter

NagyZoltanPeter commented Feb 13, 2026 via email

Copy link
Copy Markdown
Contributor

@chaitanyaprem

Copy link
Copy Markdown
Contributor Author

But unsure that a relay node inside and app (like status desktop) is better

personally i thing relay is a good option because participating in the network and also using it..like a tit-for-tat model. even the chat app as of now participates in relay as well as mix overlay network :)

But that needs to be evaluated based on resource availability etc.

@jm-clius

Copy link
Copy Markdown
Contributor

This PR we can shorten it only to contain kernel_api,library changes which we will not merge.
Let me know if this approach is fine.

I think you've already done this, but sounds like a good idea to me :)

@chaitanyaprem
chaitanyaprem changed the base branch from master to feat/waku-kademlia-mix February 13, 2026 12:48
Base automatically changed from feat/waku-kademlia-mix to master February 19, 2026 04:56
@NagyZoltanPeter

Copy link
Copy Markdown
Contributor

@chaitanyaprem : Let's discuss over this Kernel API change in focus of subscribe/unsubscribe.
How it is expect to work with chat+mix?
To note the main concern with it is unsubscribe, as a single content_topic unsubs will unsubscribe a complete shard which might be serving other content_topic + the node will not be able to participate in relevant pubsub meshes after it.
Normally your relay node should be auto subscribed to all supported shards so such a content topic subscription sounds controversial in relay mode. But I'm convincible though.

@chaitanyaprem

Copy link
Copy Markdown
Contributor Author

@chaitanyaprem : Let's discuss over this Kernel API change in focus of subscribe/unsubscribe. How it is expect to work with chat+mix? To note the main concern with it is unsubscribe, as a single content_topic unsubs will unsubscribe a complete shard which might be serving other content_topic + the node will not be able to participate in relevant pubsub meshes after it. Normally your relay node should be auto subscribed to all supported shards so such a content topic subscription sounds controversial in relay mode. But I'm convincible though.

The unsubscribe logic is not specific to chat+mix, but i think a general problem when you do not want to expose shards/pubsubtopics to layers above core API because autosharding is being used.
If a user is running/using relay mode the default idea is that they would be subscribed to shards/pubsubTopics by default and not based on content-topics they are interested in. This is to ensure tit-for-tat model otherwise relay protocol itself would become more skewed as no one would want to subscribe to shards that they are not interested in(to save resources). I think relay would break down.

The only thing from an application perspective i can think of is provide an abstracted subscribe/unsubscribe API over content-topics that would just provide a way for apps to register messages they are intersted to receive and not all the network traffic (kind of similar to filter but a local filter). In this case even if app unsubscribes to all content-topics as long as it is running in relay mode it would still be subscribed to underlying shards.
I hope i made it clear, let me know what do you think of this.

@NagyZoltanPeter

Copy link
Copy Markdown
Contributor

@chaitanyaprem : Let's discuss over this Kernel API change in focus of subscribe/unsubscribe. How it is expect to work with chat+mix? To note the main concern with it is unsubscribe, as a single content_topic unsubs will unsubscribe a complete shard which might be serving other content_topic + the node will not be able to participate in relevant pubsub meshes after it. Normally your relay node should be auto subscribed to all supported shards so such a content topic subscription sounds controversial in relay mode. But I'm convincible though.

The unsubscribe logic is not specific to chat+mix, but i think a general problem when you do not want to expose shards/pubsubtopics to layers above core API because autosharding is being used. If a user is running/using relay mode the default idea is that they would be subscribed to shards/pubsubTopics by default and not based on content-topics they are interested in. This is to ensure tit-for-tat model otherwise relay protocol itself would become more skewed as no one would want to subscribe to shards that they are not interested in(to save resources). I think relay would break down.

The only thing from an application perspective i can think of is provide an abstracted subscribe/unsubscribe API over content-topics that would just provide a way for apps to register messages they are intersted to receive and not all the network traffic (kind of similar to filter but a local filter). In this case even if app unsubscribes to all content-topics as long as it is running in relay mode it would still be subscribed to underlying shards. I hope i made it clear, let me know what do you think of this.

It's great, that was what I thought.
But lets divide the problem. The abstracted content_topic level of subscribe (regardless of mode if its relay or edge) is just under development by @fcecin in the new API. That new API will just work exactly as you described.|
Also the concept of being run in core/relay mode application comes with the trade off to support the network thus expected to support shards (at least this we just discussed recently with @Ivansete-status). However I'm more on the side to give this freedom of choice to app dev's hand - how much their app would like to support the network (probably it will be more clear when adding that karma thingy, and you can earn use by support).
But the problem with adding this content_topic subs/unsubs to KernelAPI's relay is that it has unexpected consequences to users - until we completely deprecate libwaku in favor of liblogosdelivery.

@chaitanyaprem

Copy link
Copy Markdown
Contributor Author

The abstracted content_topic level of subscribe (regardless of mode if its relay or edge) is just under development by @fcecin in the new API. That new API will just work exactly as you described

Perfect

Also the concept of being run in core/relay mode application comes with the trade off to support the network thus expected to support shards (at least this we just discussed recently with @Ivansete-status).

Yep, agreed.

However I'm more on the side to give this freedom of choice to app dev's hand - how much their app would like to support the network (probably it will be more clear when adding that karma thingy, and you can earn use by support).

I completely agree with this but this should just be a simple choice of using relay/edge mode and imho should not go beyond that.
not sure what is karma thingy you are referring to...are we adding some sort of user reputation?

But the problem with adding this content_topic subs/unsubs to KernelAPI's relay is that it has unexpected consequences to users - until we completely deprecate libwaku in favor of liblogosdelivery

yea, which is why i did not want to merge this and use this only for demoing the current app.
i am assuming liblogosdelivery is the more abstracted API where there are many defaults and keeping it simpler for users.

@NagyZoltanPeter

Copy link
Copy Markdown
Contributor

The abstracted content_topic level of subscribe (regardless of mode if its relay or edge) is just under development by @fcecin in the new API. That new API will just work exactly as you described

Perfect

Also the concept of being run in core/relay mode application comes with the trade off to support the network thus expected to support shards (at least this we just discussed recently with @Ivansete-status).

Yep, agreed.

However I'm more on the side to give this freedom of choice to app dev's hand - how much their app would like to support the network (probably it will be more clear when adding that karma thingy, and you can earn use by support).

I completely agree with this but this should just be a simple choice of using relay/edge mode and imho should not go beyond that. not sure what is karma thingy you are referring to...are we adding some sort of user reputation?

Yes, this karma idea was presented in Split last year and afaik it will be a kind of RLN based on reputation on our gaslass chain... sorry if mistaken :-)

WDYM should not go beyond that? What you are referring to?

But the problem with adding this content_topic subs/unsubs to KernelAPI's relay is that it has unexpected consequences to users - until we completely deprecate libwaku in favor of liblogosdelivery

yea, which is why i did not want to merge this and use this only for demoing the current app. i am assuming liblogosdelivery is the more abstracted API where there are many defaults and keeping it simpler for users.

Yes exactly it is. From user/chat perspective it's a simplified config (core/edge, preset - and good set of defaults).
Than subscribe/unsubscribe by content_topic, send by content_topic (autosharding only), receive by events, health status.
So I believe that integrating with mix we need to mostly think of configuration in conjunction of mode (core/edge) choice.

@chaitanyaprem

Copy link
Copy Markdown
Contributor Author

Yes, this karma idea was presented in Split last year and afaik it will be a kind of RLN based on reputation on our gaslass chain... sorry if mistaken :-)

ok, i am not aware of this.

WDYM should not go beyond that? What you are referring to?

well, things can be made more complex that a relay node can only subscribe to shards that node is interested int (i.e based on content topics it is interested int), but this will skew network topology to be non-uniform and would not be good.

So I believe that integrating with mix we need to mostly think of configuration in conjunction of mode (core/edge) choice.
Yep, and the idea is if user wants to use mix while publishing when in relay mode as well lightpush over mix would be used to publish. This would be agnostic to the user though. The only decision to be made is whether to expose mix at per message level for publishing or at global config level(i.e if enabled all messages published would be via mix).

@NagyZoltanPeter

Copy link
Copy Markdown
Contributor

Yes, this karma idea was presented in Split last year and afaik it will be a kind of RLN based on reputation on our gaslass chain... sorry if mistaken :-)

ok, i am not aware of this.

WDYM should not go beyond that? What you are referring to?

well, things can be made more complex that a relay node can only subscribe to shards that node is interested int (i.e based on content topics it is interested int), but this will skew network topology to be non-uniform and would not be good.

Exactly, I see the trade off is between network health and user's acceptance on higher bandwidth and traffic.
But we cannot predict ahead for each and every application that while they are using core still can afford high traffic.
Me personally would add a fleet mode to enable network node setup (we might needed sooner because of logos-core headless node runner.)

So I believe that integrating with mix we need to mostly think of configuration in conjunction of mode (core/edge) choice.
Yep, and the idea is if user wants to use mix while publishing when in relay mode as well lightpush over mix would be used to publish. This would be agnostic to the user though. The only decision to be made is whether to expose mix at per message level for publishing or at global config level(i.e if enabled all messages published would be via mix).
How should mix work through relay? That's lightpush already isn't it?
I see several options (we might discuss at offsite too).

  • drive by config - use mix -> use lightpush only for send (relay can be still used to receive)
  • enable in config but drive by message, also possible to handle internally. current send API is sending content_topic and payload (which is a MessageEnvelop (content with some attributes)) that can embed useMix flag too.

@chaitanyaprem

Copy link
Copy Markdown
Contributor Author
  • drive by config - use mix -> use lightpush only for send (relay can be still used to receive)

If we are doing this, then when mix is enabled all publishing should happen via lightpush+mix in both core and edge mode.

  • enable in config but drive by message, also possible to handle internally. current send API is sending content_topic and payload (which is a MessageEnvelop (content with some attributes)) that can embed useMix flag too.

i would personally offer this choice. note that when mix flag is enabled even in relay mode publish should happen via lightpush. In edge mode anyways lightpush is used.

@chaitanyaprem

Copy link
Copy Markdown
Contributor Author

Me personally would add a fleet mode to enable network node setup (we might needed sooner because of logos-core headless node runner.)

ok, that is interesting.

@chaitanyaprem

Copy link
Copy Markdown
Contributor Author
  • drive by config - use mix -> use lightpush only for send (relay can be still used to receive)
  • enable in config but drive by message, also possible to handle internally. current send API is sending content_topic and payload (which is a MessageEnvelop (content with some attributes)) that can embed useMix flag too.

Do also note that whatever approach we use, a size limit for msgSize has to be set that matches mix payload size which maybe smaller than network maxMsgSize. we took default size (4K but with reply header include it became 3360bytes) based on average msgSize in status app. But this msgSize or payloadSize for mix needs to be defined for the network. Too large a size would be problematic as even small msgs would end up taking all the space (because packet size is fixed in mixnets). Too small a size would lead to too much chunking. I think there is logic of chunking implemented somewhere at chat layer so maybe this can be fed to that layer (based on if msg is sent via mix or not).

@Ivansete-status Ivansete-status left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding some nitpick comments
If that PR won't be merged, shall we set it as draft ?

Comment thread library/kernel_api/protocols/filter_api.nim
).isOkOr:
# If contentTopic is provided and non-empty, use ContentSub, otherwise use PubsubSub
let subscription =
if contentTopic != nil and len($contentTopic) > 0:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick :) Applies elsewhere

Suggested change
if contentTopic != nil and len($contentTopic) > 0:
if not isNil(contentTopic) and len($contentTopic) > 0:

@chaitanyaprem
chaitanyaprem marked this pull request as draft February 20, 2026 12:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants