Skip to content

Commit edabcf5

Browse files
igor-sirotinclaude
andauthored
refactor(messaging): lift rfc26 decoding to Transport (#7524)
* feat(messaging): add neutral ReceivedMessage push seam on the waku adapter Adds the receive half of the logos-delivery-shaped MessagingAPI: a reliable push channel of neutral ReceivedMessages (mirroring the logos-delivery MessageReceived event), published from processMessage in parallel with the existing NotifyWatchers decode/match path. Nothing consumes it yet, so reception behaviour is unchanged; the transport-side decode/match handler follows. - types.ReceivedMessage: neutral, send-symmetric shape {Hash, ContentTopic, Payload, Ephemeral, Meta} plus transitional Version/Timestamp (status-go#7522). - types.Waku / transport.MessagingAPI gain Subscribe/UnsubscribeMessageReceived. - *Waku implements them with a single reliable buffered channel (blocking with backpressure, ctx-guarded, no-op until subscribed). Refs #7464 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(messaging): move receive decode/route/match into the transport The transport now owns the reception pipeline: a receiveLoop consumes the adapter's neutral ReceivedMessage push channel, routes each message by content topic to the matching filters, decodes it via rfc26 using the filter's key material, and buffers the decoded result. RetrieveRawAll drains that buffer (instead of polling waku.GetFilterMessages) and dedups against the persistent processed-message cache. The matched-tickle (SubscribeFilterMatched) now originates in the transport. The waku adapter's processMessage stops decoding/matching/buffering and just forwards every received envelope; the transport's processed-message cache is the single de-duplication authority (keyed by message hash), so the adapter marks nothing processed — preserving store re-fetch for late-installed filters. - FiltersManager: WatchersByContentTopic + a runtime-only decodeKeys map (sym/asym key material captured at subscribe time; never on the persisted Filter), cleaned on Remove/RemoveNoListenFilters. - transport.decode mirrors transport.encode (version==0 passthrough); signature/sender match collapses since common.Filter.Src was always nil. - New TestReceivePushPath covers route -> decode -> drain end to end. The adapter-side decode/match/buffer (NotifyWatchers, Open, MemoryMessageStore, GetFilterMessages) is now dead and removed in the next commit. Refs #7464 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(messaging): remove the dead waku-side reception pipeline With decode/route/match now in the transport, delete the adapter-side reception machinery and the wrong-direction dependency it carried: - waku/common: drop Open (and its transport/rfc26 import — the wrong-direction dependency the staged refactor was working off), NotifyWatchers, MatchMessage, the per-filter MemoryMessageStore + Trigger/Retrieve, the topic-match index, and the matchedPublisher. common.Filter is now just a wire-subscription record. - waku adapter: drop GetFilterMessages, ToWakuMessage, MarkP2PMessageAsProcessed, the SubscribeFilterMatched delegation, the write-only storeMsgIDs state, and the per-filter buffer wiring in Subscribe/Clean. - Trim types.Waku and transport.MessagingAPI accordingly; drop the now-unused MarkP2PMessageAsProcessed facade method and its messenger call (it only deleted from the dead storeMsgIDs map; dedup is the transport's processed-message cache). - Port the waku/common filter tests to the registry-only shape; the decode/match coverage now lives in the transport (TestReceivePushPath). make statusgo ✅ Refs #7464 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(messaging): per-candidate decode, pubsub routing, reuse sym keys Addresses review feedback on the receive pipeline: - Routing keys on (pubsubTopic, contentTopic) again, not content topic alone. PubsubTopic is carried on the neutral ReceivedMessage (transitional, #7522) and a filter with no explicit pubsub topic is normalized to the default shard, so the waku layer's routing is preserved. - Decode per candidate instead of "decode once, reuse". Content topics are a 4-byte hash of the chat id, so distinct chats can collide on one content topic with different keys; each candidate is now decoded with its own key and a successful authenticated decrypt is what confirms the match (this also replaces the old per-filter key-hash check). - Drop the redundant symmetric entries from the runtime key map: symmetric keys are resolved on demand via keysManager.RawSymKey(SymKeyID) (the same path the send side uses). Only asymmetric private keys — not reachable from a persisted Filter — are kept in the runtime map (renamed decodeKeys -> asymKeys). TestReceivePushPath gains a wrong-key case (message on the right topic but the wrong key is dropped). make statusgo / make lint clean. Refs #7464 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(messaging): deliver received messages over the envelope feed Replace the bespoke SubscribeMessageReceived channel with the existing envelope event feed, addressing review feedback: - The waku adapter no longer exposes a dedicated received-message channel. Its processMessage attaches the neutral *types.ReceivedMessage as the Data of the existing EventEnvelopeAvailable envelope event. MessagingAPI now exposes the generic SubscribeEnvelopeEvents (one event seam, the shape logos-delivery uses) instead of a per-event method; the transport's receiveLoop consumes it and decodes/routes the available events. - This fixes reception in unit tests, where a single waku instance is shared by multiple cores: the previous single-consumer channel let one core steal another core's messages. The envelope feed is a broadcast, so every core's transport sees every event and routes it against its own filters (matching the old NotifyWatchers fan-out). - Drops the "reliable channel" wording, which collided with logos-delivery's unrelated ReliableChannelsAPI. make statusgo / make lint clean; protocol MessengerSuite reception tests (own/their public, their private 1:1, group, non-existing chats) pass. Refs #7464 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(communities): wait for bob to learn alice is a poster before she posts TestAnnouncementsChannelPermissions relied on bob receiving the community description that promotes alice to poster before he received alice's channel message. Reception order is not guaranteed (Waku), so when bob processed the message first he rejected it ("user can't post in community") against stale community state and never retried it — the known, deferred limitation in #3869. Make the test deterministic: after the owner reevaluates members, wait for bob to apply the change (alice == poster, 2 channel members) before alice posts, so ordering no longer decides the outcome. This is a test-only change; #3869 itself is unchanged. Refs #7464, #3869 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(profile-showcase): wait for showcased communities before asserting TestProfileShowcaseProofOfMembershipEncryptedCommunity asserted on the first profile-showcase update for alice's contact. The showcased communities are validated and populated asynchronously (membership proof for the encrypted community arrives separately), so under CI timing the entries could be absent when asserted — the showcase came back empty and the test failed with "[]" should have 2 item(s), but has 0. Wait until bob's stored showcase for alice actually has both communities before asserting their data, instead of on the first showcase update. Test-only change; reception order no longer decides the outcome (see #3869). Refs #7464, #3869 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(messaging): dedup received messages by hash within a drain The push receive buffer was a per-filter slice, so the same envelope pushed more than once before RetrieveRawAll drained it was returned (and processed) multiple times — the persistent processed-message cache only dedups across drains, not within one. The waku adapter's old per-filter store was a hash-keyed map and collapsed these duplicates on insert. Restore that invariant by keying the buffer on message hash. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(push-notification): re-advertise so alice learns both paired devices Each paired device advertises only its own installation's push info on the shared contact-code topic, and processing one such advertisement stamps a query timestamp that suppresses the authoritative server query for 24h. Under the new push receive path alice processes the later device's advertisement before she sends her message (on the old poll path she queried first), so she permanently sees only one installation. Re-advertise from both devices inside the retry loop, after alice is subscribed to the contact-code topic, so she receives both. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent b11c24c commit edabcf5

17 files changed

Lines changed: 485 additions & 675 deletions

pkg/messaging/api_processor.go

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,6 @@ package messaging
33
import (
44
"crypto/ecdsa"
55

6-
ethcommon "github.com/ethereum/go-ethereum/common"
7-
86
"github.com/status-im/status-go/internal/connection"
97
adapters "github.com/status-im/status-go/pkg/messaging/adapters"
108
types "github.com/status-im/status-go/pkg/messaging/types"
@@ -38,10 +36,6 @@ func (a *API) ClearProcessedMessageIDsCache() error {
3836
return a.core.stack.Transport.ClearProcessedMessageIDsCache()
3937
}
4038

41-
func (a *API) MarkP2PMessageAsProcessed(hash ethcommon.Hash) {
42-
a.core.stack.Transport.MarkP2PMessageAsProcessed(hash)
43-
}
44-
4539
func (a *API) SetEnvelopeEventsHandler(handler types.EnvelopeEventsHandler) error {
4640
return a.core.stack.Transport.SetEnvelopeEventsHandler(handler)
4741
}

pkg/messaging/layers/transport/filters_manager.go

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,12 @@ type FiltersManager struct {
4747
logger *zap.Logger
4848
mutex sync.Mutex
4949
filters map[string]*Filter
50+
// asymKeys maps an asymmetric filter's FilterID to the private key used to
51+
// decode messages received on it. Symmetric keys are resolved on demand via
52+
// keysManager.RawSymKey(SymKeyID) (the same path the send side uses), so only
53+
// the asymmetric keys — which are not otherwise reachable from a persisted
54+
// Filter — are held here. Runtime-only; never persisted.
55+
asymKeys map[string]*ecdsa.PrivateKey
5056
}
5157

5258
// NewFiltersManager returns a new filtersManager.
@@ -66,10 +72,47 @@ func NewFiltersManager(persistence KeysPersistence, service FiltersService, priv
6672
persistence: persistence,
6773
keys: keys,
6874
filters: make(map[string]*Filter),
75+
asymKeys: make(map[string]*ecdsa.PrivateKey),
6976
logger: logger.With(zap.Namespace("filtersManager")),
7077
}, nil
7178
}
7279

80+
// WatchersByTopic returns the filters interested in the given (pubsubTopic,
81+
// contentTopic) pair. A filter with no explicit pubsub topic listens on the
82+
// default shard, so it is normalized before comparison — preserving the
83+
// (pubsub, content) routing the waku layer used. A single content topic can be
84+
// shared by distinct chats (content topics are a 4-byte hash of the chat id),
85+
// so the caller must still confirm the match by decoding with each candidate's
86+
// own key.
87+
func (f *FiltersManager) WatchersByTopic(pubsubTopic, contentTopic string) []*Filter {
88+
f.mutex.Lock()
89+
defer f.mutex.Unlock()
90+
91+
var res []*Filter
92+
for _, filter := range f.filters {
93+
if filter.ContentTopic.ContentTopic() != contentTopic {
94+
continue
95+
}
96+
filterPubsubTopic := filter.PubsubTopic
97+
if filterPubsubTopic == "" {
98+
filterPubsubTopic = wakuv2.DefaultShardPubsubTopic()
99+
}
100+
if filterPubsubTopic == pubsubTopic {
101+
res = append(res, filter)
102+
}
103+
}
104+
return res
105+
}
106+
107+
// AsymKey returns the asymmetric private key used to decode messages received
108+
// on the filter with the given FilterID, if it is an asymmetric filter.
109+
func (f *FiltersManager) AsymKey(filterID string) (*ecdsa.PrivateKey, bool) {
110+
f.mutex.Lock()
111+
defer f.mutex.Unlock()
112+
k, ok := f.asymKeys[filterID]
113+
return k, ok
114+
}
115+
73116
func (f *FiltersManager) Init(
74117
filtersToInit []FiltersToInitialize,
75118
publicKeys []*ecdsa.PublicKey,
@@ -281,6 +324,7 @@ func (f *FiltersManager) Remove(ctx context.Context, filtersToRemove ...*Filter)
281324
if filter.SymKeyID != "" {
282325
f.service.DeleteSymKey(filter.SymKeyID)
283326
}
327+
delete(f.asymKeys, filter.FilterID)
284328
for k, v := range f.filters {
285329
if filter.FilterID == v.FilterID {
286330
delete(f.filters, k)
@@ -312,6 +356,7 @@ func (f *FiltersManager) RemoveNoListenFilters() error {
312356
if filter.SymKeyID != "" {
313357
f.service.DeleteSymKey(filter.SymKeyID)
314358
}
359+
delete(f.asymKeys, filter.FilterID)
315360
for k, v := range f.filters {
316361
if filter.FilterID == v.FilterID {
317362
delete(f.filters, k)
@@ -677,6 +722,9 @@ func (f *FiltersManager) addAsymmetric(chatID string, pubsubTopic string, identi
677722
if err != nil {
678723
return nil, err
679724
}
725+
726+
f.asymKeys[id] = identity
727+
680728
return &RawFilter{FilterID: id, Topic: types.BytesToTopic(topic)}, nil
681729
}
682730

pkg/messaging/layers/transport/messaging_api.go

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,12 @@ type MessagingAPI interface {
1818
// (see rfc26.Encode). Returns the wire hash on success.
1919
Send(ctx context.Context, pubsubTopic, contentTopic string, payload []byte, ephemeral bool, priority *int) ([]byte, error)
2020

21-
// GetFilterMessages returns the messages that match the filter criteria
22-
// and were received between the last poll and now.
23-
GetFilterMessages(id string) ([]*types.Message, error)
21+
// SubscribeEnvelopeEvents returns the backend's event stream. The transport
22+
// consumes it both for reception (EventEnvelopeAvailable carries a neutral
23+
// *types.ReceivedMessage in Data, which the transport decodes and routes to
24+
// its filters) and for the send-side lifecycle events the EnvelopesMonitor
25+
// tracks. This is the single, logos-delivery-shaped event seam.
26+
SubscribeEnvelopeEvents(events chan<- types.EnvelopeEvent) types.Subscription
2427
}
2528

2629
// Compile-time assertion: the waku adapter satisfies MessagingAPI.

0 commit comments

Comments
 (0)