Skip to content

Commit b703f4d

Browse files
committed
Incorporate feedback and focus on two features for now
# Conflicts: # requirements/codex.md
1 parent 82d33b6 commit b703f4d

File tree

1 file changed

+54
-61
lines changed

1 file changed

+54
-61
lines changed

requirements/codex.md

Lines changed: 54 additions & 61 deletions
Original file line numberDiff line numberDiff line change
@@ -1,112 +1,105 @@
11
# Waku's Requirements on Codex
22

3-
## Publish Large Messages - Uploader is online
3+
## Message Archival
44

55
To be used for messages archival in Chat SDK, Qaku, opchan, etc.
6-
It assumes that a special user (admin) regularly bundles messages and pushes them to an external system.
7-
It then pushes the CID (or any other reference to retrieve the bundle) over Waku.
6+
It assumes that a special user (admin, referred to as "original uploader") regularly bundles messages and pushes them to an external system.
7+
It then pushes the reference to the bundle over Waku.
88

9-
New users retrieve and listen to new messages using Waku.
10-
Thanks to SDS, they learn whether they miss messages, and if so, can proceed with retrieval from the latest bundle.
9+
New users retrieve and listen to new messages using Waku upon start up.
10+
Then, they may retrieve bundles, likely because the know they are missing message via SDS.
1111

12-
(clearly, spec is needed).
12+
The original uploader is the one to determine durability, aka, relevance of data over time.
13+
It is application specific (eg until a Q&A is completed), and not related to users having downloaded the data.
1314

14-
### Functionality
15+
Which means it's a scenario where:
16+
17+
- Time from upload to retrieval is **not** critical (latest messages are available on Waku)
18+
- Several users can seed and download the bundle.
19+
20+
This is very similar to BitTorrent integration in Status. I need to find specs to be more explicit about difference.
21+
Some notes on BitTorrent integration in Status (AFAIK, asking @osmaczko for help):
22+
23+
1. Known issue is that the bundle is very large, and hence consumes a lot of bandwidth. I don't know if the bundle is "updated" or overridden.
24+
On Waku app side, we need to be open to have one large bundle vs a series of manageable bundle. The latter offers more flexibility such as attaching bloom filters,
25+
for selective download.
26+
2. The bundle download is indiscriminate, meaning every user will download it at some point, with SDS, we can do something smarter
1527

16-
1. Ability to transfer a message of 1MB or more between two or more nodes.
17-
2. Message's CID is less than 100kB.
28+
Also note (more of a personal opinion), usage of BitTorrent/webtorrent could be an acceptable starting point.
1829

30+
### Functionality
31+
32+
1. Ability to transfer a bundle of 1MB or more between two or more nodes.
33+
2. Reference to bundle is 50kB or less.
1934

2035
### Usability
2136

22-
1. Developer can implement upload feature with 10 lines of code or less.
23-
2. No configuration is necessary.
37+
1. Developer can implement feature with 10 lines of code or less.
2438

2539
### Reliability
2640

2741
1. Download operation can be resumed.
2842
2. Upload operation can be resumed.
29-
3. Uploader can be expected to be online when user are downloading.
43+
3. As long as original developer is online, bundle should be retrievable.
44+
4. As long as N out of M users are online, bundle should be retrievable.
3045

3146
### Performance
3247

33-
None
48+
1. Time between bundle uploaded, and retrieved by users can be in the span of minutes and hours (we assumes messages are available in Waku store for several hours).
49+
2. The burden of re-upload is shared by users.
3450

3551
### Supportability
3652

3753
1. Library for Browser applications.
3854
2. Library for Nim desktop applications.
3955
3. Library for Nim mobile applications.
56+
4. Most users may be behind NAT routers and other domestic network setup.
4057

41-
**+ (Privacy, Anonymity, Censorship-Resistance, Deployments)**
58+
### + (Privacy, Anonymity, Censorship-Resistance, Deployments)
4259

4360
1. The unavailability of a static host (IP, DNS) does not prevent a user to upload or download (censorship-resistance).
44-
2. TODO (privacy)
61+
2. A participant cannot determine original uploader's PII (anonymity).
4562

46-
## Publish Large Messages - Uploader is offline
63+
## Large File Transfer
4764

48-
To be used for
65+
To be used when 2 users or more, are transferring a large payload. This may be a large image or video in a private chat.
66+
Or it could be a llm prompt that returns a large image or video.
4967

50-
- large messages transfers (such as images, videos, audio) in Chat SDK, Opchan, etc.
51-
- Enhancement of message archival (uploader does not need to be online for messages to be retrieved).
68+
Due to the broadcast nature of Waku, it would hog too much bandwidth if every large file sent between users where sent over Waku.
5269

53-
Builds on [Publish Large Messages - Uploader is online](#publish-large-messages---uploader-is-online)
70+
In terms of durability, it can be assumed that once all participants have downloaded the payload, it does not need to be retrievable anymore.
71+
It should also be assumed that the users may not be online at the same time (mobile).
72+
There is more expectation on timeliness of retrievability, as one would want to be able to download seconds after the upload happened.
5473

5574
### Functionality
5675

57-
1. After upload, message is retrievable without sender being online.
58-
2. Best effort in terms of message retention; expectations on restrictions are documented.
76+
1. Ability to transfer a payload of 1MB or more between two or more peers.
77+
2. Reference to payload is less than 50kB.
5978

6079
### Usability
6180

62-
1. Receiver can download the large message, even if sender is offline, as long as they get the CID out-of-band.
81+
1. Developer can implement feature with 10 lines of code or less.
6382

6483
### Reliability
6584

66-
1. Uploader may be offline when receiver is retrieving the large message.
85+
1. Download operation can be resumed.
86+
2. Upload operation can be resumed.
87+
3. Payload should be retrievable even if original uploader goes offline.
88+
4. Once all recipients have downloaded the payload, there is no more durability expectations.
6789

6890
### Performance
6991

70-
None
92+
1. Payload download should start within seconds of the upload start.
7193

7294
### Supportability
7395

7496
1. Library for Browser applications.
75-
2. Library for Nim applications.
76-
77-
### + (Privacy, Anonymity, Censorship-Resistance, Deployments)**
78-
79-
TODO (privacy)
80-
81-
## Publish Large Messages - Retention is guaranteed
82-
83-
In this scenario, the uploader wants to ensure the data is persisted and is willing to pay for it.
84-
This may be a Qaku Q&A admin, a opchan cell owner or Status Communities owner.
85-
86-
Builds on previous requirements.
87-
88-
### Functionality
89-
90-
1. Uploader may pay for large message storage to have guaranteed retention.
91-
92-
### Usability
93-
94-
1. Receiver can download the large message, even if sender is offline, as long as they get the CID out-of-band.
95-
2. Receiver does not need to pay to retrieve the large message.
96-
97-
### Reliability
98-
99-
1. Uploader may be offline when receiver is retrieving the large message.
100-
2. Uploader is guaranteed a period of retention for a given price.
101-
102-
### Performance
103-
104-
See previous requirements.
105-
106-
### Supportability
107-
108-
See previous requirements.
97+
2. Library for Nim desktop applications.
98+
3. Library for Nim mobile applications.
99+
4. Most users may be behind NAT routers and other domestic network setup.
109100

110-
### + (Privacy, Anonymity, Censorship-Resistance, Deployments)**
101+
### + (Privacy, Anonymity, Censorship-Resistance, Deployments)
111102

112-
See previous requirements.
103+
1. The unavailability of a static host (IP, DNS) does not prevent a user to upload or download (censorship-resistance).
104+
2. An external observer cannot tie the PIIs of the uploader and downloaders of one payload;
105+
it is assumed that the reference to the payload (eg, CID) is not leaked outside the participants.

0 commit comments

Comments
 (0)