You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: requirements/codex.md
+23-6Lines changed: 23 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,24 +9,41 @@ It then pushes the reference to the bundle over Waku.
9
9
New users retrieve and listen to new messages using Waku upon start up.
10
10
Then, they may retrieve bundles, likely because the know they are missing message via SDS.
11
11
12
-
The original uploader is the one to determine durability, aka, relevance of data over time.
12
+
The original uploader is the one to determine how much persistence and guarantee they want for the bundle.
13
13
It is application specific (eg until a Q&A is completed), and not related to users having downloaded the data.
14
14
15
15
Which means it's a scenario where:
16
16
17
17
- Time from upload to retrieval is **not** critical (latest messages are available on Waku)
18
18
- Several users can seed and download the bundle.
19
19
20
-
This is very similar to BitTorrent integration in Status. I need to find specs to be more explicit about difference.
21
-
Some notes on BitTorrent integration in Status (AFAIK, asking @osmaczko for help):
20
+
This is very similar to BitTorrent integration in Status ([specs](https://github.com/vacp2p/rfc-index/blob/main/status/61/community-history-service.md))
21
+
Some notes on BitTorrent integration in Status:
22
22
23
23
1. Known issue is that the bundle is very large, and hence consumes a lot of bandwidth. I don't know if the bundle is "updated" or overridden.
24
24
On Waku app side, we need to be open to have one large bundle vs a series of manageable bundle. The latter offers more flexibility such as attaching bloom filters,
25
25
for selective download.
26
26
2. The bundle download is indiscriminate, meaning every user will download it at some point, with SDS, we can do something smarter
27
27
28
+
**Technical solutions**
29
+
30
+
A comment on possible solutions:
31
+
28
32
Also note (more of a personal opinion), usage of BitTorrent/webtorrent could be an acceptable starting point.
29
33
34
+
Web is being solved by @vpavlin with vpavlin/qaku-cache which acts as a pinning gateway:
35
+
36
+
> it is basically a pinning gateway, but instead of using HTTP to upload you use the Codex network itself.
37
+
> It would be cool to add some auth - I was thinking about using Semaphore or RLN to limit who and how much can cache - WDYT?
38
+
39
+
Or with a Codex web-client that can retrieve and upload from the network:
40
+
41
+
> I think if we can get a Codex web-client which can upload to the network somehow, we are immediately
42
+
> solving one of the biggest pain points of IPFS and have a great story for anyone "why are you building a new storage
43
+
> and why should I use it?" - I definitely want this to happen:-)
44
+
45
+
These are inferred in the FURPS below with requirements on web support and censorship-resistance.
46
+
30
47
### Functionality
31
48
32
49
1. Ability to transfer a bundle of 1MB or more between two or more nodes.
@@ -40,7 +57,7 @@ Also note (more of a personal opinion), usage of BitTorrent/webtorrent could be
40
57
41
58
1. Download operation can be resumed.
42
59
2. Upload operation can be resumed.
43
-
3. As long as original developer is online, bundle should be retrievable.
60
+
3. As long as original uploader is online, bundle should be retrievable.
44
61
4. As long as N out of M users are online, bundle should be retrievable.
45
62
46
63
### Performance
@@ -85,7 +102,7 @@ There is more expectation on timeliness of retrievability, as one would want to
85
102
1. Download operation can be resumed.
86
103
2. Upload operation can be resumed.
87
104
3. Payload should be retrievable even if original uploader goes offline.
88
-
4. Once all recipients have downloaded the payload, there is no more durability expectations.
105
+
4. Once all recipients have downloaded the payload, there is no more expectations on being able to retrieve the payload.
89
106
90
107
### Performance
91
108
@@ -102,4 +119,4 @@ There is more expectation on timeliness of retrievability, as one would want to
102
119
103
120
1. The unavailability of a static host (IP, DNS) does not prevent a user to upload or download (censorship-resistance).
104
121
2. An external observer cannot tie the PIIs of the uploader and downloaders of one payload;
105
-
it is assumed that the reference to the payload (eg, CID) is not leaked outside the participants.
122
+
it is assumed that the reference to the payload (eg, CID) is not leaked outside the participants.
0 commit comments