TTL for files and on-demand file deletion #144
Description
Add time to live (TTL) for files so that they can be automatically deleted after a given time period. Given we have file proof of ownership already, users with proof of file ownership can request that their files be deleted (even if they didn't set a TTL for that file). Users with ownership claims to a file should also be able to extend the TTL (or remove it all together) should they wish to.
Notes
The TTL and ownership data will be stored with the infohashes, and not the pieces themselves. The reasons are as follows:
- Simply storing this info once in the infohash is cheaper than storing the same data across
$n$ pieces. - If two users upload files with the exact same data (the generated piece data will also be the same unless they set different redundancy levels or they encrypt their data), then their infohashes (although different) will point to the same data. This is to prevent uploading the same data multiple times. Therefore, if someone sets a TTL on their file but someone else has no TTL on their file, the infohash can be deleted when the TTL expires, but still keep the underlying data intact for other users.
- Only when no infohashes point to the pieces will the validator broadcast to the miners that the pieces are no longer needed. If the miner isn't online to be notified of this to garbage collect, then that's too bad --- they should've been online for that anyway. And besides, the miner might garbage collect automatically if they aren't receiving challenge requests for those pieces.
For big file challenges especially, we want to start uploading "useful" synthetic data (one example is a large dataset) and challenging miners based on that. But initially, we won't have this and having a TTL for large synthetic files is highly beneficial so the "useless" data can be deleted periodically. We should also be testing miners with different TTLs as well as it reflects real usage of the network.