Files, Assets, CDNs oh my

[hulto]

Problem statement

Tomes should be able to reference ad-hoc CDN files.
Tomes should be able to reference files in other tomes.
Ensure that imported repo files are guaranteed.

---

Next steps:

• Remove the constraint of tome assets from same tome. Allow CDN assets to be accessed by assets.*
• Add "hash" optional param to asset functions that's used to deconflict (error if file hash isn't matched)
• Fix bug where assets with same file path delete each other: Throw error when tome from other repo imports the same file.

assets.copy('example/linux/test-file', '/tmp/win')
+ assets.copy('example/linux/test-file', '/tmp/win', hash="abcde...")

• Add UI component to manage file uploads, and links (associated with [feature] CDN View #984 )
• Rename CDN files to CDN assets in docs.

---

Fetch assets tome doesn't work but we agree that it should.

• Assets are scoped to the current tome only.
• Can't fetch across tomes
• Can't fetch from CDN
• Currently tome names are unique but asset paths are not
  • BUG: currently assets with the same folder path delete each other 🤯
• Currently assets are accessible from CDN. cdn/example/linux/test-file
  • FEAT: Want to have control over this files shouldn't just be exposed publicly esp when we have knowable paths.
• Currently when you upload a file

Why do we need this?

• Real quick load a DLL

Questions on how to do this?

• Currently we use folder path to reference but this isn't unique.

• Maybe prepend the repo name, repo owner and domain?

  • Would pose an opsec issues since this is exposed to the agent system's memory.

• Decouple assets from tomes but keep them linked to repos.

  • maybe repo_name/assets dir is enforced?
  • assets are still limited to the same repo
  • But also allow random unassociated files to be used as assets.

• Random unique id: DB id, timestamp, rand generated ID

  • Could solve this but is not user friendly

• Offer a way to specify a file hash so that users can ensure they're getting the right file.

Requirements:

• Want capability to access ad-hoc files (files not associated with the tome) from within a tome.

• Adhoc files can be overwritten by anyone at anytime.

• Repo linked tomes are guaranteed to be unique per repo won't be overwritten.

• Someone forks a tome repo modifies the tomes and then imports overwriting another tome repo

  • Need domain/user/repo/

• Upload

  • Warn users they're about to overwrite a file

• Tome import

  • Shouldn't be able to overwrite each others files

• Download

  • Make sure you're getting the file you expect

• Uploaded files that would modify a tome asset should be blocked.

User stories:

• I'm super prepared my tomes are ready and rock solid no one mess with this.
• I'm not prepared i just have a binary someone bootstrap me with these files i'm uploading.

repo_name/tome_name/assets/bin.exe
repo_name/assets/bin.exe
cdn/bin.exe
username/tome_name/assets/bin.exe ??? RADICAL

Example:

core:
metadata.yml name: Super Cool whoami
filepath name: whoami/assets/bin.exe

3rd party
metadata.yml name: Super Cool whoami2
filepath name: whoami/assets/bin.exe

asset.fetch("whoami/assets/bin.exe")