Support for drive composition

[miltonjonat]

As described in the Cartesi documentation, there is a limit of 8 drives in a Cartesi Machine, which in practice limits the number of 3rd-party bundles a user can use when creating his machine.

It is however possible to offer a solution for people to compose bundles, so as to "squash" their contents into a single drive. The idea would be to simply copy the contents of a number of assets into a single drive. Furthermore, if this composition is executed by a Cartesi Machine itself, then it will be deterministic and thus guaranteed to result in the same exact final content. As such, a Carti machine configuration could contain the specification of a drive composition, and then any user could use that configuration to build the same final machine (i.e., with the same template hash)

Composing drives using a Cartesi Machine would obviously only work in environments where the machine-emulator is installed. At first, the Cartesi team is only targeting official support for packaging it for Ubuntu 20.04.

The idea would thus be the following:

1. Create command carti compose to execute the composition

   • Example: carti compose flash --name my-util bundle1 bundle2 bundle3 would produce a local bundle my-util with the contents of the given list of bundles

2. Muliple calls to carti machine add ... <label> with the same label to specify composition for a Carti machine config

   • Carti would check that the composing bundles do not conflict: they should be of the same "type" (e.g., flash) and an error should be thrown if attempting to compose with a non-bundle drive
   • carti machine install would install base bundles, compose them and produce the final Lua config (it would throw an error if it needs to compose bundles and the machine-emulator is not available)

[zcstarr]

This is a good idea maybe this is a target for after this sort of inaugural release

[miltonjonat]

It definitely is a target for after Feb 1st. But it's a very nice target :)

[zcstarr]

Hey kicking things off on this arc. Noticed this in the docs The genext2fs command produces a file-system that includes modification times (and user IDs, permissions etc).

My original thought here was to do something like the following. Use the emulator genext2fs to generate a new drive that could contain any combination of flash drives specified. using the emulator copy the items into the "new bundled drive". reference that new bundled drive in the machine config.

Does this work in terms of reproducibility? If it does then, I guess I have a slight confusion about how to generate the initial output drive. essentially

I think I want something like

cartesi-machine .... \ 
   -- "genext2fs -b sizeof(n flash drives) output-bundle.ext2"
# issue is I'm not sure how to output output-bundle.ext2 from the emulator
# If I make it outside the emulator then it won't be reproducible
....

Am I missing something here? @miltonjonat