Albatross.manifest_devices_match is never invoked

[hannesm]

This is not done during a "normal" deploy (i.e. unikernel_create). The normal deploy is also missing a "force" checkbox to do a force-create.

But with #73 in mind and streaming, we'll have to revise that manifest_devices_match in some way anyways, storing another bigarray with the unikernel binary in memory is not a good idea (we need a streaming interface for Solo5_elftool, where it doesn't need to have its hands on the entire binary).

[hannesm]

Update: after we merged the streaming work, the manifest_devices_match is not invoked at all. It needs to be rewritten (including solo5-elftool likely) in a way that it can use a stream of the ELF binary without needing the entire image to be in memory.

[reynir]

I did some work on that yesterday, but today I discovered a significant blocker. See the issue in robur-coop/ocaml-solo5-elftool#3 (comment)

[reynir]

Looking at the (* TODO *) it is when we fetch a unikernel from builder-web. With that in mind I think we should be able to query builder-web about the manifest (possibly a new endpoint is required) and trust on that for reporting errors early.

mollymawk/unikernel.ml

Lines 1345 to 1448 in 1ae1512

	let process_change ~unikernel_name ~job ~to_be_updated_unikernel
	~currently_running_unikernel (unikernel_cfg : Vmm_core.Unikernel.config)
	(user : User_model.user) store http_client
	(change_kind : [ `Rollback | `Update ]) albatross =
	let change_string = function
	| `Rollback -> "rollback"
	| `Update -> "update"
	in
	let unikernel_update : User_model.unikernel_update =
	{
	name = unikernel_name;
	job;
	uuid = currently_running_unikernel;
	config = unikernel_cfg;
	timestamp = Mirage_ptime.now ();
	}
	in
	Store.update_user_unikernel_updates store unikernel_update user >>= function
	| Error (`Msg err) ->
	let data =
	Yojson.Basic.to_string
	(User_model.unikernel_update_to_json unikernel_update)
	in
	Logs.warn (fun m ->
	m
	"%s: storing update information for %s failed with: %s for data \
	%s"
	(change_string change_kind)
	unikernel_name err data);
	Lwt.return
	(Error
	( `Msg
	(change_string change_kind ^ " storing update information for "
	^ unikernel_name ^ " failed with: " ^ err ^ " for data: " ^ data
	),
	(`Internal_server_error : H1.Status.t) ))
	| Ok () -> (
	let data_stream, push_chunks = Lwt_stream.create () in
	let push () = Lwt_stream.get data_stream in
	let url =
	Builder_web.base_url ^ "/job/" ^ job ^ "/build/"
	^ to_be_updated_unikernel ^ "/main-binary"
	in
	let f resp _acc chunk =
	if
	Http_mirage_client.Status.is_successful
	resp.Http_mirage_client.status
	then Lwt.return (push_chunks (Some chunk))
	else Lwt.return_unit
	in
	Lwt.both
	( Http_mirage_client.request http_client ~follow_redirect:true url f ()
	>>= fun e ->
	push_chunks None;
	match e with
	| Error (`Msg err) -> Lwt.return (Error (`Msg err))
	| Error `Cycle -> Lwt.return (Error (`Msg "returned cycle"))
	| Error `Not_found -> Lwt.return (Error (`Msg "returned not found"))
	| Ok (resp, ()) ->
	if
	Http_mirage_client.Status.is_successful
	resp.Http_mirage_client.status
	then Lwt.return (Ok ())
	else
	Lwt.return
	(Error
	(`Msg
	("accessing " ^ url ^ " resulted in an error: "
	^ Http_mirage_client.Status.to_string resp.status
	^ " " ^ resp.reason))) )
	(force_create_unikernel ~unikernel_name ~push unikernel_cfg user
	albatross)
	>>= function
	| Error (`Msg err), _ ->
	Logs.err (fun m ->
	m
	"%s: builds.robur.coop: Error while fetching the binary of \
	%s with error: %s"
	(change_string change_kind)
	unikernel_name err);
	Lwt.return
	(Error
	( `Msg
	(change_string change_kind
	^ " :an error occured while fetching the binary from \
	builds.robur.coop with error: " ^ err),
	`Internal_server_error ))
	| Ok (), Error (msg, status) -> Lwt.return (Error (msg, status))
	| Ok (), Ok () -> Lwt.return (Ok ()))
	(* TODO: check manifest in a streaming fashion
	match
	Albatross.manifest_devices_match ~bridges:unikernel_cfg.bridges
	~block_devices:unikernel_cfg.block_devices unikernel_image
	with
	| Error (`Msg err) ->
	Lwt.return
	(Error
	( `Msg
	(change_string change_kind
	^ " :an error occured with the unikernel \
	configuration: " ^ err),
	`Bad_request ))
	| Ok () -> *)

[hannesm]

With that in mind I think we should be able to query builder-web about the manifest

That sounds like a good idea, also with #130 in mind.

But there's also the path you want to deploy your custom compiled unikernel... But I guess it is fine to not have the check then, and instead focus on the 'deploy-unikernel-from-builder-web' path.