https by default

[lool]

Review http vs https usage in the project and use https where it makes sense.

• fix!(debos): Use https for APT by default
• fix(ci): Bootstrap chroots with https
• feat(Makefile): Also set https_proxy

This is particularly important in the context of compliance as http:// might be seen as insecure, or at least lacking confidentiality.

Fixes: #290

[basak-qcom]

I'm OK with shipping sources.list with https, but can we continue to use plain http for development and testing please? Otherwise caching becomes impossible.

[basak-qcom]

(which is going to slow down development iterations massively)

[github-actions]

Test Results

 2 files  ±0   6 suites  ±0   6m 50s ⏱️ ±0s
20 tests ±0  20 ✅ ±0  0 💤 ±0  0 ❌ ±0 
64 runs  ±0  64 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 8b58150. ± Comparison against base commit 8c913de.

[github-actions]

Test jobs for commit 8b58150

• Job 116560 on qrb2210-rb1
• Job 116561 on qcs6490

[basak-qcom]

I understand this will be more of a pain to implement whatever we do. The best I can think of is to parameterise whether we want http or https, perhaps default to https for safety, allow the developer to override for local builds, and maybe add some tests to ensure that there are no plain http:// in sources.list at the end.

[lool]

After feedback from upstream, I'm not sure I'll pursue this branch, but I should at least move it to draft.

[lumag]

@basak-qcom what kind of caching do you use? I think apt-cacher (or apt-cacher-ng) should support https caching.

[lool]

I'm using apt-cacher-ng, and https support is not fun to use

[lumag]

Okay :-(

[basak-qcom]

AIUI, apt-cacher-ng only supports HTTPS passthrough, ie. not caching? The only way to cache it would be to provide an MITM certificate for use on the client.

I also bumped into there being no HTTP support for https://qartifactory-edge.qualcomm.com/artifactory/qsc-deb-releases. That is shipping hexagon-dsp-binaries currently, so this hurts caching :-(

[lool]

There's one or more hacks in apt-cacher-ng to do caching with https, see https://www.unix-ag.uni-kl.de/~bloch/acng/html/howtos.html#ssluse but you have to rewrite your sources.list.

I'm configuring acng with passthrough and also successfully used HTTPS///, but they suck (no caching or ugly cache-specific sources.list).

[basak-qcom]

What I'd like to do is:

1. Use http everywhere as much as possible inside qcom-deb-images.
2. Write a recipe that converts sources.list to https, using perhaps an allowlist of known URL hosts that support both. This could be quite limited - it's only necessary to look inside sources.list.d/ for files of the new format, for example. Even though the key is URIs, apparently only one is supported.
3. We should also apt-get clean at this stage as a matter of hygiene.
4. Add tests that a) ensure there is no plain HTTP remaining; and b) verifies that apt-get update succeeds without errors or warnings.

I noticed that both the rootfs and image recipes require apt, so annoyingly this will either need to be done in the image recipe, or in a new separate recipe. I can think of some other ways of rearranging things, but none of them are nice.

[lool]

I'd like to get rid of APT in the image recipe, but that requires some boot flow reviews