Debug end-to-end encryption

[gabrc52]

It appears that end-to-end encryption issue are somewhat common(!) So we need to test which setups result in end-to-end encryption woes. This could kill the entire effort.

For now let's assume 2-device setups of laptop and phone. Let's try doing some 6.102 partitioning:

• Partition on whether the first device is online while logging in to the second device.

  • true
  • false

• Partition on when the first end-to-end conversation is started:

  • Before signing up (?)
  • After signing up, before attempting to add second device
  • After adding second device
    • Partition on which device started the E2E conversation
      • First device
      • Second device

• Partition on whether the user joined via an email invite:

  • true
  • false

• Partition on first device:

  • Laptop
  • Phone

• Partition on laptop OS (probably unimportant unless there are any bugs specific to Safari or Edge):

  • Windows
  • Linux
  • macOS

• Partition on mobile OS:

  • Android
  • iOS

• Login method

  • Touchstone
  • Password
    (I have no other Touchstone accounts to test, so any easy test accounts are password-based. Touchstone auth is mandated on the production homeserver, so it could be an additional confound).

• Whether logouts are involved (I've been assuming no)

  • yes
  • no

• User accepts the verification prompt in the first place

  • accept
  • decline
  • other / postpone / this wasn't me / dismiss dialog

[gabrc52]

Test suite 1

Cartesian product on ('iOS', 'Android'), (mobile, web), (send encrypted message after first login, send encrypted message after second login).

What is kept consistent is:

• Chrome on Linux - yes, the proprietary one since it is what most people use
• Password sign-up and log-in
• uplink.mit.edu is the homeserver
• app.element.io is the Element client - better for reporting bugs upstream
• No email invites are involved
• Both devices will be online at once
• The end-to-end encrypted messages are DMs to @rgabriel:uplink.mit.edu which has 2 verified devices: web and FluffyChat

Normal-ish tests (baseline):

For consistency these tests will all use Chrome on Linux and uplink.mit.edu as the homserver and hosted Element instance. They will all use regular password-based accounts.

1. Sign up from iOS, send an encrypted message, log in from web
2. Sign up from Android, send an encrypted message, log in from web
3. Sign up from web, send an encrypted message, log in from iOS
4. Sign up from web, send an encrypted message, log in from Android

Testing what happens if there is more than one device by the time when the first end-to-end encrypted device is sent

5. Sign up from iOS, log in from web, send an encrypted message
6. Sign up from Android, log in from web, send an encrypted message
7. Sign up from web, log in from iOS, send an encrypted message
8. Sign up from web, log in from Android, send an encrypted message

[gabrc52]

https://docs.google.com/document/d/14N5Pg5NFpxz9jb-TdQntd4iy_5fGNk3cS7YQ42UqODM/edit

I tested a couple cases and mostly everything worked. Found other minor bugs.

Having 2 devices wasn't an issue, but having only one at a time may (I was able to reproduce the "room upgrade" bug which led to broken UI), but it was recoverable by setting up secure backup then verifying.