Device dehydration v2

[BillCarsonFr]

User story

As a user, if I log out my last device, other users should still be able to send messages to me and I should be able to read them once I come back and log in with some device.

Concept

Initial device dehydration feature has several issues:

• Tricky to implement, as it requires that the device rehydration be done before any other API calls
• User loses the advantage of verifying their other devices via emoji or QR code (passphrase needed anyhow to rehydrate)

It's now replaced with MSC3814.

Tasks

Groundwork

• Ruma support for MSC3814 #1898
• Rust crypto SDK Support for MSC3814 - Dehydrated devices v2 matrix-org/matrix-rust-sdk#2349
• https://github.com/vector-im/crypto-internal/issues/126
• MSC updates/Polishing MSC3814: Document new dehydrated device format (probably just a concatenation of the various bits of information needed, similar to existing pickle format) #2602
• Synapse support part 2 Support MSC3814: Dehydrated Devices Part 2 matrix-org/synapse#16010
• Fix endpoint improperly declaring support for MSC3814 matrix-org/synapse#16068

Initial Dehydrated device support

• IOS SDK Device dehydration v2 matrix-org/matrix-ios-sdk#1807
• iOS App Device dehydration v2 element-ios#7630
• WebR full support for dehydrated device v2 #2278
• Implement new dehydration format for Dehydrated Devices #2601
• KeySharing, DevicebasedStrategy | Exclude insecure dehydrated devices when sending messages matrix-org/matrix-rust-sdk#4313
• Dehydrated devices: drop incoming to-device messages from dehydrated devices matrix-org/matrix-rust-sdk#4466
• Device Dehydration: Cache Dehydration Key #2603

Production ready on Web

• Dehydrated devices: drop .well-known check element-web#29387
• Dehydrated devices: gossip the dehydration key between devices when you verify #2703
• Dehydrated devices: gossip the dehydration key to verified devices when you create/change the dehydration key #2753
• Dehydrated devices: API change POST->GET #2704
• Dehydrated devices: API change to make paging consistent with other APIs #2799
• Dehydrated devices: when resetting SSSS, if some prior step fails, we won't reset device dehydration element-web#29084
• Dehydrated devices: Add some stuff to the crypto dev tools element-web#29081
• Device dehydration: upgrade existing user accounts to add a dehydrated device element-web#29080
• Device dehydration: users should be able to see their own dehydrated device(s) element-web#29264
• Dehydrated devices: does not detect if another client changed the dehydration key in 4S element-web#28760
• Device dehydration: detect and handle when user has a dehydrated device for which we don't have the key element-web#29579
• Device dehydration: state explicitly on UI that recovery enables/disables dehydration #2828

Future work

• Rust SDK: Support the resumption of the rehydration process #1978
• Deprecate and remove legacy device dehydration (MSC2697)
• EX: indicate when I have a dehydrated device #2635
• EX full support for dehydrated device v2
• https://github.com/element-hq/crypto-internal/issues/402

[pmaier1]

Design/UX inputs needed for settings and option to enable disable

As discussed today, we want to make this feature transparent to the user as far as possible. As it requires key backup being enabled, we'd bind the availability of dehydrated devices to the setting to enable/disable key backup.

[emmaburton1]

Work on this can be timesheeted to the project on https://element-io.atlassian.net/browse/PSB-87

[poljar]

So a status update here:
Ruma PR: ruma/ruma#1605
Rust SDK branch: https://github.com/matrix-org/matrix-rust-sdk/tree/poljar/device-dehydration
Synapse branch that can be used for testing: https://github.com/poljar/synapse/tree/poljar/msc3814-v2

@stefanceriu above are the main ingredients that you'll need to get started on this on the iOS side, the Rust SDK branch depends on the Ruma PR, so you only need the later two.

[poljar]

Work on this can be timesheeted to the project on https://element-io.atlassian.net/browse/PSB-87

Where exactly do I find this in the myhours thingy? I can't seem to find a PSB-87 in the task list.

[pmaier1]

Where exactly do I find this in the myhours thingy? I can't seem to find a PSB-87 under the BWI task list.

It should just be the title of this ticket.

[emmaburton1]

@poljar I've added the task to relevant customer retainer project.

[pmaier1]

Considering that device dehydration

• will first be a labs feature and we probably can't ask all the end users to turn it on over and over again (when logging out and logging in again)
• the server-side support will also be an experimental feature at first

can we automatically enable the feature on the client if the server-side setting is enabled? Probably doesn't make sense to have two switches to enable it and this way we could control availability centrally.

[t3chguy]

@pmaier1 we don't tend to have HS-driven labs features due to the potential for data loss or work preventing bugs whilst thinking you are running a stable release of Element

[pmaier1]

then we need a separate switch on the backend to force enabling the labs feature on the client. otherwise this will be unusable.

[t3chguy]

Element web/desktop already support Such via config.json but not via anything HS driven.

[Xyz00777]

hi everyone, what is the state, because in element-hq/element-web#28173 dehydration devices are removed...

[ara4n]

i believe that's the v1 code being removed to make room for the v2 code.

[uhoreg]

For "production ready" on Element Web, we should have (aside from the current tasks in progress):

• the SSSS issues resolved
• gossiping of the Dehydration Key
• better way of enabling the feature (currently, requires a flag in .well-known -- maybe not needed, if we're just going to turn on decryption everywhere)
• any required UI improvements

I'd also really like us make the API changes (https://github.com/matrix-org/matrix-spec-proposals/pull/3814/files#r948469437 and https://github.com/matrix-org/matrix-spec-proposals/pull/3814/files#r1540896531) and have the MSC ready for FCP. This isn't technically required for production ready from a product perspective, but it would be very sad if had dehydrated devices "production ready", but have the MSC stall out.

[mxandreas]

Trying to summarize the current state from a user perspective.

• The user must have key storage & recovery must be enabled.
  • This is partially a technical dependency as the dehydration key (used to encrypt the offline device before its creation/upload to the server) needs to be stored on the server.
  • However, also, if user goes offline (no logged in devices), the only way to verify a new device next time they log in, is with a recovery key.
• The user needs to log out and log in again from Element Web (EW) after the feature has been enabled on the server.
  • This login/logout is not considered a significant friction as the feature is meant for users who have to log out regularly from all of their devices. This issue will be addressed.
  • Example: Users log out at the end of the workday. Admin enables the feature on Monday. Tuesday morning users still have messages that fail to decrypt. However, starting from Wednesday morning, all messages sent over night will decrypt successfully.
  • The enabling is done in Synapse with msc3814_enabled as this is currently a beta/experimental feature. There is currently no user-specific toggle to enable/disable this as it is not sure if this is needed.
• Only EW is currently supported, this means:
  • If the user uses only EW, all messages sent while they were offline should be always successfully received and decrypted when they log in again.
  • If the user does not use EW at all, the messages sent while the user was offline can not be decrypted.
  • In case of mixed usage (e.g. EW and EX):
    • User needs to logout/login with EW at least once for the feature to take effect (as already explained above).
    • If user logs into EX after they were offline - the messages sent while the user was offline can not be decrypted (on EX).
    • As soon as the user logs into EW - the messages sent while the user was offline are successfully decrypted, and after that also get decrypted on EX.
• The user has to verify their device with recovery key in order to decrypt the messages that were sent while they were offline.
  • This is FYI only. If the user can verify with another device, it means that they were not offline and hence they did not have the problem in the first place.
• The user is expected to be not offline for extended periods (e.g. several months) as their offline device has limited capacity to fit all the keys required to decrypt the messages that are sent while they are offline.
  • The exact limits - e.g. how many messages can the offline device fit - are currently not precisely measured.
• If the user is online for extended periods (e.g. several months), their offline device may exhaust its capacity.
  • This happens only when all of the user's current devices were verified with another device. As there is no device that was verified with recovery key, the offline device can not be rotated (its capacity freed).
  • This problem will be addressed by gossiping dehydration key between devices so any device can rotate the offline device.

[uhoreg]

• This login/logout is not considered a significant friction as the feature is meant for users who have to log out regularly from all of their devices.

Users to log in and out regularly is one of the use cases for dehydrated devices. But it is also useful for people who only have one device, and then lose that device. So the requirement to log in may be OK for now, but it definitely should be fixed eventually.

• In case of mixed usage (e.g. EW and EX):
  • ...
  • As soon as the user logs into EW - the messages sent while the user was offline are successfully decrypted, and after that also get decrypted on EX.

Assuming that they have key backup enabled, so that EX can obtain the keys that were received by EW