Key backup cannot be reset, after it did not accept my saved password for it on another machine

[mpeter50]

Steps to reproduce

1. Open the Settings, Security menu
2. Delete the broken key backup configuration
3. Set up key backup again ("Set up" button)
4. Insert password for backup from password manager (same password that should have worked previously)
5. Observe error message

Outcome

What did you expect?

I expected that the key backup function would resume working.

What happened instead?

Key backup cannot be set up. The menu shows this error message:

👎 Unable to access secret storage. Please verify that you entered the correct Security Phrase.

Operating system

Windows

Browser information

Firefox

URL for webapp

https://riot.grin.hu

Application version

1.11.95

Homeserver

https://matrix.grin.hu

Will you send logs?

Yes

[florianduros]

Hey, if you use instead the settings in the encryption tab of the user settings. Are you able to enable your key storage and create a recovery key?

[mpeter50]

That menu has an option to change the recovery key:

So it seems to me that different parts of the webapp are in disagreemant about whether it is set up.

If I click "change recovery key", I'm given a new key, which if I save to my password manager and click confirm, it hangs for a few seconds and then the submenu disappears. But the "Security & Privacy" menu still looks like as if key backup would not be set up:

After doing the above, I have opened element web on the machine where I wanted to log in, and it shows a popup in the top left to set up Secure Backup. I declined it for now.
After that, I have opened the Settings > Encryption menu, and successfully verified my device with this new key.
Settings > Security & Privacy shows the same thing as the above screenshot, that this session is not backing up keys.

But now I dont understand.
What is the relation of this recovery key that the webapp generates, to the former key backup password that was thinked up by the user?
And why couldn't I use the key backup password that I have set up formerly, to recover my backed up data?

I consider myself a technical person. I'm running several selfhosted services (not Matrix though) for myself at home on linux, with Docker, Nginx, Proxmox, and been doing that for years now, but this many keys and passwords are becoming very confusing, because the descriptions in the client dont seem to check out. And I'm basing this off of the English interface.
Of course we have the login password, thats no question. Then we have (had?) the key backup password, to encrypt the key storage for holding by my homeserver, to be able to recover access to my encrpted messages when element logs me out from all my sessions.
But then there is this recovery key that is something else. Its the same thing according to the client's description, but its a different format, generated instead of prompted for, and claims to do key backup while the other menu says that has not been set up.
And now I just noticed that my password manager has 2 more passwords saved for matrix, "Security Phrase" and "Security Key", which are probably some kind of duplicates from when key abckup was renamed twice, or something else.

I think it would make sense to have a document about all of these (made for users, not just programmers familiar with Element's code and history), that clears up what kinds of passwords/keys are required from the user today, and what were former names of these same secrets historically.

[florianduros]

https://element.io/help#encryption a bit of documentation

Let me explain. The key backup is a backup where the message keys are stored in order to be able to decrypt the message if the user don't have any other device to share these keys.

Now, we prefer to not talk about key backup with the user and to use the word key storage (as you can see in the encryption tab of the user settings). The key storage stores the message keys and the user secrets (which are keys).

In order to decrypt the keys in the key storage, the user must use his recovery key known previously as security key/primary key. Or with a passphrase (which is deprecated, we don't propose this option anymore in the new encryption tab). This passphrase is the key backup password that you're talking about. When the user choose the passphrase option and enter its for the first time, a recovery key is generated and can be accessed only with the given passphrase. We are trying to get rid of this and to only have the option to generate the recovery key. Of course the former passphrase can still be used for backward compatibility.

We make these changes and renaming to be less technical and to have the same words used across the Element clients (Element Web, Element X). The key backup section in Security & Privacy will be removed soon (#26468). The goal is to have only a simplified encryption tab for the users to handle encryption settings.

To summarize:

• Key storage -> key backup + store user secrets
• Recovery key = security key/primary key
• Key backup passphrase/password -> gives access to recovery key stored when the passphrase was created. We are trying to get rid of it to simplify the option.
• Only key storage and recovery key words are displayed to the user in the encryption tab. Key backup section will not be displayed anymore and only available in the devtools.

[mpeter50]

Thanks for the explainer!

Now, we prefer to not talk about key backup with the user and to use the word key storage (as you can see in the encryption tab of the user settings). The key storage stores the message keys and the user secrets (which are keys).

I dont see key storage mentioned in the Encryption tab of the settings, but maybe thats a change in the pipeline. I'm looking at Element Web 1.11.95.
But essentially the key storage is what the server stores, right? Basically the backing store for the backed up keys and user secrets.

In order to decrypt the keys in the key storage, the user must use his recovery key known previously as security key/primary key. Or with a passphrase (which is deprecated, we don't propose this option anymore in the new encryption tab). This passphrase is the key backup password that you're talking about. When the user choose the passphrase option and enter its for the first time, a recovery key is generated and can be accessed only with the given passphrase. We are trying to get rid of this and to only have the option to generate the recovery key. Of course the former passphrase can still be used for backward compatibility.

So "recovery key" is the same thing as the former "security key", and if I have a separate "security key" saved I can delete that, because its been replaced when I clicked the "change recovery key" button, right?
I agree that recovery key is a better term for this.

And the "security phrase" user-made password was an alternative to the recovery key, not a wrapper, and for some time now the client automatically creates a recovery key the first time the user enters the security phrase, but lets both work until "change recovery key" is used, right?

We make these changes and renaming to be less technical and to have the same words used across the Element clients (Element Web, Element X). The key backup section in Security & Privacy will be removed soon (#26468). The goal is to have only a simplified encryption tab for the users to handle encryption settings.

The mentioned issue makes me think that something is missing in the Encryption tab for me. I dont have a key storage enable/disable thing thats mentioned in point 1.i . Is that expected in my version? If so, thats fine, just want to make sure my client works as you expect it.
But otherwise the changes look nice, especially the visible warning when recovery is not set up :)

https://element.io/help#encryption

What is key storage?

And key storage in its current form is a (relatively) new thing that is basically a switch from the old system of syncing keys with to-device messages one by one, to syncing the whole storage in bulk, right?

[florianduros]

Thanks for the explainer!

Now, we prefer to not talk about key backup with the user and to use the word key storage (as you can see in the encryption tab of the user settings). The key storage stores the message keys and the user secrets (which are keys).

I dont see key storage mentioned in the Encryption tab of the settings, but maybe thats a change in the pipeline. I'm looking at Element Web 1.11.95. But essentially the key storage is what the server stores, right? Basically the backing store for the backed up keys and user secrets.

Key storage in encryption tab is introduced in 1.11.96 https://github.com/element-hq/element-web/releases/tag/v1.11.96
Yes, it stores the room keys and the user secrets.

In order to decrypt the keys in the key storage, the user must use his recovery key known previously as security key/primary key. Or with a passphrase (which is deprecated, we don't propose this option anymore in the new encryption tab). This passphrase is the key backup password that you're talking about. When the user choose the passphrase option and enter its for the first time, a recovery key is generated and can be accessed only with the given passphrase. We are trying to get rid of this and to only have the option to generate the recovery key. Of course the former passphrase can still be used for backward compatibility.

So "recovery key" is the same thing as the former "security key", and if I have a separate "security key" saved I can delete that, because its been replaced when I clicked the "change recovery key" button, right? I agree that recovery key is a better term for this.

Yes, be sure to save your new recovery key somewhere.

And the "security phrase" user-made password was an alternative to the recovery key, not a wrapper, and for some time now the client automatically creates a recovery key the first time the user enters the security phrase, but lets both work until "change recovery key" is used, right?

Hmm, to my knowledge, you couldn't have both. The user was able to generate a recovery key or to use a passphrase (which is generating a recovery key under the hood but not exposed to the client), not both at the same time.

We make these changes and renaming to be less technical and to have the same words used across the Element clients (Element Web, Element X). The key backup section in Security & Privacy will be removed soon (#26468). The goal is to have only a simplified encryption tab for the users to handle encryption settings.

The mentioned issue makes me think that something is missing in the Encryption tab for me. I dont have a key storage enable/disable thing thats mentioned in point 1.i . Is that expected in my version? If so, thats fine, just want to make sure my client works as you expect it. But otherwise the changes look nice, especially the visible warning when recovery is not set up :)

Key storage tab is introduced in 1.11.96 https://github.com/element-hq/element-web/releases/tag/v1.11.96

element.io/help#encryption

What is key storage?

And key storage in its current form is a (relatively) new thing that is basically a switch from the old system of syncing keys with to-device messages one by one, to syncing the whole storage in bulk, right?

It's not a new thing, it was here for a long time. Key storage is a generic name to group existing feature(Secret storage, key backup...)

There is two way to recover the secrets for a long time:

• Shared by another verified device (recovery key is also shared by verified device)
• Access to the key storage

[mpeter50]

Hmm, to my knowledge, you couldn't have both. The user was able to generate a recovery key or to use a passphrase (which is generating a recovery key under the hood but not exposed to the client), not both at the same time.

Aha, I see.

Ok, now that we cleared up my confusion, do you think we can find out why didnt element accept my security phrase? Maybe there are others out there with a similar issue.
I cant test it anymore unfortunately as I have set up a recovery key, but I think I have sent in logs from 2 devices when the security phrase was not accepted.

[florianduros]

I'll take a look at the logs

[richvdh]

@mpeter50 sorry for the delay in getting back to you here.

Unfortunately, the logs aren't going to be very revealing here: all we can see is that the recovery key you entered did not match the data stored on the server.

I think there are three possibilities as to what has happened.

1. Somebody malicious gained access to your server and reset your recovery data. Let's hope this hasn't happened.

2. A client has, at some point, reset the recovery store.

   More unfortunately still: there is no audit trail within the synapse database of who changed the recovery store when, and it could have been any client that you've used at any time, and unless you have server-side logs that happen to cover the point at which it was changed, we're going to be out of luck figuring out how it happened.

   I think it's possible that there were bugs in the early days of Element X where it would have reset recovery, so it's possible it was that. Otherwise, we're not aware of such bugs in any Element clients.

3. A final possibility which could only affect you if you created your account many years ago: the current concept of "recovery key/security key" (a passphase or key which is used to encrypt a server-side store of secrets, aka SSSS or 4S) was introduced sometime in late 2019. Before that, we had a similar concept, but it was only used for room key storage (aka key backup). This was also known as "security phrase" or "recovery phrase".

   For a long time, the Element clients had support for these legacy security phrases, but the number of accounts still affected is miniscule, and the legacy support wasn't maintained, and is now removed altogether.

   In short, if you set up your account before/during 2019, and haven't used the security phrase since then, it's possible that the "security phrase" that you had in your password manager was actually a security phrase for this key backup, rather than 4S, and as such wouldn't work in modern clients.

[mpeter50]

I see, thanks for the help and the information! I have briefly used very early versions of Element X on my phone (before it got a release on F-droid), so perhaps it was point 2. I think point 3 is unlikely, because I remember that I have used the security phrase a few times.

Since we cannot continue the investigation, I'm closing this issue.