Historical messages decrypt on cache reloading, but not before

[richvdh]

Steps to reproduce

Sign into a new instance of Element X
Verify your device
Open some chats with historic E2EE message, those messages can appear as UTDs
Clear EX cache in developer settings
All historic messages are now decrypted properly

Outcome

What did you expect?

Historic messages decrypt successfully, and are displayed as such, without having to cache reload.

What happened instead?

Historic messages that are shown as UTDs in the timeline seem stuck that way indefinitely. It could be that the keys are being downloaded or processed at the same time as a user trying to open the message, EX is unable to show the message and presents it as a UTD, and subsequently doesn't attempt to decrypt the message again in future.

[richvdh]

This is tracked on the EXA and EXI repos as element-hq/element-x-android#3234 and element-hq/element-x-ios#3055 respectively

[manuroe]

If the user paginates in the room, the received paginated messages get decrypted in the timeline. The 20-30 last messages on the room stay UTDs but previous messages are decrypted.

Since client side sorting, we have a mechanism to prefill the room timeline from the room list by subscribing to each displayed rooms. The decryption retries may fail in this condition or they succeed but we fail to notify the UI of the timeline.

[poljar]

I think it's rather that events are received before the backup becomes active, and the timeline doesn't listen for the backup becoming active.

It should listen to the backup becoming active and retry the decryption of all events, since we download room keys on demand, i.e. when we fail to decrypt a event.

Relevant code should be here:

matrix-rust-sdk/crates/matrix-sdk-ui/src/timeline/builder.rs

Lines 322 to 349 in 68c612f

	let room_key_from_backups_join_handle = {
	let inner = inner.clone();
	let room_id = inner.room().room_id();
	let stream = client.encryption().backups().room_keys_for_room_stream(room_id);
	spawn(async move {
	pin_mut!(stream);
	while let Some(update) = stream.next().await {
	let room = inner.room();
	match update {
	Ok(info) => {
	let mut session_ids = BTreeSet::new();
	for set in info.into_values() {
	session_ids.extend(set);
	}
	inner.retry_event_decryption(room, Some(session_ids)).await;
	}
	// We lagged, so retry every event.
	Err(_) => inner.retry_event_decryption(room, None).await,
	}
	}
	})
	};

[richvdh]

We're still seeing reports of this happening in rageshakes.

[andybalaam]

@poljar are you planning to investigate this? If not, we will need to put it back into the backlog.

[andybalaam]

I am unable to reproduce this in my dev environment, with the latest Element X code, using matrix.org homeserver.

Steps:

• Log in 2 test users on matrix.org in Element Web and verify their devices.
• Send lots of messages from user 2 to user 1 in a DM.
• Open dev Element X. Sign out from the logged-in user. Sign in as user 1.
• Open DM with user 2 on Element X.

I see all the messages, correctly annotated with "The authenticity of this encrypted message can't be guaranteed on this device".

I have tried this multiple times.

So my tentative conclusion is that the problems we have been seeing lately are not caused by this bug, but some different problem.

[uhoreg]

Found a new rageshake that seems to be an instance of this.

[richvdh]

I'm going to open a new issue for this

[richvdh]

#4429

[richvdh]

The original issue was fixed in rust-sdk dcc20b6, which landed 9 September.

That was adopted in https://github.com/matrix-org/matrix-rust-components-kotlin/releases/tag/sdk-v0.2.43, which was first used in https://github.com/element-hq/element-x-android/releases/tag/v0.6.0.

It was also adopted in https://github.com/element-hq/matrix-rust-components-swift/releases/tag/v1.0.45, which was first used in https://github.com/element-hq/element-x-ios/releases/tag/1.8.0.