Search messages in a room

[smarizvi110]

Draft PR for adding functionality to search messages, fixes #122

So far, the code is for writing the logic for this. Once that is finalized, work can be done on the UI for it.

After a lot of confusion, this is what I've come up with:

1. User Initiates Search

• User enters a search term.
• Calls submit_async_request(MatrixRequest::SearchRoomMessages).
• Request is sent to the async worker.

2. Async Worker Sends Request

• Constructs a search_events::v3::Request with search criteria.
• Calls client.send(search_request, None).await.
• If the request fails, logs an error and exits.

3. Extract Event IDs From Search Results

• The search response only contains event references.
• Extracts event_ids from AnyTimelineEvent::{MessageLike, State}.

4. Fetch Full Event Details

• Calls fetch_details_for_event(event_id) for each result.
• Fetches full event data and sends it to the UI thread.

5. UI Processes Search Results

• SEARCH_RESULTS_SENDER sends SearchUpdate::NewResults.
• process_search_updates() updates the UI and redraws.

6. Pagination (If Needed)

• If next_batch exists, calls paginate_search_results().
• Submits another MatrixRequest::SearchRoomMessages with next_batch.

sequenceDiagram
    participant User
    participant UI
    participant AsyncWorker
    participant MatrixServer

    User->>UI: Enter search term
    UI->>AsyncWorker: submit_async_request(MatrixRequest::SearchRoomMessages)
    AsyncWorker->>MatrixServer: Send search_events::v3::Request
    MatrixServer-->>AsyncWorker: Return event references (event_id)
    
    AsyncWorker->>MatrixServer: Fetch event details (fetch_details_for_event)
    MatrixServer-->>AsyncWorker: Return full event data

    AsyncWorker->>UI: Send SearchUpdate::NewResults via SEARCH_RESULTS_SENDER
    UI->>UI: process_search_updates() -> Update search UI

    UI->>AsyncWorker: paginate_search_results() (if next_batch exists)
    AsyncWorker->>MatrixServer: Send another search request (pagination)

Loading

Could someone verify this flow to be correct? @kevinaboos, I tried to gain some insight through LoadingPane as you had mentioned in the comments of #122, but I was not able to find something of that sort to get an idea of going about this. The logic and code in this PR is based on the logic and code for the pagination done for the Timeline in a Room, or at least on what I could understand from it all

[smarizvi110]

This shouldn't be public ideally, but adding it as a part of the RoomScreen struct makes it so I would need to add an associated function for making a new RoomScreen object, which I'm not sure is the right thing to do. I'd appreciate guidance on this!

[kevinaboos]

In general, the right pattern for this kind of thing would be to create a channel when you first show the UI widget/view that will display the search results. Then, the UI widget holds onto to the receiver side of the channel, but sends the sender side to the background worker thread as part of the MatrixRequest::SearchRoomMessages.

That way, the background worker task can use the sender to send search updates to the UI widget, which will continuously poll the receiver for new updates upon any signal.

We use this pattern quite often (see how TimelineUpdates are sent from a background task to the RoomScreen UI widget). With this pattern, you don't need a static global variable like the SEARCH_RESULTS_SENDER here.

[smarizvi110]

Right! That makes sense! Thank you!

[smarizvi110]

Hi, not sure why this is tagged 'waiting-on-author'. Can someone clarify?

[alanpoon]

Hi, not sure why this is tagged 'waiting-on-author'. Can someone clarify?

Please resolve the conflict. It is 944 commit behind main

[ZhangHanDong]

@smarizvi110

You have marked this as a "Draft PR" , “waiting on author” means waiting for you to prepare this PR.

[smarizvi110]

I’ve resolved conflicts and merged the latest changes from search-messages-in-room into my local branch, as suggested by @alanpoon (thank you!).

The PR currently says "waiting-on-author", but I actually need more input before proceeding. When @kevinaboos suggested using LoadingPane as a reference for structuring the search logic, it hadn't yet been merged into the upstream main branch, so my current logic doesn’t reflect any insights from it. Now that LoadingPane is available, I’m unsure whether I should rewrite the search logic from scratch using it as a guide or adjust what I’ve already written.

Some checks are failing, but they seem related to code warnings rather than functional issues, let me know if there’s anything specific you'd like me to address.

I want to move forward but need your guidance on how best to approach the search logic now. Could someone review the current flow I outlined in the first comment and advise whether a full rewrite is necessary or if there’s a way to refine what's already there?

Thanks!

[kevinaboos]

hi @smarizvi110, thanks for the contribution, and my apologies for the delay! This PR got lost in the fray.

Yes, your logic and diagram look generally correct! I haven't dug into the code deeply yet, but this looks appropriate for unencrypted rooms.

When @kevinaboos suggested using LoadingPane as a reference for structuring the search logic, it hadn't yet been merged into the upstream main branch, so my current logic doesn’t reflect any insights from it. Now that LoadingPane is available, I’m unsure whether I should rewrite the search logic from scratch using it as a guide or adjust what I’ve already written.

What I was suggesting here is only relevant for encrypted rooms, where the search_events::v3::Request API cannot work
(although you could certainly use this same approach for public/unencrypted rooms too).

The main idea is to implement a pagination-based search by re-using the same logic that we already use for BackwardsPaginateUntilEventRequest. That request type is very similar to what you need to do here, but instead of back-paginating until we find a specific event, we back-paginate forever and use the sender I mentioned here to send updates with matching events, until the user cancels the search.

To implement this, you could change BackwardsPaginateUntilEventRequest into an enum that looks something like this:

pub enum BackwardsPaginateWhileConditionRequest {
    UntilEventFound {
        room_id: OwnedRoomId,
        target_event_id: OwnedEventId,
        starting_index: usize,
        current_tl_len: usize,
    },
    WhileSearchingFor {
        room_id: OwnedRoomId,
        conditions_to_match: OwnedEventId,
        search_update_sender: crossbeam_channel::Sender<SearchUpdate>,
    }
}

and then handle these two conditions in the timeline_subscriber_handler()'s tokio::select loop.

This is not a simple feature, so you should probably skip this feature and save it for a future PR.

[kevinaboos]

When you feel that your basic search functionality (for unencrypted rooms) is complete and ready for review, please request a review from me (in the upper-right corner of this PR screen).

[alanpoon]

I suggest you to do self-testing of matrix sdk of search api in a separate test repo and ask question in Robius General Element.io chat group. Create a pull request when the integration test / proof of concept is done.

[smarizvi110]

Thank you all for the input! Yes, this is a very mind-boggling issue. I will try to incorporate all the feedback, especially alanpoon's suggestion of testing.

I suppose I will keep this on the back burner for now like so, and look for a simpler feature to implement. If anyone is already aware of any issue that would be suitable, I'd be grateful for the guidance on that front too!

[alanpoon]

@smarizvi110 Hi, could you help to review and test my PR #483 in replacement for this PR?

[smarizvi110]

@smarizvi110 Hi, could you help to review and test my PR #483 in replacement for this PR?

Hello @alanpoon. Sure, I'd be happy to!