Replace broken links send by ChatGpt

[Zabuzard]

Problem

ChatGpt attempts to answer each question posted in #question and it provides "useful links", which is great.
However, and especially because ChatGpt runs with old models, it sometimes posts dead links. That leads to unecessary user frustration.

For example in this post:

The first and last link are both 404 NOT FOUND.

Proposed Solution

We should write code that either removes those links or at least replaces them, for example by (broken link).

For that, we should add some helper methods to the already existing LinkDetection class and also use one of the already existing methods there:

• Add a method like CompletableFuture<Boolean> isLinkDead(String url) to it (which uses HttpClient#sendAsync to check the status. Dont download the full content, just check the status code, it must be 2xx or 3xx. Make sure this runs non-blocking, i.e. with a CompletableFuture.
• Add a method like CompletableFuture<String> replaceDeadLinks(String text, String replacement) that uses above methods and the existing extractLinks method to replace all dead links in the given text with the given replacement. Since isLinkDead runs async, this will run async as well with all futures combined

Once that is done, use that method for the ChatGpt responses. They are constructed in HelpSystemHelper#145. Ideally we send out the chatgpt response without delay, check the links async and then, when the result for that is available through the future, we edit the existing chatgpt response.

This can be done by adding more actions onto the queue in HelpThreadCreatedListener#createAIResponse. RestAction allows adding something like "and then do this" to it.
If we have trouble with this, we can also try to do the link replacement in a blocking way directly, without any edits. But that as a fallback.

Ideally all the helper methods added are all unit tested properly. For the isLinkDead we can mock the HttpClient, so no actual HTTP requests are send out.

[tferdous17]

Would like to take this one on (anax). Can I get assigned?