Skip to content
This repository has been archived by the owner on Feb 11, 2025. It is now read-only.

why use websocket #93

Open
NoraEr opened this issue Sep 21, 2024 · 0 comments
Open

why use websocket #93

NoraEr opened this issue Sep 21, 2024 · 0 comments

Comments

@NoraEr
Copy link

NoraEr commented Sep 21, 2024

Hello, thanks for this amazing repo.
I am using the NALLM backend with a custom Streamlit app.
I have been facing issues with the websocket (mainly timeout issues and disconnections) implemented in api/src/main.py under @app.websocket("/text2text") endpoint.
Apologies if my question is basic but I am just trying to understand.

Is there a good reason for using websockets instead of HTTP endpoint?
My understanding is that websockets are mainly used for chat applications between users (where users don't know when they will be getting a new message). If the chat application is between a human and AI where a human always initiate the conversation by asking the question and the AI/ LLM responds with an answer, why do you use a websocket? I assume using HTTP for the "/text2text" endpoint could work as well?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant