-
Notifications
You must be signed in to change notification settings - Fork 4.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Low performance issues under parallel requests ? #2311
Comments
Hello @abcfy2! Welcome to the elizaOS community. Thank you for opening your first issue; we appreciate your contribution. You are now an elizaOS contributor! |
Can you try the same on our latest main instead of the starter please and feedback me on the results you got ? |
@AIFlowML How to run the latest main branch, I always get this error:
|
@AIFlowML After removed all the plugins except Here's the new result: https://app.artillery.io/share/sh_5bc91a7fb20e1370d165e4b2563a07f8673b8dfc616d4e03af4871e821483650 Still the same result. Since js is single-threaded language, so I doubt there might be something block the main loop thread in eliza. The same is true from the log output of the mock openAI server. Its requests are always one after another (one by one, not bulk), so I suspect that each agent call will block the main thread. Thanks. |
Describe the bug
I do a stress testing to
DirectClient
, but low performance. Here's the stress testing report: https://app.artillery.io/share/sh_973944e7b55ad120fef7a5e6c2a71980c176b5ab38b788baa7c3caa2a973f367You can see almost all the requests timeout.
I guess
generateMessageResponse
will block event loop, so there can only be one concurrent request, but I'm not sure.eliza/packages/core/src/generation.ts
Lines 1270 to 1282 in 89b6a19
To Reproduce
Use eliza-starter to start DirectClient, set
OPENAI_API_URL
in.env
:use mockai to setup a mockai server, and set
data/contents.txt
to:start mockai and eliza-starter, then use artillery to run stress testing. Here's artillery config.yml:
Just post
{"text": "hi"}
tohttp://localhost:3000/{uuid}/message
.The text was updated successfully, but these errors were encountered: