Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Slow TwitIE Performance due to lack of multi-threading #6

Open
greenwoodma opened this issue Sep 14, 2020 · 0 comments
Open

Slow TwitIE Performance due to lack of multi-threading #6

greenwoodma opened this issue Sep 14, 2020 · 0 comments
Labels
enhancement New feature or request

Comments

@greenwoodma
Copy link

TwitIE is accessed via a web service endpoint which can support multiple concurrent requests. Unfortunately the wrapper works through the tweets sequentially passing each one in turn to TwitIE and waiting for a response.
@greenwoodma greenwoodma added the enhancement New feature or request label Sep 14, 2020
@greenwoodma greenwoodma mentioned this issue Sep 14, 2020
Closed
@greenwoodma greenwoodma mentioned this issue Sep 21, 2020
Closed
greenwoodma added a commit that referenced this issue Sep 22, 2020
@greenwoodma
complete e-working of the calls to TwitIE which should ensure better …
5eb34a6 
…performance. We know parallelise the reading from the ES stream and handle each tweet in turn. This not only means the tweets are processed in parallel but we can start processing them before we've read the full result set from ES. see issue #6 for more details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@greenwoodma