Conclusion and Future Improvements

After analyzing the performance of each microservice via load testing, spike testing and fault tolerance testing, we conclude that each microservice (except the data service) can serve a real-time load of up to 250,000 requests with very low error rates (<0.5%) and a throughput ranging from 280 to 500 per second when replica count is kept at 3 instances per container. Increasing the number of replicas will increase the throughput as expected keeping the error rate same. Decreasing the number of replicas will decrease throughput as expected with a relatively slight increase in the error rate.

Our data service’s response time is significantly higher than all our other services and hence brings down the overall throughput of our system when load testing all microservices at the same time.

We have considered the below improvements for our system, especially the data service:

We plan to convert the synchronous communication of client and data service to asynchronous communication using below mechanisms:

• Cache Management:
  Persisting status of each request using cache management service like Redis to prevent frequent loss of requests to widget endpoint.
• Polling:
  Employing polling mechanism between frontend and backend. Client service will intermittently request for the status of graph generation from the data-service.
• Messaging Queue:
  On successful graph generation, the object will be pushed in to a message queue where it can be the consumed by the client service after validation using polling.