Persistant Metrics

[wymangr]

Is it possible to save the metrics in a way so they don't get reset every time my application restarts? My application is a flask api with each endpoint in their own container to keep them modular. I have it running in multiproc mode and each container is sharing a volume for the multiproc directory. Then I can aggregate all the metrics under one /metrics endpoint. The problem is I get errors when I restart one of the containers and then I need to delete the volume for it to work again, clearing out all the historic counters. It would be awesome if there was a way to keep all the metrics even after restarting.

[csmarchbanks]

Hello, I just converted this to a github discussion.

There is not a supported way to avoid resets when an application restarts, and the multiproc directory must be wiped. However, Prometheus is designed to handle resets in counters whenever a process is restarted so it shouldn't cause too much of an issue. What sort of issues/missing information are you seeing?