How to use this client across some application in Pythonic way?

[igor-susic]

Hi all,

Sorry if this questions is too broad or inappropriate. Basically I'm wondering how should I use this client in my project. Let's say I have 30 methods or/and functions throughout my project that I would love to track execution time etc. I would probably define something like this:

# metrics.py module
MODEL_PREDICTION_PROCESSING_TIME = Summary('model_prediction_processing_time', 'Time spent while running prediction')
# Other 30ish lines each defining some metric like the above one 

And would use it as a decorator on my method for example like this:

# Some other module for example: model.py
from metrics import MODEL_PREDICTION_PROCESSING_TIME
class SomeModel:
    @MODEL_PREDICTION_PROCESSING_TIME.time()
    def _predict(self, features: List) -> tuple:
        pass

Now, is this good approach for the python client? Should I define maybe these global variables in the module that uses them or?
Should I mock the prometheus-client in unit tests and how to deal with that, are there any best practices?

Thank you upfront

[csmarchbanks]

Typically I will try to define a metric in the same file as where it is used, but I have certainly come across some cases where that has not made sense. Sometimes that means I change around where the timing is done, e.g. I will time predict from where it is being called instead of having to remember to time it for each implementation. What you describe will work fine though, keeping the definition and usage close is just something I find easier to reason about/maintain.

[igor-susic]

I agree, the approach I have described is influenced more by languages like Java. It's not totally clear to me what do you mean when you say:

I will time predict from where it is being called instead of having to remember to time it for each implementation.

That would insinuate that I have multiple predict functions/methods defined, but isn't it easier to do it once where you define the function rather than call timing on the call.

And how do you even time it at the invocation location? Is it something like:

o  = SomeModel()

with MODEL_PREDICTION_PROCESSING_TIME.time():
    o.predict()

Or am I missing something?

Thank you

[csmarchbanks]

You are exactly correct about how to time from where predict is being called. To be clear, that suggestion is for if you have multiple models being called from a shared piece of code, otherwise your decoration is great. For example, I have had code where predict might be called for one of many different models depending on the request, so we time it at invocation location.

[igor-susic]

This is exactly what interests me, I have the similar situation, different model per different request. I have one class implementation, with predict method in that class. Class get's instantiated N times with different models inside different instantiated object for each of N available models. Is there some Python background magic that won't allow to just use the annotation like I have above, at the place of method definition and will prometheus-client then report wrong timings, or? Why would I need timing at invocation location, if you could elaborate I would be grateful.

[csmarchbanks]

Your method with the decorator will work fine, I have just personally found it slightly harder to maintain with the imports and remembering to add the decorator to each new model. Both options will work though, and I apologize for any confusion!

[igor-susic]

Thanks for the discussion @csmarchbanks