Skip to content

Conversation

weekenthralling
Copy link

It might be better to clear the historical records each time the notebook_running metric is calculated.

For example, if I create a notebook in the test namespace and then delete it, never using the test namespace again, the notebook_running metric will still contain the record notebook_running{namespace="test"} 1.

This is because the stsCache will no longer track data related to the test namespace, and m.runningKernels.WithLabelValues(ns).Set(v) will never update the data for the test namespace.

Copy link

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign kimwnasptd for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@weekenthralling weekenthralling changed the title fix(metrics): reset running notebooks metrics before scraping statefulsets fix(nb): reset running notebooks metrics before scraping statefulsets Sep 5, 2025
@andyatmiami
Copy link
Contributor

/area v1

@google-oss-prow google-oss-prow bot added the area/v1 area - version - kubeflow notebooks v1 label Sep 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/v1 area - version - kubeflow notebooks v1 size/XS
Projects
Status: Needs Triage
Development

Successfully merging this pull request may close these issues.

2 participants