Skip to content

Add terminated watches metrics #2194

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 15, 2022
Merged

Add terminated watches metrics #2194

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions clusterloader2/testing/load/modules/measurements.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
{{$CEP_PROPAGATION_DELAY_SLO_PERCENTILE := DefaultParam .CL2_CEP_PROPAGATION_DELAY_SLO_PERCENTILE 95.0}}
{{$ENABLE_CONTAINER_RESTARTS_MEASUREMENT := DefaultParam .CL2_ENABLE_CONTAINER_RESTARTS_MEASUREMENT false}}
{{$ENABLE_CONTAINER_RESOURCES_MEASUREMENT := DefaultParam .CL2_ENABLE_CONTAINER_RESOURCES_MEASUREMENT false}}
{{$ENABLE_TERMINATED_WATCHES_MEASUREMENT := DefaultParam .CL2_ENABLE_TERMINATED_WATCHES_MEASUREMENT false}}
{{$ENABLE_QUOTAS_USAGE_MEASUREMENT := DefaultParam .CL2_ENABLE_QUOTAS_USAGE_MEASUREMENT false}}
{{$ALLOWED_CONTAINER_RESTARTS := DefaultParam .CL2_ALLOWED_CONTAINER_RESTARTS 1}}
{{$CUSTOM_ALLOWED_CONTAINER_RESTARTS := DefaultParam .CL2_CUSTOM_ALLOWED_CONTAINER_RESTARTS ""}}
Expand Down Expand Up @@ -158,6 +159,30 @@ steps:
- name: Perc50
query: quantile_over_time(0.50, sum by (container) (container_memory_working_set_bytes / 1024 / 1024)[%v:])
{{end}}
{{if $ENABLE_TERMINATED_WATCHES_MEASUREMENT}}
- Identifier: TerminatedWatchesMetrics
Method: GenericPrometheusQuery
Params:
action: {{$action}}
metricName: Terminated Watches
metricVersion: v1
dimensions:
- resource
queries:
- name: Terminated watches
query: sum(increase(apiserver_terminated_watchers_total[%v:])) by (resource)
- Identifier: WatchCacheInitializations
Method: GenericPrometheusQuery
Params:
action: {{$action}}
metricName: Watch Cache Initializations
metricVersion: v1
dimensions:
- resource
queries:
- name: Watch cache reinitializations
query: sum(increase(apiserver_watch_cache_initializations_total[%v:])) by (resource)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we split the second query into a separate measurement (but under the same {{if $ENABLE_TERMINATED_WATCHES_MEASUREMENT}} branch) with different Identifier and metricName, say WatchCacheInitializations and Watch Cache Initializations, respectively?

Sorry for the late notice about this, I realized it just won't look nice in https://perf-dash.k8s.io/ after we enable it by default in our tests.

{{end}}
{{if $ENABLE_QUOTAS_USAGE_MEASUREMENT}}
- Identifier: Quotas total usage
Method: GenericPrometheusQuery
Expand Down