Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: contextual flow for remaining metrics #2379

Merged
merged 6 commits into from
Feb 10, 2025
Merged

feat: contextual flow for remaining metrics #2379

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
7b9cb05
processing rate contextual flow
veds-g Feb 3, 2025
c3f99a3
Merge branch 'main' into contextual-rest
veds-g Feb 4, 2025
30cd877
adding CPU and Memory flow
veds-g Feb 5, 2025
f972b2e
adding preset option for navigation
veds-g Feb 6, 2025
ceef578
removed pattern name usage from UI
veds-g Feb 9, 2025
d347864
updating initial dropdown filter val
veds-g Feb 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 56 additions & 54 deletions config/advanced-install/namespaced-numaflow-server.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,39 +141,40 @@ data:
which the metrics proxy will connect\n# url: service_name + \".\" + service_namespace
+ \".svc.cluster.local\" + \":\" + port\n# example for local prometheus service\n#
url: http://prometheus-operated.monitoring.svc.cluster.local:9090\npatterns:\n-
name: vertex_gauge\n objects: \n - vertex\n title: Vertex Pending Messages\n
\ description: This query is the total number of pending messages for the vertex\n
\ expr: |\n sum($metric_name{$filters}) by ($dimension, period)\n params:\n
\ - name: start_time\n required: false\n - name: end_time\n required:
false\n metrics:\n - metric_name: vertex_pending_messages\n display_name:
Vertex Pending Messages\n metric_description: This gauge metric keeps track
of the total number of messages that are waiting to be processed over varying
time frames of 1min, 5min, 15min and default period of 2min. \n # set \"Units\"
or unset for default behaviour\n # unit: Units\n required_filters:\n
name: vertex_gauge\n objects: \n - vertex\n title: Vertex Gauge Metrics\n
\ description: This pattern represents the gauge metrics for a vertex across different
dimensions\n expr: |\n sum($metric_name{$filters}) by ($dimension, period)\n
\ params:\n - name: start_time\n required: false\n - name: end_time\n
\ required: false\n metrics:\n - metric_name: vertex_pending_messages\n
\ display_name: Vertex Pending Messages\n metric_description: This gauge
metric keeps track of the total number of messages that are waiting to be processed
over varying time frames of 1min, 5min, 15min and default period of 2min. \n #
set \"Units\" or unset for default behaviour\n # unit: Units\n required_filters:\n
\ - namespace\n - pipeline\n - vertex\n dimensions:\n
\ - name: pod\n # expr: optional expression for prometheus query\n
\ # overrides the default expression\n filters:\n -
name: pod\n required: false\n - name: period\n required:
false\n - name: vertex\n # expr: optional expression for prometheus
query\n # overrides the default expression\n filters:\n -
name: period\n required: false\n\n- name: mono_vertex_gauge\n objects:
\n - mono-vertex\n title: Pending Messages Lag\n description: This query
is the total number of pending messages for the mono vertex\n expr: |\n sum($metric_name{$filters})
by ($dimension, period)\n params:\n - name: start_time\n required: false\n
\ - name: end_time\n required: false\n metrics:\n - metric_name: monovtx_pending\n
\ display_name: MonoVertex Pending Messages\n metric_description: This
gauge metric keeps track of the total number of messages that are waiting to be
processed over varying time frames of 1min, 5min, 15min and default period of
2min. \n # set \"Units\" or unset for default behaviour\n # unit: Units\n
\ required_filters:\n - namespace\n - mvtx_name\n dimensions:\n
\ - name: pod\n # expr: optional expression for prometheus query\n
\ # overrides the default expression\n filters:\n -
name: pod\n required: false\n - name: period\n required:
false\n - name: mono-vertex\n # expr: optional expression for
prometheus query\n # overrides the default expression\n filters:\n
\ - name: period\n required: false\n\n- name: mono_vertex_histogram\n
\ objects: \n - mono-vertex\n title: Processing Time Latency\n description:
This query pattern is for P99,P90 and P50 quantiles for a mono-vertex across different
\n - mono-vertex\n title: MonoVertex Gauge Metrics\n description: This pattern
represents the gauge metrics for a mono-vertex across different dimensions\n expr:
|\n sum($metric_name{$filters}) by ($dimension, period)\n params:\n - name:
start_time\n required: false\n - name: end_time\n required: false\n
\ metrics:\n - metric_name: monovtx_pending\n display_name: MonoVertex
Pending Messages\n metric_description: This gauge metric keeps track of the
total number of messages that are waiting to be processed over varying time frames
of 1min, 5min, 15min and default period of 2min. \n # set \"Units\" or unset
for default behaviour\n # unit: Units\n required_filters:\n -
namespace\n - mvtx_name\n dimensions:\n - name: pod\n #
expr: optional expression for prometheus query\n # overrides the default
expression\n filters:\n - name: pod\n required:
false\n - name: period\n required: false\n - name:
mono-vertex\n # expr: optional expression for prometheus query\n #
overrides the default expression\n filters:\n - name: period\n
\ required: false\n\n- name: mono_vertex_histogram\n objects: \n
\ - mono-vertex\n title: MonoVertex Histogram Metrics\n description: This
pattern is for P99, P95, P90 and P50 quantiles for a mono-vertex across different
dimensions\n expr: |\n histogram_quantile($quantile, sum by($dimension,le)
(rate($metric_name{$filters}[$duration])))\n params:\n - name: quantile\n
\ required: true\n - name: duration\n required: true\n - name:
Expand Down Expand Up @@ -216,37 +217,38 @@ data:
behaviour\n # unit: Units\n required_filters:\n - namespace\n
\ - mvtx_name\n dimensions:\n - name: mono-vertex\n -
name: pod\n filters:\n - name: pod\n required:
false\n- name: pod_cpu_memory_utilization\n objects: \n - mono-vertex\n -
vertex\n title: cpu-memory utilization by pod\n description: cpu and memory
utilization by pod for mono-vertex\n expr: avg_over_time($metric_name{$filters}[$duration])\n
\ params:\n - name: duration\n required: true\n - name: start_time\n
\ required: false\n - name: end_time\n required: false\n metrics:
\n # set your cpu metric name here\n - metric_name: namespace_pod_cpu_utilization\n
\ # set display name as per metric name\n display_name: CPU Utilization
per Pod\n metric_description: This metric represents the percentage utilization
of cpu usage over cpu resource limits for a pod.\n required_filters:\n -
namespace\n - pod \n dimensions:\n - name: mono-vertex\n filters:
false\n\n- name: pod_cpu_memory_utilization\n objects: \n - mono-vertex\n
\ - vertex\n title: CPU and Memory Utilisation by Pod\n description: This
pattern represents the CPU and Memory utilisation by pod for mono-vertex and vertex\n
\ expr: avg_over_time($metric_name{$filters}[$duration])\n params:\n - name:
duration\n required: true\n - name: start_time\n required: false\n
\ - name: end_time\n required: false\n metrics: \n # set your cpu metric
name here\n - metric_name: namespace_pod_cpu_utilization\n # set display
name as per metric name\n display_name: CPU Utilization per Pod\n metric_description:
This metric represents the percentage utilization of cpu usage over cpu resource
limits for a pod.\n required_filters:\n - namespace\n - pod
\ \n dimensions:\n - name: mono-vertex\n filters: \n -
name: pod\n # expr: optional expression for prometheus query\n #
overrides the default expression\n required: false\n - name:
vertex\n filters: \n - name: pod\n # expr: optional
expression for prometheus query\n # overrides the default expression
\n required: false\n # set your memory metric name here\n -
metric_name: namespace_pod_memory_utilization\n # set display name as per
metric name\n display_name: Memory Utilization per Pod\n metric_description:
This metric represents the percentage utilization of memory usage in bytes over
memory resource limits for a pod.\n required_filters:\n - namespace\n
\ - pod \n dimensions:\n - name: mono-vertex\n filters:
\n - name: pod\n # expr: optional expression for prometheus
query\n # overrides the default expression\n required:
false\n - name: vertex\n filters: \n - name: pod\n
\ # expr: optional expression for prometheus query\n #
overrides the default expression \n required: false\n # set your
memory metric name here\n - metric_name: namespace_pod_memory_utilization\n
\ # set display name as per metric name\n display_name: Memory Utilization
per Pod\n metric_description: This metric represents the percentage utilization
of memory usage in bytes over memory resource limits for a pod.\n required_filters:\n
\ - namespace\n - pod \n dimensions:\n - name: mono-vertex\n
\ filters: \n - name: pod\n # expr: optional expression
for prometheus query\n # overrides the default expression \n required:
query\n # overrides the default expression \n required:
false\n - name: vertex\n filters: \n - name: pod\n
\ # expr: optional expression for prometheus query\n #
overrides the default expression \n required: false\n- name: container_cpu_memory_utilization\n
\ objects: \n - mono-vertex\n - vertex\n title: cpu-memory utilization
by container for mono-vertex\n description: cpu and memory utilization by container
for mono-vertex\n expr: avg_over_time($metric_name{$filters}[$duration])\n params:\n
\ - name: duration\n required: true\n - name: start_time\n required:
false\n - name: end_time\n required: false\n metrics:\n # set your
cpu metric name here\n - metric_name: namespace_app_container_cpu_utilization\n
overrides the default expression \n required: false\n\n- name: container_cpu_memory_utilization\n
\ objects: \n - mono-vertex\n - vertex\n title: CPU and Memory Utilisation
by Container\n description: This pattern represents the CPU and Memory utilisation
by container for mono-vertex and vertex\n expr: avg_over_time($metric_name{$filters}[$duration])\n
\ params:\n - name: duration\n required: true\n - name: start_time\n
\ required: false\n - name: end_time\n required: false\n metrics:\n
\ # set your cpu metric name here\n - metric_name: namespace_app_container_cpu_utilization\n
\ # set display name as per metric name\n display_name: CPU Utilization
per Container\n metric_description: This metric represents the percentage
utilization of cpu usage over cpu resource limits for a container.\n required_filters:\n
Expand Down
Loading
Loading