Add basic (single) pipeline bucket stats. #9599

dwelsch-esi · 2025-04-11T00:19:31Z

Description

Add:

avg_bucket
min_bucket
max_bucket
sum_bucket

These pages are almost identical. Example is reused; only the stat is change.

Dependent on addition of the /_aggregations/pipeline directory. See #9598.

Issues Resolved

Version

Frontend features

Checklist

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Dave Welsch <[email protected]>

github-actions · 2025-04-11T00:19:43Z

Thank you for submitting your PR. The PR states are In progress (or Draft) -> Tech review -> Doc review -> Editorial review -> Merged.

Before you submit your PR for doc review, make sure the content is technically accurate. If you need help finding a tech reviewer, tag a maintainer.

When you're ready for doc review, tag the assignee of this PR. The doc reviewer may push edits to the PR directly or leave comments and editorial suggestions for you to address (let us know in a comment if you have a preference). The doc reviewer will arrange for an editorial review.

kolchfa-aws · 2025-04-11T14:56:36Z

@jainankitk Could you please review this PR? Thanks!

kolchfa-aws · 2025-04-11T14:57:36Z

@dwelsch-esi Could you remove the info related to these aggregations from the pipeline aggregation index page?

dwelsch-esi · 2025-04-11T15:50:33Z

@dwelsch-esi Could you remove the info related to these aggregations from the pipeline aggregation index page?

@kolchfa-aws

Not sure which info you're referring to here?

I removed the old pipeline aggregation page (_aggregations/pipeline-agg.md) in PR#9598. Its replacement, _aggregations/pipeline/index.md, cites these aggregations as examples of sibling type pipeline aggregations, and sum_buckets is used in an example illustrating the buckets_path syntax. Do you want me to remove these examples for some reason?

kolchfa-aws · 2025-04-11T20:22:19Z

@dwelsch-esi I initially missed that you replaced the pipeline-agg with pipeine/index in #9598. Please ignore. The link checker is failing because #9598 is not merged yet.

_aggregations/pipeline/avg-bucket.md

_aggregations/pipeline/max-bucket.md

_aggregations/pipeline/min-bucket.md

_aggregations/pipeline/sum-bucket.md

jainankitk · 2025-05-08T22:36:18Z

_aggregations/pipeline/avg-bucket.md

+parent: Pipeline aggregations
+nav_order: 10
+redirect_from:
+  - /query-dsl/aggregations/pipeline-agg#avg_bucket-sum_bucket-min_bucket-max_bucket/


I am assuming this redirect is part of sibling-aggregations? Currently the indexed path is https://docs.opensearch.org/docs/latest/aggregations/pipeline/index/#sibling-aggregations. This does not really look consistent with it

This is only needed for moved pages. Since this is a new page, removing the redirect.

jainankitk · 2025-05-08T22:38:27Z

_aggregations/pipeline/avg-bucket.md

+| Parameter             | Required/Optional | Data type       | Description |
+| :--                   | :--               |  :--            | :--         |
+| `buckets_path`        | Required          | String          | The path of the aggregation buckets to be aggregated. See [Pipeline aggregations]({{site.url}}{{site.baseurl}}/aggregations/pipeline/index#pipeline-aggregation-syntax). |
+| `gap_policy`          | Optional          | String          | The policy to apply to missing data. Valid values are `skip`, `insert_zeros`, and `keep_values`. Default is `skip`. |


While skip and insert_zeros seem intuitive, I am wondering what keep_values mean?

keep_values is not a valid parameter. Removing.

jainankitk · 2025-05-08T22:40:20Z

_aggregations/pipeline/avg-bucket.md

+
+## Example
+
+The following example creates a date histogram with a one-month interval from the OpenSearch Dashboards e-commerce sample data. The `sum` sub-aggregation calculates the sum of all bytes for each month. Finally, the `avg_bucket` aggregation calculates the average number of bytes per month from these sums:


average bytes per month reads better instead of average number of bytes per month

sum of bytes reads better instead of sum of all bytes

jainankitk · 2025-05-08T22:43:07Z

_aggregations/pipeline/max-bucket.md

+
+## Example response
+
+The aggregation returns the maximum number of bytes from the monthly buckets:


While it is reasonably intuitive, should explicitly specify what keys indicate. For example - it is array because more than one key might have same value as max

Added an extended description in both max and min bucket pages.

Signed-off-by: Fanit Kolchina <[email protected]>

kolchfa-aws · 2025-05-19T23:06:43Z

@jainankitk Thank you for the review! I addressed your comments.

natebower

@kolchfa-aws Please see my comments and changes and let me know if you have any questions. Thanks!

_aggregations/pipeline/avg-bucket.md

_aggregations/pipeline/max-bucket.md

_aggregations/pipeline/avg-bucket.md

_aggregations/pipeline/max-bucket.md

natebower · 2025-05-21T11:11:52Z

_aggregations/pipeline/max-bucket.md

+
+## Example response
+
+The `max_bucket` aggregation returns the maximum value from a specified metric across multiple buckets. In this example, it calculates the maximum number of bytes per month from the `sum_of_bytes` metric inside `visits_per_month`. The `value` field shows the maximum value found across all buckets. The `keys` array contains the bucket keys where this maximum value was observed. It's an array because more than one bucket can have the same maximum value. In such cases, all matching bucket keys are included. This ensures the result is accurate even if multiple time periods (or terms) tied for the maximum:


4th sentence: Something like "The keys array contains the keys of the buckets in which this maximum value was observed"? Last sentence: "This ensures that the result is accurate even if multiple time periods (or terms) have the same maximum value"?

_aggregations/pipeline/min-bucket.md

natebower · 2025-05-21T11:13:30Z

_aggregations/pipeline/min-bucket.md

+
+## Example response
+
+The `max_bucket` aggregation returns the minimum value from a specified metric across multiple buckets. In this example, it calculates the minimum number of bytes per month from the `sum_of_bytes` metric inside `visits_per_month`. The `value` field shows the minimum value found across all buckets. The `keys` array contains the bucket keys where this minimum value was observed. It's an array because more than one bucket can have the same minimum value. In such cases, all matching bucket keys are included. This ensures the result is accurate even if multiple time periods (or terms) tied for the minimum:


Same comment as in previous file re: rephrasing

_aggregations/pipeline/sum-bucket.md

_aggregations/pipeline/max-bucket.md

_aggregations/pipeline/min-bucket.md

Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]>

* Add basic (single) pipeline bucket stats: avg, sum, min, max. Signed-off-by: Dave Welsch <[email protected]> * Doc review Signed-off-by: Fanit Kolchina <[email protected]> * Apply suggestions from code review Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> --------- Signed-off-by: Dave Welsch <[email protected]> Signed-off-by: Fanit Kolchina <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> Co-authored-by: Fanit Kolchina <[email protected]> Co-authored-by: kolchfa-aws <[email protected]> Co-authored-by: Nathan Bower <[email protected]> (cherry picked from commit 0795814) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

Add basic (single) pipeline bucket stats: avg, sum, min, max.

6354fbc

Signed-off-by: Dave Welsch <[email protected]>

dwelsch-esi requested review from kolchfa-aws, Naarcha-AWS, AMoo-Miki, natebower, dlvenable and epugh as code owners April 11, 2025 00:19

github-actions bot assigned kolchfa-aws Apr 11, 2025

kolchfa-aws added Content gap backport 2.19 3 - Tech review PR: Tech review in progress labels Apr 11, 2025

dwelsch-esi commented Apr 28, 2025

View reviewed changes

_aggregations/pipeline/avg-bucket.md Outdated Show resolved Hide resolved

dwelsch-esi commented Apr 28, 2025

View reviewed changes

_aggregations/pipeline/max-bucket.md Outdated Show resolved Hide resolved

dwelsch-esi commented Apr 28, 2025

View reviewed changes

_aggregations/pipeline/min-bucket.md Outdated Show resolved Hide resolved

dwelsch-esi commented Apr 28, 2025

View reviewed changes

_aggregations/pipeline/sum-bucket.md Outdated Show resolved Hide resolved

kolchfa-aws added backport 3.0 and removed backport 2.19 labels May 6, 2025

jainankitk reviewed May 8, 2025

View reviewed changes

This was referenced May 19, 2025

Add rare_terms bucket aggregation #9826

Merged

Add moving_avg pipeline aggregation. #9657

Merged

kolchfa-aws added 2 commits May 19, 2025 18:51

Merge branch 'main' into aggs-pipeline-sum-1

53493ff

Doc review

bd83057

Signed-off-by: Fanit Kolchina <[email protected]>

jainankitk approved these changes May 20, 2025

View reviewed changes

natebower reviewed May 21, 2025

View reviewed changes

kolchfa-aws reviewed May 21, 2025

View reviewed changes

_aggregations/pipeline/max-bucket.md Outdated Show resolved Hide resolved

kolchfa-aws reviewed May 21, 2025

View reviewed changes

_aggregations/pipeline/min-bucket.md Outdated Show resolved Hide resolved

Apply suggestions from code review

8fa1d19

Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]>

kolchfa-aws approved these changes May 21, 2025

View reviewed changes

Merge branch 'main' into aggs-pipeline-sum-1

bb71d48

kolchfa-aws merged commit 0795814 into opensearch-project:main May 21, 2025
5 checks passed

opensearch-trigger-bot bot mentioned this pull request May 21, 2025

[Backport 3.0] Add basic (single) pipeline bucket stats. #9950

Merged

github-actions bot pushed a commit that referenced this pull request May 21, 2025

Add basic (single) pipeline bucket stats. (#9599) (#9950)

99215e3


		## Example

		The following example creates a date histogram with a one-month interval from the OpenSearch Dashboards e-commerce sample data. The `sum` sub-aggregation calculates the sum of all bytes for each month. Finally, the `avg_bucket` aggregation calculates the average number of bytes per month from these sums:


		## Example response

		The aggregation returns the maximum number of bytes from the monthly buckets:


		## Example response

		The `max_bucket` aggregation returns the maximum value from a specified metric across multiple buckets. In this example, it calculates the maximum number of bytes per month from the `sum_of_bytes` metric inside `visits_per_month`. The `value` field shows the maximum value found across all buckets. The `keys` array contains the bucket keys where this maximum value was observed. It's an array because more than one bucket can have the same maximum value. In such cases, all matching bucket keys are included. This ensures the result is accurate even if multiple time periods (or terms) tied for the maximum:

Add basic (single) pipeline bucket stats. #9599

Add basic (single) pipeline bucket stats. #9599

Uh oh!

Conversation

dwelsch-esi commented Apr 11, 2025

Description

Issues Resolved

Version

Frontend features

Checklist

Uh oh!

github-actions bot commented Apr 11, 2025

Uh oh!

kolchfa-aws commented Apr 11, 2025

Uh oh!

kolchfa-aws commented Apr 11, 2025

Uh oh!

dwelsch-esi commented Apr 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kolchfa-aws commented Apr 11, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kolchfa-aws commented May 19, 2025

Uh oh!

natebower left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dwelsch-esi commented Apr 11, 2025 •

edited

Loading