otelconf: add env var substitution in ParseYAML #6215

codeboten · 2024-10-07T22:38:20Z

This replaces the last bit of functionality that was opened in #4826 to support env variable replacement. Pulled the envprovider.go code from https://github.com/open-telemetry/opentelemetry-collector/blob/main/confmap/provider/envprovider/provider.go

Fixes #4373

config/envprovider.go

config/config.go

MrAlias · 2025-02-20T18:53:43Z

Blocked by #6796: #6796 (comment)

This replaces the last bit of functionality that was opened in open-telemetry#4826 to support env variable replacement. Pulled the envprovider.go code from https://github.com/open-telemetry/opentelemetry-collector/blob/main/confmap/provider/envprovider/provider.go Signed-off-by: Alex Boten <[email protected]>

Signed-off-by: Alex Boten <[email protected]>

codeboten · 2025-03-04T23:19:15Z

@MrAlias moved the code to the new package, this is no longer blocked by that. There's other comments to address that I'll get back to this week

Signed-off-by: Alex Boten <[email protected]>

codecov · 2025-03-04T23:24:30Z

Codecov Report

Attention: Patch coverage is 90.32258% with 6 lines in your changes missing coverage. Please review.

Project coverage is 75.8%. Comparing base (5ffb7ff) to head (1bc772a).
Report is 445 commits behind head on main.

Files with missing lines	Patch %	Lines
otelconf/v0.2.0/config.go	40.0%	2 Missing and 1 partial ⚠️
otelconf/v0.3.0/config.go	40.0%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@          Coverage Diff          @@
##            main   #6215   +/-   ##
=====================================
  Coverage   75.8%   75.8%           
=====================================
  Files        207     208    +1     
  Lines      19494   19554   +60     
=====================================
+ Hits       14783   14837   +54     
- Misses      4275    4279    +4     
- Partials     436     438    +2

Files with missing lines	Coverage Δ
otelconf/internal/provider/envprovider.go	`100.0% <100.0%> (ø)`
otelconf/v0.2.0/config.go	`76.9% <40.0%> (-3.5%)`	⬇️
otelconf/v0.3.0/config.go	`87.2% <40.0%> (-2.6%)`	⬇️

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Alex Boten <[email protected]>

dmathieu · 2025-03-05T08:12:24Z

It is still blocked, until the release is cut. We agreed config and otelconf would remain in sync for this release.

codeboten · 2025-03-05T15:30:39Z

@dmathieu thanks for clarifying, with the next release started I don't suspect this change will be merged by then anyways :)

Signed-off-by: Alex Boten <[email protected]>

linux-foundation-easycla · 2025-03-05T19:42:40Z

The committers listed above are authorized under a signed CLA.

✅ login: codeboten / name: Alex Boten (324c779, 227190f, 7d530dd, 80f7997, 99b0ce3, 6f8fe8d, 3743682, 410ec86, eedcd05, 38be890, c1c262c, 62b03d1, 58915db, aeb26d7, 26d809b, 8a47562, 319f0b7, 1bc772a, 569f610, 4fbb72a, 0ad6643, 4788612, 5723ff5, ce0ffbc, 5e0cd7d, a2f364a)
✅ login: pellared / name: Robert Pająk (b92684a)

codeboten · 2025-03-05T21:45:31Z

CLA faillures is being investigated linuxfoundation/easycla#4605

jarias-lfx · 2025-03-06T13:47:56Z

/easycla

CHANGELOG.md

Co-authored-by: Damien Mathieu <[email protected]>

codeboten · 2025-03-18T15:39:17Z

@pellared would love a review from you as you've requested changes which I made

pellared · 2025-03-26T06:09:21Z

@pellared would love a review from you as you've requested changes which I made

I will do my best to review this week. Thanks a lot.

#6215 (comment)

otelconf/internal/provider/envprovider.go

CHANGELOG.md

Co-authored-by: Robert Pająk <[email protected]>

Signed-off-by: Alex Boten <[email protected]>

pellared · 2025-03-27T19:21:01Z

otelconf/internal/provider/envprovider_test.go

+	}
+}
+
+func TestReplaceEnvVars(t *testing.T) {


Can we add all test cases from https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/configuration/data-model.md#environment-variable-substitution (the ones that are in the table)?

Added a bunch more tests, still need to ensure all the scenarios are covered and passing, but it's underway

Signed-off-by: Alex Boten <[email protected]>

CHANGELOG.md

otelconf/v0.2.0/config.go

Co-authored-by: Robert Pająk <[email protected]>

CHANGELOG.md

Co-authored-by: Robert Pająk <[email protected]>

pellared

Overall looks very good 👍

I would like only this to get resolved as e.g. I want to have tests that ensures that $$ are escaped correctly 😉

Signed-off-by: Alex Boten <[email protected]>

codeboten requested review from pellared and a team as code owners October 7, 2024 22:38

dmathieu reviewed Oct 8, 2024

View reviewed changes

config/envprovider.go Outdated Show resolved Hide resolved

dmathieu approved these changes Oct 11, 2024

View reviewed changes

pellared reviewed Oct 11, 2024

View reviewed changes

config/envprovider.go Outdated Show resolved Hide resolved

pellared previously requested changes Oct 11, 2024

View reviewed changes

config/config.go Outdated Show resolved Hide resolved

codeboten force-pushed the codeboten/env-var-parsing branch from e08dc29 to 7fc2745 Compare November 5, 2024 23:01

MrAlias added the area: file-config Related to file-based configuration label Feb 20, 2025

codeboten added 3 commits March 4, 2025 13:54

changelog

6f8fe8d

Signed-off-by: Alex Boten <[email protected]>

move code to otelconf

99b0ce3

Signed-off-by: Alex Boten <[email protected]>

codeboten force-pushed the codeboten/env-var-parsing branch from 7fc2745 to 99b0ce3 Compare March 4, 2025 23:18

changelog

7d530dd

Signed-off-by: Alex Boten <[email protected]>

add tests for provider type validation

eedcd05

Signed-off-by: Alex Boten <[email protected]>

codeboten added 4 commits March 5, 2025 09:39

update code to return an error if env var parsing errors out

4fbb72a

Signed-off-by: Alex Boten <[email protected]>

Merge branch 'main' into codeboten/env-var-parsing

324c779

clean up empty value

26d809b

Signed-off-by: Alex Boten <[email protected]>

appease linters

80f7997

Signed-off-by: Alex Boten <[email protected]>

Merge branch 'main' into codeboten/env-var-parsing

62b03d1

dmathieu reviewed Mar 7, 2025

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

codeboten and others added 2 commits March 7, 2025 07:12

Update CHANGELOG.md

aeb26d7

Co-authored-by: Damien Mathieu <[email protected]>

Update CHANGELOG.md

38be890

codeboten added 3 commits March 7, 2025 07:17

Merge branch 'main' into codeboten/env-var-parsing

58915db

Merge branch 'main' into codeboten/env-var-parsing

5e0cd7d

Merge branch 'main' into codeboten/env-var-parsing

ce0ffbc

Merge branch 'main' into codeboten/env-var-parsing

3743682

pellared reviewed Mar 26, 2025

View reviewed changes

otelconf/internal/provider/envprovider.go Outdated Show resolved Hide resolved

pellared reviewed Mar 26, 2025

View reviewed changes

otelconf/internal/provider/envprovider.go Outdated Show resolved Hide resolved

pellared reviewed Mar 26, 2025

View reviewed changes

otelconf/internal/provider/envprovider.go Outdated Show resolved Hide resolved

pellared reviewed Mar 26, 2025

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

pellared changed the title ~~config: add support for parsing env variables in configuration~~ config: add env var substitution in ParseYAML Mar 26, 2025

codeboten and others added 3 commits March 27, 2025 11:40

Apply suggestions from code review

0ad6643

Co-authored-by: Robert Pająk <[email protected]>

Merge branch 'main' into codeboten/env-var-parsing

c1c262c

add package comment

227190f

Signed-off-by: Alex Boten <[email protected]>

pellared reviewed Mar 27, 2025

View reviewed changes

codeboten added 2 commits March 27, 2025 12:50

remove code for len check

4788612

Signed-off-by: Alex Boten <[email protected]>

added check for empty data in checkRawConfType

5723ff5

Signed-off-by: Alex Boten <[email protected]>

pellared reviewed Mar 27, 2025

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

otelconf/v0.2.0/config.go Show resolved Hide resolved

pellared mentioned this pull request Mar 27, 2025

otelconf: Improve package documentation and add testable examples #7037

Open

Update CHANGELOG.md

319f0b7

Co-authored-by: Robert Pająk <[email protected]>

pellared reviewed Mar 27, 2025

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

codeboten and others added 2 commits March 27, 2025 13:19

Update CHANGELOG.md

8a47562

Co-authored-by: Robert Pająk <[email protected]>

Update CHANGELOG.md

b92684a

pellared reviewed Mar 27, 2025

View reviewed changes

pellared changed the title ~~config: add env var substitution in ParseYAML~~ otelconf: add env var substitution in ParseYAML Mar 27, 2025

codeboten added 3 commits March 27, 2025 14:41

Merge branch 'main' into codeboten/env-var-parsing

a2f364a

add more tests, ensure $$ is escaped

569f610

Signed-off-by: Alex Boten <[email protected]>

set int value type

1bc772a

Signed-off-by: Alex Boten <[email protected]>

otelconf: add env var substitution in ParseYAML #6215

Are you sure you want to change the base?

otelconf: add env var substitution in ParseYAML #6215

Uh oh!

Conversation

codeboten commented Oct 7, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MrAlias commented Feb 20, 2025

Uh oh!

codeboten commented Mar 4, 2025

Uh oh!

codecov bot commented Mar 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

dmathieu commented Mar 5, 2025

Uh oh!

codeboten commented Mar 5, 2025

Uh oh!

linux-foundation-easycla bot commented Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codeboten commented Mar 5, 2025

Uh oh!

jarias-lfx commented Mar 6, 2025

Uh oh!

Uh oh!

codeboten commented Mar 18, 2025

Uh oh!

pellared commented Mar 26, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pellared Mar 27, 2025

Choose a reason for hiding this comment

Uh oh!

codeboten Mar 27, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pellared left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov bot commented Mar 4, 2025 •

edited

Loading

linux-foundation-easycla bot commented Mar 5, 2025 •

edited

Loading