re-enable aws tests #3733

wconti27 · 2024-12-19T19:55:04Z

Motivation

AWS tests were previously disabled due to flakiness problems. This PR reverts AWS tests to run locally instead of with actual AWS. The PR changes weblogs to accept an env variable SYSTEM_TESTS_AWS_URL, so that we can easily configure our AWS backend in the future. Re-enables the test cases to run

Changes

Workflow

⚠️ Create your PR as draft ⚠️
Work on you PR until the CI passes (if something not related to your task is failing, you can ignore it)
Mark it as ready for review
- Test logic is modified? -> Get a review from RFC owner. We're working on refining the codeowners file quickly.
- Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

If PR title starts with [<language>], double-check that only <language> is impacted by the change
No system-tests internal is modified. Otherwise, I have the approval from R&P team
CI is green, or failing jobs are not related to this change (and you are 100% sure about this statement)
A docker base image is modified?
- the relevant build-XXX-image label is present
A scenario is added (or removed)?
- Get a review from R&P team

cbeauchesne

Could you resolve the conflict?

I'm sorry for the delay, I may have missed this PR

wconti27 · 2025-02-03T20:58:24Z

Could you resolve the conflict?

I'm sorry for the delay, I may have missed this PR

Resolved! And no worries, but we should definitely get this in soon!

cbeauchesne

Also, CI is failing

utils/_context/containers.py

utils/_context/_scenarios/integrations.py

utils/_context/containers.py

utils/_context/_scenarios/integrations.py

cbeauchesne

Just a small questions. If the change was on purpose, AGTM

re-enable aws integrations tests (#3733) feat(tests/test_data_integrity): update test_datadog_external_env for Go v1.72.0 and forward (#3961) Activate ruff rules on tests/ folder (#3999) [python] Avoid passing global tracer to pin in weblog apps (#4004) All classes must declare feature ids (#4003) Extend mypy scope (#4002) Onboarding: bug marker profiling (#4005) Docker SSI: fix scenario (#4006) [ruby] Enable IP blocking tests for Ruby (#3937) Activate ruff rules on tests/ folder (#4007) [nodejs] remove auto login event skip (#3998) Email HTML Injection detection in IAST Java (#3906) Co-authored-by: Mario Vidal Domínguez <[email protected]> Add test to check absence of client computed stats (#3812) [java] Skip payara/CROSSED_TRACING_LIBRARIES/prod (#4009) Add GraphQL error extension tests (#3986) Co-authored-by: William Conti <[email protected]> Use prod agent, dev agent broke lot of tracers (#4011) update xpassing baggage tests for unimplemented languages (#3773) [NodeJS] skip failing baggage tests (#4015) [python] fix 500 errors in sql queries (#3997) Activate ruff rules on tests/ folder (#4010) Hotfix Fix fuzzer [Nodejs] Enable untrusted deserialization stack trace test for Node.js (#3995) [python] use main again for dev branch (#4008) Co-authored-by: erikayasuda <[email protected]> Co-authored-by: Charles de Beauchesne <[email protected]> Revert agent dev fix (#4013) [PHP] Enable rasp telemetry tests for PHP (#3972) [skipci] Update CODEOWNERS for static files (#4012) [Java] Enable more easy wins (#4018) [java] Bump GraalVM system test to JDK 22 (#4001) [NodeJS] skip more failing baggage tests (#4021) [Debugger] Update dotnet Exception Replay tests (#3974) Test multiple rasp during one request (#3989) Add test for location extended data (#3978) Fix APPSEC_NO_STATS scenario name (#4019) Avoid false XPASS on APPSEC_WAF_TELEMETRY (#4029) [java] Enable Test_Blocking_strip_response_headers in some variants (#4033) [java] Remove some outdated manifest entries (#4039) [java] Fix xpass for Test_SecurityEvents_Appsec_Metastruct_Disabled (#4038) Consolidate remote config tests into same directory/file (#4031) [python] use last patch version of python for django weblogs (#4025) crashtracking: assert si_signo is set to 11 (#4023)

Extend vulnerability location data with class re-enable aws integrations tests (#3733) feat(tests/test_data_integrity): update test_datadog_external_env for Go v1.72.0 and forward (#3961) Activate ruff rules on tests/ folder (#3999) [python] Avoid passing global tracer to pin in weblog apps (#4004) All classes must declare feature ids (#4003) Extend mypy scope (#4002) Onboarding: bug marker profiling (#4005) Docker SSI: fix scenario (#4006) [ruby] Enable IP blocking tests for Ruby (#3937) Activate ruff rules on tests/ folder (#4007) [nodejs] remove auto login event skip (#3998) Email HTML Injection detection in IAST Java (#3906) Co-authored-by: Mario Vidal Domínguez <[email protected]> Add test to check absence of client computed stats (#3812) [java] Skip payara/CROSSED_TRACING_LIBRARIES/prod (#4009) Add GraphQL error extension tests (#3986) Co-authored-by: William Conti <[email protected]> Use prod agent, dev agent broke lot of tracers (#4011) update xpassing baggage tests for unimplemented languages (#3773) [NodeJS] skip failing baggage tests (#4015) [python] fix 500 errors in sql queries (#3997) Activate ruff rules on tests/ folder (#4010) Hotfix Fix fuzzer [Nodejs] Enable untrusted deserialization stack trace test for Node.js (#3995) [python] use main again for dev branch (#4008) Co-authored-by: erikayasuda <[email protected]> Co-authored-by: Charles de Beauchesne <[email protected]> Revert agent dev fix (#4013) [PHP] Enable rasp telemetry tests for PHP (#3972) [skipci] Update CODEOWNERS for static files (#4012) [Java] Enable more easy wins (#4018) [java] Bump GraalVM system test to JDK 22 (#4001) [NodeJS] skip more failing baggage tests (#4021) [Debugger] Update dotnet Exception Replay tests (#3974) Test multiple rasp during one request (#3989) Add test for location extended data (#3978) Fix APPSEC_NO_STATS scenario name (#4019) Avoid false XPASS on APPSEC_WAF_TELEMETRY (#4029) [java] Enable Test_Blocking_strip_response_headers in some variants (#4033) [java] Remove some outdated manifest entries (#4039) [java] Fix xpass for Test_SecurityEvents_Appsec_Metastruct_Disabled (#4038) Consolidate remote config tests into same directory/file (#4031) [python] use last patch version of python for django weblogs (#4025) crashtracking: assert si_signo is set to 11 (#4023) # This is the commit message #2: [nodejs] Fix get_target_branch - "quotes, parens and # (#3952)"" (#3953) Co-authored-by: Charles de Beauchesne <[email protected]> # This is the commit message #3: Fix Standalone Billing values (#3965) # This is the commit message #4: Add success to valid WAF telemetry tags (#3966) # This is the commit message #5: Ensure tracer release coherence (#3967) [Nodejs] Code injection telemetry metric (#3959) Do not use special chars in "redacted" string used in scrubber (#3977) fix Context Propagation: Extraction feature_id (#3970) [dotnet] Enable Custom Blocking Response tests (#3971) [python] update flask base image (#3979) [ruby] Skip failing test for APMAPI-1141 (#3980) update weblog sampling_test use sampling rules instead of deprecated envvar (#3984) [python] enable Python IAST+SSRF tests (#2512) Signed-off-by: Juanjo Alvarez <[email protected]> [python] IAST Header source in werkzeug 3.1 (#3991) [python] switch flaky decorator for bug (#3990) [python] update versions for major release (#3993) [test optimization] Update config shape sent as telemetry (#3992) Add profiling scenarios to Python onboarding tests (#3002) Update changelog re-enable aws integrations tests (#3733) feat(tests/test_data_integrity): update test_datadog_external_env for Go v1.72.0 and forward (#3961) [python] Avoid passing global tracer to pin in weblog apps (#4004) All classes must declare feature ids (#4003) Extend mypy scope (#4002) Onboarding: bug marker profiling (#4005) Docker SSI: fix scenario (#4006) [ruby] Enable IP blocking tests for Ruby (#3937) [nodejs] remove auto login event skip (#3998) [NodeJS] skip failing baggage tests (#4015) [python] fix 500 errors in sql queries (#3997) Hotfix Fix fuzzer [python] use main again for dev branch (#4008) Co-authored-by: erikayasuda <[email protected]> Co-authored-by: Charles de Beauchesne <[email protected]> Revert agent dev fix (#4013) [skipci] Update CODEOWNERS for static files (#4012) [Java] Enable more easy wins (#4018) [java] Bump GraalVM system test to JDK 22 (#4001) [NodeJS] skip more failing baggage tests (#4021) [Debugger] Update dotnet Exception Replay tests (#3974) Test multiple rasp during one request (#3989) Add test for location extended data (#3978) Fix APPSEC_NO_STATS scenario name (#4019) Avoid false XPASS on APPSEC_WAF_TELEMETRY (#4029) [java] Enable Test_Blocking_strip_response_headers in some variants (#4033) [java] Remove some outdated manifest entries (#4039) [java] Fix xpass for Test_SecurityEvents_Appsec_Metastruct_Disabled (#4038) Consolidate remote config tests into same directory/file (#4031) [python] use last patch version of python for django weblogs (#4025) crashtracking: assert si_signo is set to 11 (#4023) class vs path

initial commit

529b3c5

wconti27 requested review from a team as code owners December 19, 2024 19:55

wconti27 requested review from Kyle-Verhoog, wantsui, manuel-alvarez-alvarez and Mariovido and removed request for a team December 19, 2024 19:55

wconti27 marked this pull request as draft December 19, 2024 19:55

wconti27 self-assigned this Dec 19, 2024

wconti27 changed the title ~~re-enabled aws tests~~ re-enable aws tests Dec 20, 2024

wconti27 added 7 commits December 20, 2024 10:42

fix lint

856c9bd

fix lint again

f52551f

Merge branch 'main' into conti/re-enable-aws-tests

a025562

fix format

8a4f9a4

fix js format

9f16bc8

fix container

03f2f5f

fix builds

11a54c6

wconti27 added the build-buddies-images label Dec 20, 2024

wconti27 added 4 commits December 20, 2024 15:26

enable all tests

4a0d6df

remove delete resources calls

f500235

fix python and nodejs aws tests

32f327a

fix lint

cca3707

wconti27 and others added 7 commits December 23, 2024 10:22

fix lint

1609e2b

add env for buddy

185d694

fix python buddy

2c5e23c

fix formatting

a15f778

disable sqs xray test

a0238d4

fix failing python tests

3ed3971

Merge branch 'main' into conti/re-enable-aws-tests

1d289fa

wconti27 marked this pull request as ready for review January 6, 2025 17:22

cbeauchesne requested changes Jan 22, 2025

View reviewed changes

Merge branch 'main' into conti/re-enable-aws-tests

58e2cc8

wconti27 requested a review from a team as a code owner February 3, 2025 20:58

wconti27 requested a review from cbeauchesne February 3, 2025 20:58

cbeauchesne requested changes Feb 4, 2025

View reviewed changes

utils/_context/containers.py Outdated Show resolved Hide resolved

utils/_context/_scenarios/integrations.py Outdated Show resolved Hide resolved

utils/_context/containers.py Outdated Show resolved Hide resolved

wconti27 and others added 3 commits February 4, 2025 12:31

address reviewer comments

d59b757

Merge branch 'main' into conti/re-enable-aws-tests

52c7c41

fix

d0f6efa

cbeauchesne reviewed Feb 5, 2025

View reviewed changes

utils/_context/_scenarios/integrations.py Show resolved Hide resolved

cbeauchesne approved these changes Feb 5, 2025

View reviewed changes

wconti27 merged commit 0dc6c94 into main Feb 5, 2025
435 of 436 checks passed

wconti27 deleted the conti/re-enable-aws-tests branch February 5, 2025 16:16

robertomonteromiguel pushed a commit that referenced this pull request Mar 17, 2025

re-enable aws integrations tests (#3733)

be1bf5f

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

re-enable aws tests #3733

re-enable aws tests #3733

wconti27 commented Dec 19, 2024 •

edited

Loading

cbeauchesne left a comment

wconti27 commented Feb 3, 2025

cbeauchesne left a comment

cbeauchesne left a comment

re-enable aws tests #3733

re-enable aws tests #3733

Conversation

wconti27 commented Dec 19, 2024 • edited Loading

Motivation

Changes

Workflow

Reviewer checklist

cbeauchesne left a comment

Choose a reason for hiding this comment

wconti27 commented Feb 3, 2025

cbeauchesne left a comment

Choose a reason for hiding this comment

cbeauchesne left a comment

Choose a reason for hiding this comment

wconti27 commented Dec 19, 2024 •

edited

Loading