Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(amazonq): add retries to flaky e2e tests #6757

Merged
merged 1 commit into from
Mar 10, 2025

Conversation

jpinkney-aws
Copy link
Contributor

Problem

  • The inline tests are flaky when calling the backend and showing the result
  • Chat tests are flaky when doing simple

It's not entirely clear why the chat tests are flaky. They just started being flaky after 0989414 was merged, which is completely unrelated

Solution

  • Add upto 3 retries

  • Treat all work as PUBLIC. Private feature/x branches will not be squash-merged at release time.
  • Your code changes must meet the guidelines in CONTRIBUTING.md.
  • License: I confirm that my contribution is made under the terms of the Apache 2.0 license.

@jpinkney-aws jpinkney-aws requested a review from a team as a code owner March 10, 2025 15:39
@jpinkney-aws
Copy link
Contributor Author

/runIntegrationTests

1 similar comment
@jpinkney-aws
Copy link
Contributor Author

/runIntegrationTests

@jpinkney-aws jpinkney-aws merged commit 514e78d into aws:master Mar 10, 2025
31 of 32 checks passed
chungjac added a commit to chungjac/aws-toolkit-vscode that referenced this pull request Mar 11, 2025
fix endpoint

refactor(cw): remove unused keep-alive workaround.  (aws#6694)

This code is no longer used, and appears to have been moved to the LSP
side:
aws/aws-toolkit-vscode-staging#1214 (comment)

This fix also no longer works, and the work it was based off has
significantly changed their approach:
https://github.com/sourcegraph/cody/blob/62d73f78c432036d1f99bc9631ed534cc2ed846b/vscode/src/net/net.patch.ts

- remove it.

---

- Treat all work as PUBLIC. Private `feature/x` branches will not be
squash-merged at release time.
- Your code changes must meet the guidelines in
[CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines).
- License: I confirm that my contribution is made under the terms of the
Apache 2.0 license.

test(e2e): increase file watcher count to prevent ENOSPC errors

test(amazonq): retry inline tests if no valid response was found

Merge pull request aws#6757 from jpinkney-aws/stabilize-test

test(amazonq): add retries to flaky e2e tests

telemetry(amazonq): status field tracking user actions aws#6750

- The current unit test generation telemetry event lacks a field to
distinguish between user actions: acceptance, rejection, failure, or
cancellation.

- Enhancing the amazonq_utgGenerateTests telemetry event by
incorporating a status field to capture user actions in toolkit metrics.
- Bumping telemetry version to `1.0.307`

fix(amazonq): incorrect zip entry path for file scans aws#6741

Reproduce steps:
1. Open a window with multiple workspace folders, for example
`workspaceFolders: [folder1, folder2]`
2. Open any file and run a code review on the active file, for example
`folder1/sample.py`
3. The zip entry will be created with the workspaceFolder duplicated,
for example `folder1/folder1/sample.py`

Explicitly set `includeWorkspaceFolder` to false when getting the
relative path of the file.

fix(amazonq): keyboard navigation in "Create a new prompt" overlay aws#6738

When a user selects "Create a new prompt", a mouse is required to 1)
select the prompt name input field and 2) click the Enter button. This
flow should be able to be completed with just the keyboard

Autofocus prompt input field, and allow submit on Enter key press.

fix(amazonq): duplicates in lsp getContextCommandPrompt aws#6756

There are two types of context shown in the context list:
1. the ones by explicit @
2. the ones by @workspace 1) has relativePaths with start/end line being
-1, 2) has relativePaths with non -1 start/end lines,
1) is doing dedupe with 2) but 1) also needs to dedupe itself

ci(jscpd): show hint message on detection of clones aws#6737

There appears to be cases where the fix pushed here fails:
aws#6572.
ex: aws#6735
- re-add the hint message for the case described here:
aws#6564

feat(amazonq): feedback system for internal users for /test aws#6748

- The unit test generation team lacks sufficient feedback to enhance the
quality of their produced tests.

- Implementing the feedback system for internal users to evaluate and
enhance unit test generation quality.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants