Skip to content

chore: update playwright in ci to run only on aria changes #4452

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 22 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
2d04a06
chore: update playwright in ci to run only on aria changes
nmerget Jul 1, 2025
9d48e75
fix: issue with wrong playwright path
nmerget Jul 2, 2025
98856b9
Merge branch 'main' into chore-run-tests-only-if-aria-changed
nmerget Jul 2, 2025
8d46833
fix: issue with wrong dependency
nmerget Jul 2, 2025
5f5edfb
Merge branch 'main' into chore-run-tests-only-if-aria-changed
nmerget Jul 2, 2025
e187166
fix: issue with playwright configs
nmerget Jul 2, 2025
418bc68
fix: issue with wrong playwright config
nmerget Jul 3, 2025
2e92df0
fix: issue with wrong playwright config
nmerget Jul 3, 2025
123eb06
fix: issue with wrong playwright config
nmerget Jul 3, 2025
ef5e593
fix: issue with checks done
nmerget Jul 3, 2025
eb0e156
test: changed aria
nmerget Jul 7, 2025
4d3c596
fix: issue with missing branch for git diff
nmerget Jul 7, 2025
1f8b470
chore: update check.ts to check against remote
nmerget Jul 7, 2025
962f964
auto update snapshots (#4477)
github-actions[bot] Jul 7, 2025
a172cce
chore: update playwright config
nmerget Jul 7, 2025
dbc6ec2
auto update snapshots (#4478)
github-actions[bot] Jul 7, 2025
aa32f0e
test: aria snapshot trigger
nmerget Jul 8, 2025
1917d80
test: without fetch repo
nmerget Jul 8, 2025
4b3ce81
chore: revert test changes
nmerget Jul 8, 2025
af9c7cd
auto update snapshots (#4482)
github-actions[bot] Jul 8, 2025
9d2815f
Merge branch 'main' into chore-run-tests-only-if-aria-changed
nmerget Jul 8, 2025
5b613e1
fix: issue with comparing aria snapshots in current branch
nmerget Jul 8, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .config/.jscpd.json
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
"**/test-results/**",
"**/tests/**",
"**/tsconfig.json",
"**/e2e/**/*.spec.ts",
"packages/components/scripts/post-build/components.ts",
"packages/components/src/components/**/*.spec.tsx",
"packages/components/src/components/**/index.html",
Expand Down
77 changes: 77 additions & 0 deletions .github/workflows/02-e2e-patternhub.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
name: 🎭 Playwright E2E

on:
workflow_call:
inputs:
version:
required: true
type: string
path:
required: false
type: string

permissions:
actions: write
contents: write

jobs:
playwright-patternhub:
name: 🧪🎭 - patternhub
runs-on: ubuntu-24.04 # Use Ubuntu 24.04 explicitly
container:
image: mcr.microsoft.com/playwright:v${{ inputs.version }}
options: --user 1001
steps:
- name: ⏬ Checkout repo
uses: actions/checkout@v4

- name: 🔄 Init Cache
uses: ./.github/actions/npm-cache

- name: ⏬ Download foundations build
uses: actions/download-artifact@v4
with:
name: db-ux-foundations-build
path: packages/foundations/build

- name: ⏬ Download output
uses: actions/download-artifact@v4
with:
name: db-ux-output
path: output

- name: ⏬ Download components styles build
uses: actions/download-artifact@v4
with:
name: db-ux-components-build
path: packages/components/build

- name: ⏬ Download patternhub
uses: actions/download-artifact@v4
with:
name: db-ux-patternhub
path: build-showcases/patternhub

# We test everything for patternhub and main branch
- name: 👩‍🔬 Test showcase with Playwright 🎭
shell: bash
env:
NEXT_PUBLIC_BASE_PATH: ${{ inputs.path }}
run: |
mkdir -p ./build-showcases${{ inputs.path }}
cp -RT ./build-showcases/patternhub ./build-showcases${{ inputs.path }}
npm run test:e2e --workspace=patternhub
- name: 🔣 Print GitHub Report
if: failure()
shell: bash
run: |
npx playwright merge-reports --reporter github ./blob-report
- name: 🆙 Upload test results
if: failure()
uses: actions/upload-artifact@v4
with:
name: patternhub-playwright-results
path: ./showcases/patternhub/test-results
retention-days: 30
3 changes: 2 additions & 1 deletion .github/workflows/02-e2e-regenerate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,8 @@ jobs:
cp -RT ./build-showcases/${{ inputs.type }} ./build-showcases${{ inputs.path }}
npm run regenerate:screenshots --workspace=patternhub
else
npm run regenerate:screenshots --workspace=react-showcase
npm run regenerate:visual-snapshots --workspace=react-showcase
npm run regenerate:aria-snapshots --workspace=react-showcase
fi
- name: 🆙 Upload components
Expand Down
77 changes: 56 additions & 21 deletions .github/workflows/02-e2e-showcases.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,16 @@ on:
showcase:
required: true
type: string
path:
required: false
type: string
outputs:
aria-snapshots-changed:
description: "If aria snapshots changed in current PR"
value: ${{ jobs.playwright-showcases.outputs.aria-snapshots-changed }}
aria-snapshots-changed-to-main:
description: "If aria snapshots changed to main branch"
value: ${{ jobs.playwright-showcases.outputs.aria-snapshots-changed-to-main }}
visual-snapshots-changed:
description: "If visual snapshots changed"
value: ${{ jobs.playwright-showcases.outputs.visual-snapshots-changed }}

permissions:
actions: write
Expand All @@ -27,11 +34,18 @@ jobs:
strategy:
fail-fast: false
matrix:
shardIndex: [1, 2, 3, 4, 5, 6]
shardTotal: [6]
shardIndex: [1, 2, 3, 4]
shardTotal: [4]
outputs:
aria-snapshots-changed: ${{ steps.aria-snapshots.outputs.aria-snapshots-changed }}
aria-snapshots-changed-to-main: ${{ steps.aria-snapshots.outputs.aria-snapshots-changed-to-main }}
visual-snapshots-changed: ${{ steps.visual-snapshots.outputs.visual-snapshots-changed }}
steps:
- name: ⏬ Checkout repo
uses: actions/checkout@v4
with:
fetch-depth: 0 # fetch all history for all branches
fetch-tags: false

- name: 🔄 Init Cache
uses: ./.github/actions/npm-cache
Expand Down Expand Up @@ -60,21 +74,50 @@ jobs:
name: db-ux-${{ inputs.showcase }}
path: build-showcases/${{ inputs.showcase }}

- name: 🧴 Copy patternhub testing app
if: inputs.path
shell: bash
run: |
mkdir -p ./build-showcases${{ inputs.path }}
cp -RT ./build-showcases/${{ inputs.showcase }} ./build-showcases${{ inputs.path }}
# We test everything for patternhub and main branch
- name: 👩‍🔬 Test showcase with Playwright 🎭
shell: bash
if: github.event.pull_request == null
env:
NEXT_PUBLIC_BASE_PATH: ${{ inputs.path }}
run: |
ls ./build-showcases${{ inputs.path }}
npm run test:e2e --workspace=${{ inputs.showcase }} -- --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
# We check if aria-snapshots were changed
- name: 👩‍🔬 Test aria-snapshots with Playwright 🎭
id: aria-snapshots
shell: bash
if: github.event.pull_request != null
run: |
npm run test:aria-snapshots --workspace=${{ inputs.showcase }} -- --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --update-snapshots
OUTPUT=$(npx --no tsx scripts/github/snapshot-changes/check-aria.ts)
echo "aria-snapshots-changed=$OUTPUT" >> $GITHUB_OUTPUT
OUTPUT=$(npx --no tsx scripts/github/snapshot-changes/check-main-aria.ts)
echo "aria-snapshots-changed-to-main=$OUTPUT" >> $GITHUB_OUTPUT
- name: 👩‍🔬 Test axe-core with Playwright 🎭
shell: bash
if: github.event.pull_request != null
run: |
npm run test:axe-core --workspace=${{ inputs.showcase }} -- --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
- name: 👩‍🔬 Test visual-snapshots with Playwright 🎭
id: visual-snapshots
shell: bash
if: github.event.pull_request != null
run: |
npm run test:visual-snapshots --workspace=${{ inputs.showcase }} -- --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --update-snapshots
OUTPUT=$(npx --no tsx scripts/github/snapshot-changes/check-visual.ts)
echo "visual-snapshots-changed=$OUTPUT" >> $GITHUB_OUTPUT
- name: 👩‍🔬 Test a11y-checker with Playwright 🎭
shell: bash
if: |
github.event.pull_request != null &&
steps.aria-snapshots.outputs.aria-snapshots-changed-to-main=='true'
run: |
npm run test:a11y-checker --workspace=${{ inputs.showcase }} -- --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
- name: 🔣 Print GitHub Report
if: failure()
shell: bash
Expand All @@ -88,11 +131,3 @@ jobs:
name: ${{ inputs.showcase }}-playwright-results-${{ matrix.shardIndex }}
path: ./showcases/${{ inputs.showcase }}/test-results
retention-days: 30

- name: 🆙 Upload aChecker results
if: failure()
uses: actions/upload-artifact@v4
with:
name: ${{ inputs.showcase }}-showcase-achecker-results-${{ matrix.shardIndex }}
path: ./showcases/${{ inputs.showcase }}-showcase/results
retention-days: 30
19 changes: 14 additions & 5 deletions .github/workflows/default.yml
Original file line number Diff line number Diff line change
Expand Up @@ -133,12 +133,11 @@ jobs:
showcase: vue-showcase

test-showcase-patternhub:
uses: ./.github/workflows/02-e2e-showcases.yml
uses: ./.github/workflows/02-e2e-patternhub.yml
needs: [build-showcase-patternhub, get-playwright-version]
with:
version: ${{ needs.get-playwright-version.outputs.version }}
path: ${{ needs.build-showcase-patternhub.outputs.path }}
showcase: patternhub

regenerate-snapshots-components:
if: always() && needs.test-components.result == 'failure'
Expand Down Expand Up @@ -170,14 +169,24 @@ jobs:
get-playwright-version
]

# This job runs only if aria-snapshots were changed compared to main or if it runs on main branch
test-screen-reader:
if: |
needs.test-showcase-react.outputs.aria-snapshots-changed-to-main == 'true' ||
github.event.pull_request == null
uses: ./.github/workflows/02-e2e-screen-reader.yml
needs: [build-showcase-react, init-playwright, get-playwright-version]
needs: [test-showcase-react, init-playwright, get-playwright-version]
with:
version: ${{ needs.get-playwright-version.outputs.version }}

regenerate-snapshots:
if: always() && (needs.test-showcase-angular.result == 'failure' || needs.test-showcase-react.result == 'failure' || needs.test-showcase-vue.result == 'failure')
if: |
needs.test-showcase-angular.outputs.aria-snapshots-changed == 'true' ||
needs.test-showcase-angular.outputs.visual-snapshots-changed == 'true' ||
needs.test-showcase-react.outputs.aria-snapshots-changed == 'true' ||
needs.test-showcase-react.outputs.visual-snapshots-changed == 'true' ||
needs.test-showcase-vue.outputs.aria-snapshots-changed == 'true' ||
needs.test-showcase-vue.outputs.visual-snapshots-changed == 'true'
uses: ./.github/workflows/02-e2e-regenerate.yml
with:
version: ${{ needs.get-playwright-version.outputs.version }}
Expand Down Expand Up @@ -231,7 +240,7 @@ jobs:
resultTestShowcasePatternhub="${{ needs.test-showcase-patternhub.result }}"
resultTestScreenReader="${{ needs.test-screen-reader.result }}"
if [[ $resultTestFoundations == "success" ]] && \
[[ $resultTestScreenReader == "success" ]] && \
[[ $resultTestScreenReader == "success" || $resultTestScreenReader == "skipped" ]] && \
[[ $resultTestShowcaseStencil == "success" ]] && \
[[ $resultTestShowcaseAngular == "success" ]] && \
[[ $resultTestShowcaseReact == "success" ]] && \
Expand Down
5 changes: 2 additions & 3 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -130,12 +130,11 @@ jobs:
showcase: vue-showcase

test-showcase-patternhub:
uses: ./.github/workflows/02-e2e-showcases.yml
uses: ./.github/workflows/02-e2e-patternhub.yml
needs: [build-showcase-patternhub, get-playwright-version]
with:
path: ${{ needs.build-showcase-patternhub.outputs.path }}
version: ${{ needs.get-playwright-version.outputs.version }}
showcase: patternhub
path: ${{ needs.build-showcase-patternhub.outputs.path }}

test-screen-reader:
uses: ./.github/workflows/02-e2e-screen-reader.yml
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
- main:
- heading "DBStack" [level=1]
- link "Density"
- text: Functional
- link "Content 1"
- text: Content 2 Content 3 (Default) Regular
- link "Content 1"
- text: Content 2 Content 3 Expressive
- link "Content 1"
- text: Content 2 Content 3
- link "Variant"
- text: (Default) Simple
- link "Content 1"
- text: Content 2 Content 3 Divider
- link "Content 1"
- text: Content 2 Content 3
- link "Gap"
- text: none
- link "Content 1"
- text: Content 2 Content 3 3x-small
- link "Content 1"
- text: Content 2 Content 3 2x-small
- link "Content 1"
- text: Content 2 Content 3 x-small
- link "Content 1"
- text: Content 2 Content 3 (Default) small
- link "Content 1"
- text: Content 2 Content 3 medium
- link "Content 1"
- text: Content 2 Content 3 large
- link "Content 1"
- text: Content 2 Content 3 x-large
- link "Content 1"
- text: Content 2 Content 3 2x-large
- link "Content 1"
- text: Content 2 Content 3 3x-large
- link "Content 1"
- text: Content 2 Content 3
- link "Direction"
- text: (Default) Column
- link "Content 1"
- text: Content 2 Content 3 Row
- link "Content 1"
- text: Content 2 Content 3
- link "Wrap"
- text: "(Default) No Wrap: Column"
- link "Content 1"
- text: "Content 2 Content 3 No Wrap: Row"
- link "Content 1"
- text: "Content 2 Content 3 Wrap: Column"
- link "Content 1"
- text: "Content 2 Content 3 Wrap: Row"
- link "Content 1"
- text: Content 2 Content 3
- link "Alignment Column"
- text: (Default) Stretch
- link "Content 1"
- text: Content 2 Content 3 Start
- link "Content 1"
- text: Content 2 Content 3 Center
- link "Content 1"
- text: Content 2 Content 3 End
- link "Content 1"
- text: Content 2 Content 3
- link "Alignment Row"
- text: (Default) Stretch
- link "Content 1"
- text: Content 2 Content 3 Start
- link "Content 1"
- text: Content 2 Content 3 Center
- link "Content 1"
- text: Content 2 Content 3 End
- link "Content 1"
- text: Content 2 Content 3
- link "Justify Content Column"
- text: (Default) Start
- link "Content 1"
- text: Content 2 Content 3 Center
- link "Content 1"
- text: Content 2 Content 3 End
- link "Content 1"
- text: Content 2 Content 3 Space-Between
- link "Content 1"
- text: Content 2 Content 3
- link "Justify Content Row"
- text: (Default) Start
- link "Content 1"
- text: Content 2 Content 3 Center
- link "Content 1"
- text: Content 2 Content 3 End
- link "Content 1"
- text: Content 2 Content 3 Space-Between
- link "Content 1"
- text: Content 2 Content 3
Loading
Loading