Documentation request: automated accessibility tests in CI

[thibaudcolas]

Is your proposal related to a problem?

Some of the appeal of maintaining a pattern library is to make it easier to test vast swaths of UI components without having to figure out where they live in a site, and with controlled data mocks (as opposed to user input on a live site).

This is partly mentioned in the Automated tests docs. I’d like to have a whole page showing a step-by-step CI test example, with Pa11y.

Describe the solution you'd like

Here is my current draft in GitLab CI (high-level, lots of details removed for clarity):

test_patterns:
  image: python:3.7
  stage: test
  services:
    - postgres:9.6
  dependencies:
    - static
    - poetry
  variables:
    DJANGO_SETTINGS_MODULE: settings.production
  before_script:
    - source venv/bin/activate
  script:
    - python manage.py createcachetable
    - python manage.py collectstatic --verbosity 0 --noinput --clear
    - python manage.py render_patterns --wrap-fragment
    - mv static dpl-rendered-patterns
  artifacts:
    name: 'test_patterns-$CI_JOB_ID'
    paths:
      - ./dpl-rendered-patterns
    expire_in: 30 mins

test_a11y:
  # Trial only – likely not the most appropriate base image. https://hub.docker.com/r/alekzonder/puppeteer
  image: alekzonder/puppeteer
  stage: after_test
  dependencies:
    - test_patterns
  script:
    - npx serve dpl-rendered-patterns &
    - npm install pa11y-ci
    - ./node_modules/.bin/pa11y-ci --config pa11y.config.js

Still todo:

• Confirm the two-job split is sensible
• render_patterns should continue execution when a template fails to render #136
• Assess which base image to use for Puppeteer in GitLab CI
• Sample Pa11y / pa11y-ci configuration
• GitHub Actions variant?

Describe alternatives you've considered

This could also be a standalone blog post, or conference talk, but it can be all those things if relevant.

[thibaudcolas]

This has been implemented: https://torchbox.github.io/django-pattern-library/guides/automated-tests/#accessibility-testing