Accessibility auditing and creating an accessibility statement

Module 6

  • In the accessibility audit process, the accessibility of the service is assessed as a whole and more broadly than individual components or usage paths.
  • The auditing is based on legally prescribed requirements. The tools used include the WCAG (Web Content Accessibility Guidelines) criteria.
  • An audit carried out in accordance with the Accessibility Model will produce an up-to-date accessibility statement required by law.

Accessibility statement templates


The law obligates Helsinki to produce accessible digital services

Finnish law features the Act on the Provision of Digital Services (Ministry of Finance, 1 April 2019), which is based on the EU Directive on accessibility. The Act obligates authorities, such as the City of Helsinki, to make all publicly available digital services (websites, mobile applications) accessible. The Act defines accessibility as criteria for technical and visual implementation that the service must meet; the criteria are in turn based on the international Web Content Accessibility Guidelines (WCAG).

Web Content Accessibility Guidelines

The WCAG standard sets a series of criteria that are used to assess the accessibility of a service. The criteria are concrete, primarily consisting of requirements related to technical and visual implementation placed at the A, AA or AAA level in the standard. The law stipulates that public-sector services must meet the AA level criteria of the WCAG, with the exception of AA criterion 1.2.4, “Captions (Live)”.

The law also contains some requirements outside the WCAG, such as the requirement that the service must be secure and efficient. However, in practice, the accessibility of services produced by the City of Helsinki is tested based on the WCAG standard, as the legally prescribed accessibility statement is also based on the fulfilment of the WCAG criteria.

Accessibility auditing and accessibility statement

Every service published by the City of Helsinki must feature a legally required (link above) accessibility statement. The statement is public and provided for the information of the service users. Among other things, the statement provides an account of the state of the accessibility of the service – any shortcomings and deviations, as well as information about remedying plans – as well as a channel through which users can submit feedback regarding the accessibility of the service. More detailed information about the legally prescribed contents of the statement can be found on a website administrated by the responsible authority, the Regional State Administrative Agency, or AVI for short (www.webaccessibility.fi).

In Helsinki’s Accessibility Model, ‘accessibility auditing’ refers to testing with the purpose of establishing and reporting how well the service meets the requirements of the WCAG standard. The nature of the audit must be such that it can be used as the basis for creating an accessibility statement regarding the service. The contents of the auditing report are discussed in greater detail later in this document.

If the audit is ordered from an external supplier, the order must specify who will create the accessibility statement. The Accessibility Model recommends that the auditor produce the statement in collaboration with the owner of the service, whose expertise in the background and limitations of the service and further development plans will be of help in the formulation of the statement.

Link to the Papunet checklist (in Finnish)
https://papunet.net/saavutettavuus

The supplier must be familiar with the methods used. They can also carry out the audit on their own if they have persons well acquainted with the WCAG and aid devices at their disposal.

Ordering an accessibility audit

The supplier of the accessibility audit will be responsible for selecting the tools and methods used in the testing, as well as their suitability for WCAG compliance testing. The client is not required to be able to define the expert roles of the testing process unless they specifically want to do so.

Thus, the supplier will also be responsible for acquiring the necessary information for ensuring the comprehensiveness of the service testing process. This information includes aspects such as what sections there are in the service that require logging in but are open to the public and whether any documents on the website, e.g. PDF files, are also covered by the audit or another testing process, for example.

More detailed information on what document and media contents and organisational intranet systems are subject to accessibility legislation can be found on the AVI website (www.webaccessibility.fi). More detailed information was also added to the instructions of the Accessibility Model in March 2021.

Reporting the accessibility testing results

As a result of the accessibility audit, the client will be provided with a report detailing any elements of the service that do not meet the requirements of the WCAG criteria.

There is no need to specify every individual WCAG breach if the breach is a repetition of a previously reported occurrence. In other words, having the report point out the elements of the service in which the WCAG breach is repeated will suffice. For example, a shortcoming detected can concern an element that is always repeated in a certain page template, whereby there will be no need to list every individual page on which the element is featured. The information that the issue is repeated in the service wherever the element in question is used will suffice.

As regards WCAG breaches, at least the following information marked as required must be reported. Reporting any information marked as recommendations is optional.

  • Requirement: Where the breach occurs in the service. Recommendation: A screenshot that illustrates the phenomenon in the service is attached to the report.
  • Requirement: Which WCAG criterion or criteria the shortcoming detected is related to.
  • Requirement: The WCAG breach is categorised in accordance with its effectiveness. The gauge is how significant the effect of the accessibility issue is on the users of the service. The purpose of this categorisation is to help the owner of the service prioritise potential remedying and further development work in accordance with how the actual accessibility of the service can be improved the most efficiently. The categories are ‘critical’, ‘significant’ and ‘minor’.
    • Critical accessibility shortcoming: The program may prevent some of the users of the service from carrying out the functions of the service altogether. For example, if the language menu in the banner section of the page cannot be opened when browsing with a keyboard, the shortcoming (WCAG 2.1.1) is critical.
    • Significant accessibility shortcoming: The issue reduces the usability of the service particularly for special groups, but not necessarily completely. For example, if the service issues a notification on the page stating that the user has an unread message and the notification is not conveyed to assistive technologies as a live, status or alert notification but can still be found on the page (WCAG 4.1.3), the shortcoming is significant but not critical.
    • Minor accessibility shortcoming: The issue is a breach of the WCAG criteria and thus subject to the Act on the Provision of Digital Services, but its impact on the usability of the service is minor. For example, if the headings used on the website need specification or clarification, the issue may be a breach of WCAG criterion 2.4.6 the impact of which is quite minor.
  • Requirement: The report must provide a verbal account of how the shortcoming reported is a breach of the WCAG. In practice, the reader must be able to understand from the account how the issue occurs and why the occurrence does not comply with the assigned criterion. As such, merely listing or tabulating WCAG breaches (criterion met / not met) is not enough.
  • Recommendation: The report should also suggest a remedy for every accessibility shortcoming detected. From the client’s perspective, such remedying suggestions and instructions can be extremely useful, especially if the client does not yet have sufficient accessibility expertise to be provided to the developers.
  • Recommendation: The assignment includes reviewing the report together with the client and the supplier of the testing. At this report workshop, the client will have an opportunity to ask about and discuss the observations documented and aspects such as the most suitable remedying method for each situation.

Example: reporting a WCAG breach

The final section provides an example of how an individual WCAG shortcoming can be reported in accordance with the requirements discussed above. The method presented here is an example that does not bind the supplier in terms of structuring or the amount of content but can serve as a guideline for the client regarding what kind of a reporting method can be reasonably required from the testing.

Before reporting

An accessibility expert is testing the online service in the desktop view. They discover that the navigation menu of the page cannot be focused on at all with the Tab key when browsing with a keyboard. This prompts the tester to next test the menu with a screen reader, whereby they notice that the screen reader can be used to open the menu but no information about the status of the menu is conveyed to the aid device, and the contents of the menu do not follow the button in the browsing order. Furthermore, the accessible name of the menu button (“Valikko”, i.e. ‘Menu’ in Finnish) is different from the visual text on the button (“Menu”).

The tester’s findings are that the implementation of the menu is in breach of at least the following WCAG criteria:

  • 2.1.1 Keyboard
  • 2.4.3 Focus Order
  • 2.5.3 Label in Name
  • 4.1.2 Name, Role, Value

The following observations are immediately recorded in the test report.

Desktop view / Main navigation menu of the website: The menu button cannot be focused on with a keyboard (critical)

Object

The repeating banner section of the website > a button that opens the main navigation menu, featuring the text “Menu”.

Description

When a user uses the service with a keyboard and no mouse or screen reader and navigates the page with the Tab key, the navigation button cannot be focused on.

Impact

The issue is critical, as it prevents persons using a keyboard or keyboard interface aids from opening the navigation menu of the website, in turn preventing them from using the search function of the service and potentially accessing some of the subpages of the service. The issue does not concern screen readers, as they produce an onclick event for the button similarly to a mouse.

WCAG

2.1.1. Keyboard (A)

Remedying suggestion

  • Set the ‘tabindex=”0”’ attribute for the menu button to make the ‘<span>’ element focusable with a keyboard.
  • At the same time, ensure (because this could not be established in the testing process) that the button has a keyboard event processor that will open the menu when the user presses the Enter key (keyCode 13) or the space bar (keyCode 32). Also ensure that pressing the space bar on the button will not activate the default function of the browser, which is scrolling down (event.preventDefault()).
  • Check that the menu button indicates keyboard focus either with the default indicator of the system or, if the indicator has been customised in the styles of the page, in a way that is shown in the CSS ‘:focus’ mode (e.g. outline property). The customised frame indicator must have a 3:1 darkness contrast ratio against the background of the page. If the indicator is merely a change in the background colour of the button, the colour difference must be equivalent to the 3:1 darkness contrast ratio.

Desktop view / Main navigation menu of the website: The opened menu is placed illogically in the browsing order of the page (significant)

Object

The repeating banner section of the website > the main navigation menu in the opened state > the menu panel.

Description

The contents of the opened menu (menu panel) are placed after the banner section in the browsing order and do not immediately follow the menu button when the user browses the page forward. Instead, the button is immediately followed by the language menu (which is placed right of the button in the banner). The navigation menu is placed in the DOM after the language menu, outside the banner section. The reason for this issue is that assistive technologies and keyboard browsing follow the DOM order of the page, not its visual layout (CSS placements), which was perhaps not taken into account in the implementation of the menu.

Impact

The issue makes using the service clearly more difficult for people who use the website with a keyboard-based control device or a screen reader. A user operating without a visual context may even think that the menu did not open at all, or that the language menu is placed inside it, which deviates from the actual structuring of the page. On the other hand, the menu can be found on the page immediately after the banner, so its use is not completely prevented.

WCAG

  • • 2.4.3 Focus Order (A)

Remedying suggestion

The menu panel must be placed immediately after the button in the DOM order when the menu is open. For example, the following structural change should remedy the issue:

Menu button closed:

<nav class=”navigation-region”>
<span id=”navigation-menu-button” …>Menu<img ..></span>
</nav> 

Menu button open:

<nav class=”navigation-region”>
<span id=”navigation-menu-button” …>Menu<img ..></span>
<div class=”navigation-menu-panel”>
<ul><li><a …>… <li>…</ul>
</div>
</nav>

In other words, the menu button and the menu panel are assigned a shared parent the DOM of which both are placed in, yet so that the menu panel is not a child tag of the button but at the same DOM level with it (element sibling). This will automatically make the browsing order correct, whereby no contents outside the menu can “come between” the elements and muddle the order. At the same time, the navigation area will have its own landmark (‘nav’ tag, equivalent to the ‘role=”navigation”’ role) and neither element will accidentally leak outside the landmark.

The rest of the shortcomings described above (missing ‘aria-expanded’ attribute and ‘label in name’ issue) will be reported similarly.


You may also find these interesting

Accessibility user testing

Implementation and software testing


Original version: Tero Pesonen / Siteimprove
Created: 4 March 2021