What automated accessibility tools detect and what only expert manual review finds, criterion by criterion in WCAG 2.2, and how to combine the two.
Why running a validator is not enough
Running an automated tool over the website is the first step for almost any team that cares about accessibility. It is fast, free in many cases and returns a list of errors that looks very objective. The problem arises when that report is taken as the final verdict: a page can come out “clean” in the validator and still be impossible to use with a keyboard or a screen reader.
The W3C itself says so plainly in its guide to choosing evaluation tools: tools cannot determine accessibility, they can only assist in doing so, they cannot check every aspect automatically and human judgement is required. It also warns that they sometimes produce false or misleading results.
In this article we are not going to repeat lists of tools. We focus on a more useful question: which part of each WCAG 2.2 criterion a machine can check and which part needs an expert, and what the W3C and Spanish legislation say about it.
What the W3C and the legislation say
Combining automated and manual testing is not a consultants’ preference: it is written into the reference documents. These are the most relevant:
| Source | What it establishes about tools and manual review |
|---|---|
| W3C, “Selecting Web Accessibility Evaluation Tools” (updated 13 May 2024) | Tools cannot determine accessibility, they can only assist; they cannot check everything automatically and human judgement is required. |
| W3C, WCAG-EM 2.0 (Group Note of 23 July 2026) | Most accessibility checks are not fully automatable; tools mainly help to explore and to select the sample, and also during manual evaluation. |
| Royal Decree 1112/2018 (Spanish Real Decreto 1112/2018), Article 17.3 | Accessibility reviews must cover all requirements and combine aspects of automated review with aspects of expert manual review. |
| Commission Implementing Decision (EU) 2018/1524 | Distinguishes simplified monitoring, to detect non-compliance and carried out as far as reasonably possible with automated tests, from in-depth monitoring to verify conformance. |
The new version of the W3C evaluation methodology, WCAG-EM 2.0, was published on 23 July 2026. It builds on version 1.0 from 2014, is designed for WCAG 2 and uses WCAG 2.2 as its reference. It extends its scope beyond websites to cover web applications, native and hybrid applications, kiosks and documents. Its five steps remain: define the scope, explore the product, select a representative sample, evaluate it and report the findings.
What an automated tool does well
Automated tools are essential, provided they are used for what they are good at. Their strength is objective checks, those with a yes-or-no answer that can be read in the code:
• Presence or absence of something mandatory: alternative text on an image, an accessible name on a button or form field, a page title or the document’s language attribute.
• Calculable values, such as the contrast ratio between a text and its background when both colours are defined in the code.
• Technical consistency: broken references between cells and headers in a table, a page frame without a name or a setting that prevents zooming the page on mobile.
• Scale and repetition: reviewing hundreds of pages or running the same tests on every version of the site, something unfeasible by hand.
That is why WCAG-EM 2.0 recognises that large-scale evaluations rely mainly on automated tools, although it warns that they do not usually reach the qualitative depth the methodology aims for.
Where it falls short: human judgement
Most WCAG 2.2 criteria do not ask whether something exists, but whether it serves its purpose. Criterion 1.1.1 makes this clear: all non-text content must have a text alternative that serves the equivalent purpose. A machine can know that there is a text; it cannot know whether that text properly describes a sales chart or whether the image was decorative and should have been ignored.
Among the W3C ACT test rules, some check whether something exists and others check whether it is appropriate. A few pairs illustrate this very well:
| Objective check (usually automated) | Check that requires judgement |
|---|---|
| Image has non-empty accessible name | Image accessible name is descriptive |
| HTML page has non-empty title | HTML page title is descriptive (proposed rule) |
| Link has non-empty accessible name | Link in context is descriptive (proposed rule) |
| Form field has non-empty accessible name | Form field label is descriptive (proposed rule) |
There are also barriers that only appear when you use the page: focus getting lost when a modal window opens, a drop-down menu that cannot be closed with the keyboard, an error message that is not announced or a reading order that makes no sense. To find them you have to navigate with a keyboard, with a screen reader, with zoom and at different screen sizes.
Criterion by criterion: what each technique detects
The following table lists common WCAG 2.2 criteria and divides the work between the two techniques. It is indicative: what each specific tool detects varies. The criterion names are the official W3C ones.
| Criterion (level) | What a tool can check | What needs manual review |
|---|---|---|
| 1.1.1 Non-text Content (A) | Images without alternative text | Whether the text is equivalent and whether the image is decorative |
| 1.2.2 Captions (Prerecorded) (A) | Signs of video without a caption track | Whether captions exist, are complete and are synchronised |
| 1.3.1 Info and Relationships (A) | Tables with broken header references | Whether what looks like a heading, a list or a table is also one in the code |
| 1.4.3 Contrast (Minimum) (AA) | Text contrast with colours defined in the code | Text over images or gradients and states such as mouse hover |
| 2.1.1 Keyboard (A) | Little: some technical indications | Going through the whole page and its components using only the keyboard |
| 2.4.2 Page Titled (A) | That the page has a non-empty title | That the title describes the topic or purpose |
| 2.4.4 Link Purpose (In Context) (A) | Links without an accessible name | Whether the link text explains where it leads |
| 2.4.7 Focus Visible (AA) | Very limited | Seeing the focus indicator on every interactive element |
| 2.4.11 Focus Not Obscured (Minimum) (AA) | Very limited | That sticky headers or cookie notices do not hide the focus |
| 2.5.8 Target Size (Minimum) (AA) | Measuring the size of some controls | Applying the exceptions, such as spacing or links within a text |
| 3.1.1 Language of Page (A) | That the language attribute exists and has a valid code | That the declared language is the actual one |
| 3.3.8 Accessible Authentication (Minimum) (AA) | Almost nothing | That logging in does not depend on cognitive tests without an alternative |
| 4.1.2 Name, Role, Value (A) | Buttons and fields without an accessible name | That custom components properly announce their role and state |
The table is easy to read: the automated part usually covers the presence of elements; the manual part, their quality and their behaviour. Bear in mind too that criterion 4.1.1 Parsing no longer exists in WCAG 2.2: it is listed as obsolete and removed, although anyone who must meet WCAG 2.0 or 2.1 due to a regulatory requirement may need to keep checking it.
The public sector case: simplified and in-depth monitoring
Directive (EU) 2016/2102 requires Member States to check the accessibility of public sector websites and applications periodically using a common methodology. That methodology was set by Commission Implementing Decision (EU) 2018/1524, which combines both approaches:
• Simplified monitoring: used to detect cases of non-compliance and carried out as far as reasonably possible with automated tests. It applies only to websites, on large samples.
• In-depth monitoring: used to verify conformance and applied to a smaller sample of websites and applications.
In Spain, the 2022 methodology of the Web Accessibility Observatory (Observatorio de Accesibilidad Web) for in-depth monitoring explains it as follows: simplified monitoring automatically analyses a subset of requirements, while in-depth monitoring analyses all requirements, manually. The same document acknowledges that there are elements that could be improved which cannot be detected by automated analysis.
For the bodies concerned, Article 17 of Royal Decree 1112/2018 goes further: reviews of their own websites and applications must cover all requirements and combine automated review with expert manual review. A report generated with a tool alone does not meet that requirement for expert manual review. Remember that this article is for information only and does not replace legal advice on your specific case.
A figure to put tools in context
In February 2017, the accessibility team at the UK Government Digital Service published a very illustrative test. They created a page with 143 accessibility failures grouped into 19 categories and analysed it with 10 automated tools.
| GDS test result (2017) | Percentage of barriers detected |
|---|---|
| Most effective tool (also counting manual review warnings) | 41% |
| Least effective of the tools cited | 17% |
| All tools combined | 71% |
Tools have improved since then and these figures should not be read as a current percentage. What still holds is the GDS conclusion: tools are most effective when combined with manual testing, and the recommended approach is to bring together automated tools, manual checking, an accessibility audit and user testing.
How to combine both techniques step by step
A realistic workflow does not pit tools against people: it puts them in order. This is a sequence you can adapt to your team:
1. Define the scope and the level you must meet (usually AA) and which parts of the website, application or documents are included.
2. Run a broad automated analysis to find obvious and repeated errors, and fix them first: they usually come from templates and shared components.
3. Select a representative sample: home pages, different templates, forms, complete processes such as registration or payment, and downloadable documents.
4. Review the sample by hand: keyboard, focus, screen reader, 200% zoom, contrast in states and images, alternative texts and error messages.
5. Test with people with disabilities whenever possible: WCAG-EM 2.0 strongly recommends it because they can help uncover barriers that expert review alone does not easily detect.
6. Build automated testing into development so that objective errors do not return with every update.
7. Repeat the review periodically, reusing part of the previous sample to compare results.
Common mistakes to avoid
• Taking a tool’s score as a compliance percentage.
• Accepting “check manually” warnings without checking them.
• Relying on accessibility widgets or overlays: they do not fix the code or guarantee compliance.
• Evaluating only the home page and forgetting processes and documents.
What you gain from an audit that combines both
The automated tool tells you where to look first; manual review tells you whether people can actually use your website, your application or your documents. Together they give you a complete diagnosis, a prioritised remediation plan and a solid basis for your accessibility statement.
| Need | Automated tool only | Tool and expert manual review |
|---|---|---|
| Detect repeated errors quickly | Yes | Yes |
| Assess the quality of alternative texts and links | No | Yes |
| Check keyboard, focus and screen readers | No | Yes |
| Combine automated and expert manual review, as required by Article 17.3 of Royal Decree 1112/2018 | No | Yes |
If you want to know what your validator is missing, the Tu Web Accesible digital accessibility audit combines automated analysis and expert manual review against the WCAG 2.2 criteria, and if you then need help fixing what is found, you can rely on our accessibility remediation service.
Sources
• W3C WAI: Selecting Web Accessibility Evaluation Tools (https://www.w3.org/WAI/test-evaluate/tools/selecting/)
• W3C: WCAG Evaluation Methodology (WCAG-EM) 2.0 (https://www.w3.org/TR/WCAG-EM/)
• W3C: Web Content Accessibility Guidelines (WCAG) 2.2 (https://www.w3.org/TR/WCAG22/)
• W3C WAI: ACT Rules (Accessibility Conformance Testing) (https://www.w3.org/WAI/standards-guidelines/act/rules/)
• BOE (Spanish Official State Gazette): Royal Decree 1112/2018 on the accessibility of public sector websites and mobile applications (https://www.boe.es/buscar/act.php?id=BOE-A-2018-12699)
• EUR-Lex: Directive (EU) 2016/2102 (https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32016L2102)
• EUR-Lex: Commission Implementing Decision (EU) 2018/1524 (https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32018D1524)
• PAe (Spanish e-Government Portal): Methodology for in-depth monitoring of websites (2022) (https://administracionelectronica.gob.es/pae_Home/va/dam/jcr:d6819e20-361a-47a9-8659-8ba0d5cb9d4e/2022_Metodologia_OAW_SitiosWeb.pdf)
• GOV.UK Accessibility blog: testing automated tools (2017) (https://accessibility.blog.gov.uk/2017/02/24/what-we-found-when-we-tested-tools-on-the-worlds-least-accessible-webpage/)