Reading the scan report

The scan report is the result of a scan. While the scan is running, it shows the progress and updates itself. Pages are tested one after another, in the order they came in: if other scans are ahead of yours, the report says how many pages come first and roughly when your scan starts. The time is estimated from the pace of the most recently tested pages. You do not have to wait – the address of the page always leads to the current state and later to the result. If our scanning system is not working, the report says so too; your scan then continues by itself.

Afterwards it sits on four tabs: “Overview”, “Findings”, “Manual checks” and “Statement”. The overview answers “where do I stand?”, the other three are the work that follows from it. The tabs are real pages with their own addresses – you can link to them and move between them with the back button.

The bar “Your path to the statement”

Above the tabs, each of the four pages carries the bar “Your path to the statement”. It shows where you stand on the way from the scan to the draft of your statement: four stations, each with its status in words, and below them a sentence about the next step with a button that takes you there.

“Result” – the scan is finished and supports a claim. If the result does not suffice for that, the bar stays here and gives the reason: From scan to published statement.
“Manual checks” – done as soon as no question that blocks publication is open any more. Voluntary questions may still be open: the station says how many questions are answered, and what stays open stays “not evaluated”. For a scan without an account it reads “needs an account”: Answering the manual checks.
“Details” – who declares and, where it is asked, the applicability check; for public bodies in Germany the choice “Federal or state level” is part of it. You give these details in the “Statement” tab: Which accessibility statement do I need?.
“Draft” – “ready” as soon as the three stations before it are done. Until then the “Statement” tab shows the text as a preview: Using the draft yourself.

Verifying the domain and the mandatory details are not stations. They only come before publishing through the service, not before the draft: From scan to published statement.

On narrow screens the list of stations is collapsed and names only the current one; sentence and button stay visible. If the page below has a button to the same destination itself, the bar shows only the sentence. The bar is missing while the scan is running or has failed, and in the view of a single question – there “To the overview” leads back.

Overview: status and figures

At the very top stands the “Conformance status” of this scan – “fully conformant”, “partially conformant” or “non-conformant” – and below it the sentence explaining what it rests on.

The balance of success criteria

Next to it stand four numbers, measured against the conformance target of this scan. Every success criterion sits in exactly one basket, and the four numbers add up to the total:

“Success criteria without defects” – the scan covers the criterion fully and found nothing.
“fulfilled by own assessment” – the measurement does not reach far enough there, and you answered every question on it with “met” or “not applicable”.
“Success criteria with defects” – at least one defect, or a question you answered with “not met”.
“not evaluated” – the scan does not reach far enough there, and a question on it is open or answered with “I cannot judge this”, or there is no question for the criterion. A criterion for which an uncertain result is still waiting for your decision is listed here too; a sentence below the numbers says which: Assessing uncertain results.

The last basket matters most, and it deliberately stands beside the others as an equal: “without defects” does not mean “fine”, only “we tested this fully and found nothing”. What is not evaluated is not a defect – but it is not a confirmation either. The second basket stands next to the first and never inside it: there it was not the measurement that confirmed something, it was you. A defect from the measurement always outweighs an answer.

If the measurement could not check some spots of a criterion, a sentence below the numbers says so. The criterion then does not count as “without defects”, only as “met by your own assessment” once you have answered: Assessing uncertain results.

As long as criteria are not evaluated and a question on them in this scan is waiting for your answer, a link below the numbers names how many and leads to the manual checks. If the scan dates from before the complete catalogue of questions, it says instead that only a new scan asks the questions. Which criteria fall into which basket is listed row by row in the Accessibility Conformance Report: The conformance status – and when you may deviate. There is deliberately no second figure for affected criteria — it came from the result frozen at completion, did not count failed manual checks, and contradicted this one.

There is deliberately no percentage. Accessibility is met or not met per success criterion; “87 % accessible” would be a number that sounds good and means nothing.

The two paths

Below the status stand two cards of equal weight, because there are two things people want after a scan:

“Fix defects” names the three most urgent open findings in recommended order and leads to the “Findings” tab. Hints are marked “WCAG 2.2 — recommended” there and do not count as open findings.
“Create the statement” has one button to the next station of the path that the bar “Your path to the statement” above it shows – not to an intermediate page that merely says you cannot continue. What the stations mean: Reading the scan report. For a scan without an account the button leads into the account – “Create an account and continue” or, if you are signed in, “Move this scan into an account”: who declares, the applicability check, the domain and the mandatory details belong to a website in your account and follow after that.

The figures of the scan

“Pages tested” – how many of the selected pages could be evaluated.
“Open findings” – how many rules have at least one open occurrence.

Below that, the report explains how pages were selected: how many were found through the sitemap and through links on the home page, how many of them were scanned and how many are excluded by robots.txt.

How reliable the result is

If not all selected pages could be loaded, the report says so, with the share of evaluated pages. Below 80 %, the result does not support a conformance statement, and no statement can be published from it: From scan to published statement.

If the keyboard walk-through ended early on a page, the report says that too. It makes no statement about the elements not reached there. The PDF report states for each page how far the walk-through got and why it ended: at the scan's upper limit or time limit – or at a keyboard trap, which is a finding in itself.

Findings

The second tab. It carries the longest list in the report – which is why it no longer sits below the overview but beside it.

The list is sorted in recommended order: by severity first, and for equal severity by the number of affected pages. A defect in the page template hits every page; one in a single article hits only that one. “Severity” and “Status” narrow the list down; the figure beside them says how much of how many is currently shown.

A finding stands for one rule – for example “insufficient contrast” – and groups all occurrences of that rule. Each finding shows:

the severity: “critical”, “serious”, “moderate” or “minor”,
the affected success criteria,
on how many pages it occurs.

Findings on criteria added with WCAG 2.2 are marked “WCAG 2.2 — recommended” – in the tab, in the overview and in the PDF. They do not count as open findings, which is why the PDF states their number separately.

Expanded, a finding shows each place once: the element’s selector (or the entire page), the measured value – for example the actual contrast ratio –, where relevant the state in which the element becomes visible, such as only in the opened menu, and the pages the place occurs on. A defect in the page footer therefore appears once in the report, not once per page. Measured value, snippet and image are one example from one of these pages; the report says which. A maximum number of occurrences is recorded per rule and page; the report says when there may be more.

Evidence images are taken for the first occurrences on each page; the report shows one per place. Where there is none, the selector and measured value document the place. Depending on the plan, evidence images stay in the service for up to 60 days; the report shows the exact date, and the report itself stays readable afterwards. If you need the images longer, download the report as a PDF first: Reading the scan report.

You can assess every finding – for example as “no defect”: Reviewing findings.

Findings from the consent layer

Findings caused by a cookie banner are grouped separately. They count, but their cause usually lies in the configuration or choice of consent tool, not in your own source code.

If a banner cannot be closed in a restrained way, the check hides it without making a choice and tests the page as it appears without consent. The report names the pages where this happened. If the banner locks the page behind it, it stays in place, and the report says that too.

To review

On the same tab, below the findings.

Spots the scan could not decide reliably. You assess them per building block – identical spots on several pages at once: Assessing uncertain results.

Manual checks

The third tab. Here you assess yourself what no machine can decide conclusively – with preliminary questions about your website’s content and questions in 8 groups. Only a few of them block publication; as long as one of those is open, the bar “Your path to the statement” stays at the station “Manual checks”, and “Answer the questions” leads to the first of them: Answering the manual checks.

Your answers to the preliminary questions appear at the end of the “Findings” tab under “Questions about the website's content”, each with its source – “Self-declaration” or “found in this scan”.

Accessibility statement

The fourth tab, from top to bottom:

“Details for your statement” – who declares and the applicability check. Once the details are complete, they are shown as two lines; “Change details” opens the form again. What they determine: Which accessibility statement do I need?.
“Preview of the draft”, later “Your draft” – the text of the statement with the assessed status, next to it the date of the scan and the number of pages checked. You see the beginning of the text; “Show the whole draft” unfolds the rest. As long as a station before it is open, a note above the preview says what is missing and where you settle it. The finished draft can be copied and downloaded: Using the draft yourself.
“How does the text get onto your website – and what happens afterwards?” – a comparison of the two ways: use it yourself or publish through the service, with what happens in each case after your website changes. Only here does verifying the domain come in: From scan to published statement.
“Other documents from this scan” – the document that follows from your details is preselected; here you choose another one or the Accessibility Conformance Report.

Who can see the report

A report in your account is only visible to members of your organization. Anyone who is not signed in and opens a link to it – for example from the WordPress plugin – is taken to the sign-in page and back to the report afterwards. Reports of a free scan without an account can be reached through their unguessable address.

Downloading as PDF

At the end of the “Findings” tab you will find “Download report” — where the PDF belongs: it archives exactly what that tab shows. “Create PDF” creates a PDF with every place, its pages and one evidence image each, the uncertain results as building blocks (per rule the most affected ones with an image, per block the first pages; anything beyond that is in the “Findings” tab), your answers to the manual checks and to the preliminary questions – including the places you never expanded in the report. The manual checks appear there in the same order and with the same answers as in the tab, including “I cannot judge this”; where a No to a preliminary question set the answer, the remark says “from preliminary question”. For large reports this takes up to a minute; then “Download PDF” appears.

The PDF reflects your assessment at the time it was created. If you change an assessment afterwards, a new PDF is created next time.

Evidence images stay in the service for a limited time only; the report shows the date. The PDF is your archive: if you need the images longer, download it first. The PDF is tagged and has bookmarks from the headings.