Assessing uncertain results

Some requirements cannot always be decided even by a real browser. If text sits on a background image, its contrast cannot be measured reliably. Instead of guessing, the service puts such spots in front of you.

Where to find uncertain results

In the scan report, in the review section, with the count in brackets. It lists building blocks, not single occurrences. Each block shows the question put to you, a screenshot where there is one, the element's text with a link to the page and – folded away – the technical detail for your web agency. Without text it reads “Button without text”, “Image” or “Element”. For differences between pages – such as the same destination with different names – and for how current your statement is, it shows what the scan found instead of a text, for example the different names. Each destination is a block of its own.

What a building block is

A building block groups identical spots: the same rule, the same reason for the uncertainty and the same element – say, the header that looks the same on 25 pages. 104 occurrences often turn into just two or three blocks. Your decision applies to all pages of the block, including pages you have not looked at, and it carries over into the next run: a block you classified as no defect does not come back there – likewise a block you classified as a defect stays a defect, even if the spot shows up on another page. The link leads to the page of the example shown; the other pages of the block are not listed one by one.

How to assess a building block

Look at the spot on your website and choose:

• “not a defect” – the requirement is met here, so the answer to the question is “yes”. The block disappears from the list.
• “is a defect” – the requirement is not met. The block's occurrences become open findings and appear in the statement until they are fixed or you assess it differently: Reviewing findings.

Withdrawing a decision

If you made a mistake, open the finding of the same rule on the “Findings” tab. Under “Decisions on uncertain spots” every block you decided on is listed with “Withdraw decision”. The block then returns to the review section – on all of its pages – and you can decide again. Other blocks and the reliable findings of the same rule stay as they are. This withdraws the decision in this run only; where a later run has already inherited it, withdraw it there.

What “neither” means

As long as you have not assessed a building block, it counts neither as a defect nor as passed: it appears in no list of defects, and it does not block publication.

It has one consequence all the same. Its success criterion stays “not evaluated” for that long – even if the scan otherwise measures it completely, or you answered the question on it in the manual checks with “met”. Otherwise the criterion would be listed as fulfilled while the report still has an open question on it. The overview names the criteria that are held back for that reason alone, and the question concerned carries a notice with the way here. “fully conformant” is out of reach until then, and the Accessibility Conformance Report lists the row as “Not Evaluated”. As soon as you classify the block as a defect or as no defect, the criterion follows your decision.

If there are more uncertain results than the report evaluates individually, it says so: the remaining ones count as not evaluated, not as passed.

Not measured

Sometimes the scan cannot measure every spot: on a page it examined only a limited number of elements, and the rest stayed unmeasured. That is neither an uncertain result nor a question to you – so it is not listed under “Zu prüfen” but at the success criterion itself: in the overview, in the report including the PDF, in the Accessibility Conformance Report and at the question in the manual checks. The sentence names on how many pages and at how many spots the criterion was only partly measured. There is no list of the spots.

The consequence: the measurement alone does not carry such a criterion as “without defects”. It stays “not evaluated” until you have answered the questions on it – and once you answer them with “met” or “not applicable” it stands as “met by your own assessment”, with a note that your answer covers the unmeasured part. Unlike a waiting result, the gap does not hold a criterion back against your answer. If an uncertain result is waiting as well, that is the reason given, because that is where you have to decide something.

Typical cases

• Contrast of text over images or gradients. Check whether the text is easy to read everywhere – including over the lightest part of the image.
• Consistent labels in the navigation. If the same destination has different names on different pages, that is often a defect.
• Keyboard in a dialog. If a dialog – such as a consent layer – deliberately keeps the keyboard inside it, that is not an error as long as it can be closed with the keyboard.

Not to be confused with manual checks

Uncertain results are spots on your pages, identical spots grouped into building blocks that you decide on one block at a time. Manual checks are questions about the whole website that let you assess a success criterion yourself – and some of them must be answered before publishing: Answering the manual checks.