A closer look at cloze tests - do your words make sense?
We often assess content based on how readable it is. Readability scores give you an idea of how complex your words and sentences are and what grade level you need to be at to read them.
But have you ever considered assessing the comprehension of your message? Can your audiences understand what you are trying to tell them?
A cloze test can help you to find this out.
Cloze tests are used to assess language learning, vocabulary and comprehension in children.
They can also be used to test:
that users can understand your message
what words or phrases they prefer to use
suitability of language for certain audiences
You can’t test everything so choose content that is related to a top task or an audience that may need support. Cloze tests can be particularly useful when thinking about content such as:
complex financial information for potential undergraduate students
information for those entering university via widening participation schemes (often younger when applying for residential summer schools for example)
application information for students where English is not their first language
What is a cloze test?
A cloze test is typically an exercise where a piece of text is provided with certain words removed. The participant is asked to fill in the gaps.
Example:
"Today, I went to the ________[word 1] and bought some bread and eggs. I knew it was going to rain, but I forgot to take my ________[word 2], and got wet on the way."
Here we might expect suggestions such as shop or supermarket for word 1 and perhaps umbrella or raincoat for word 2. Successful completion of this content tells you:
the content is understandable
the sorts of words preferred by your users
Setting up your cloze test
First, select the piece of text that you wish to test.
You should usually aim for no more than 250 words to keep the test from being overwhelming. Misspellings are acceptable responses - you are not assessing the participant, you are assessing the text.
In most cases cloze tests remove every fifth word (the more you remove, the harder it becomes). Make sure that you remove words that are meaningful, removing lots of prepositions is not going to help you assess your content. Aim for around 30 blanks, more than this can become tiring for your user.
Consider the length of the text to be read and understood and adjust the amount of time you allocate to the exercise.
Then simply ask participants to fill in the gaps with words that they think fit.
Open or closed cloze test
Cloze tests are usually left ‘open’. This means that participants are free to complete the blanks with any word or phrase that they think fits. The benefit here is that you get clear insights into the language used by your audience.
You can also opt to run a closed cloze test and provide a predetermined set of words (usually around 4-5). This is most useful when you are trying to understand preferred terminology. It can help you make decisions about what words to use in your style guide for example.
Software and systems
It’s simple to set up your test. You could use a google doc to allow people to input online. Printing out the text allows people to write their responses in the gaps.
You could also use survey systems to help you create closed cloze tests with multiple choice options for each word.
Assessing the results
You are potentially going to receive lots of different word alternatives. You need to make a judgement on what you accept as ‘correct’. Many incorrect responses might suggest that your text is too complex for your audiences to understand.
As a guide, if more than 60% of the blanks are filled in correctly, then the text is considered to be easy to read. If 40%-60% of the blanks are filled in correctly, then the text is considered "educative" reading. This means that your audience is reading this text to learn. Less than 40% completion means your text is difficult to understand and probably needs rewriting.
If you have set up a closed cloze test, you will need to look for consensus across terms chosen by participants. This can help you choose the terminology that works best for your readers.
How we can help you
Cloze tests are part of a suite of assessments we can carry out on your content to figure out how well it is working for you. Get in touch to explore the options we can provide around assessing readability, accessibility and best practice.