Reading Materials Covered: UxBook Chapter 14 Rigorous Empirical Evaluation: Preparation
UxBook Chapter 15 Rigorous Empirical Evaluation: Running the Session
These two chapters are very close contacted with each other. Chapter 14 is about how to make a good preparation for evaluation. Firstly, we should know how to plan for rigorous empirical UX evaluation, especially indentify the most important design issues and user tasks to investigate. Secondly, select our team members and appropriate tasks to support evaluation. Thirdly, determine our evaluation methods and data collection techniques which can fit our evaluation goals and needs. Just remember our approach to choosing these methods are goal driven. The following procedures are participant selection, recruitment and management. The book lists concrete steps to do these things and things what we should avoid. The last one is pilot testing. It is an effective method to find weaknesses in the prototype and solve problems in and early stage.
Chapter 15 is a continuation of Chapter 14. The article focuses on the details of running a lab-based evaluation. It begins with some essential preliminaries and protocol issues. Then the author uses three sections to discuss how to generate and collect quantitative UX data, qualitative UX data and emotional impact data. At last, wrap up our evaluation session.
This week’s materials are all about usability testing. The basic question is what is usability testing, why it’s done and how it is done.
1. What is usability testing and why it is done?
As a web designer, we need to do usability testing to see if our system works correctly as we expect. Jakob Nielsen defines usability testing and he says:
Usability is a quality attribute that assesses how easy user interfaces are to use. The word “usability” also refers to methods for improving ease-of-use during the design process.
In my opinion, usability testing determines the efficiency, satisfaction and ease of use for our product. How can users find and use the features the first time they interact with our system? We want to know whether or not the users can accomplish their tasks easily, effectively and satisfactorily. That is what usability testing should do and the goal is to refine and modify our design.
2. How is usability testing done?
As UxBook mentions, there are two types of tests: lab-based test vs field-based test. Because our goal is to test whether our website product satisfies users’ needs, I will focus on lab test. There are several steps and methods to do usability testing.
- Plan and Preparation
The first thing is to make a plan and prepare for the test. Select a team, determine our product and set our evaluation goals and measurements. Do not forget to select participants and recruit them.
- User study
Learn about users’ physical and cultural characteristics, motivations and mental models. Also, why they use our products and how they use it. Especially, explore users’ goals and things they need to achieve.
- Task study
Select appropriate tasks to support our evaluation. Except for benchmark tasks to measure issues, we can also select issues that uncover problems if they exist just like controversial issues. Keep an eye on critical incidents. Sometimes they can reflect some obvious and severe UX problems.
- Conduct the usability test
Just let user finish the task and do not intervene them. Comprehension is very important. Reassure that there is no misconception. If a user asks a question, don’t answer it directly. Just observe what they behave and write it down. We can also use videotape to help us record but paper notes taking is necessary. Then we can categorize data and analyze data. Determine the causes of problems and find relevant usability issues.
Complete and polish our usability report. Keep in mind who will read the report. Well done!