Reading Reflections of Week 12: Usability Testing-Analyzing and Reporting Findings

06 Nov

Reading Material Covered: UxBook Chapter 16& 17

I have learned a lot of things about rigorous UX evaluation methods in last class. They can be formative or summative evaluation, or both. Now we have our qualitative usability data, what can we do next with these data? That’s what UxBook Chapter 16 tries to tell us: do informal summative data analysis, formative data analysis and cost-importance analysis to identity UX problems and fix them. UxBook Chapter 17 is quite straightforward. The author talked about how to report our informal summative and formative evaluation results. That is also the end of UX lifecycle process.

The goal of data analysis is to find whether the interaction design has met the UX targets. If not, how to modify the design. In one word, identify UX problems and causes as well as give suggestions to fix them. In order to provide a better explanation of the qualitative data process, the author uses several terminologies such as problems, symptoms, diagnosis, causes of causes, treatment and cure confirmation to illustrate it. Moreover, an important concept, UX problem instance, is posed in this chapter. The concept is different from critical incidents which is a record of raw data. Problem instance is more abstract and denotes the essence of a problem or cause which has been interpreted from the raw data.

The key procedure of formative data analysis is consolidating, merging and grouping these data. UxBook says that we need to extract related parts of one critical incident notes and put the critical incident pieces into one UX problem instance. This is the first step-consolidate data. After that, find and merge congruent problem instances into UX problem records. It will engender a single complete and representative UX problem description which could contribute to the cost-importance analysis as a basic reference. The third step is grouping related problem records which should be fixed together. Affinity diagram is a good tool to help us group and label all the problems.

Now we have these grouped problems, what’s next? UxBook says that we should prioritize UX problems in order to fix them. The book recommends a cost-importance table as a basic form to assess priorities. The table is powerful and effective to calculate trade-offs and I really love it! The column contains problem, importance to fix, solutions, cost to fix, priority ratio, priority rankings, cumulative cost and affordability. The form is quite straightforward and the process of cost-importance analysis is simple. I think the most difficult thing is to decide sort of problems and ordering of priorities. The book mentions that high importance and low cost problems are at the top of priority list and we can fix them at first. However, if the problem is high importance and high cost, how can we sort it? I think it is not pretty easy for me to make a decision. Maybe good feeling and engineering intuition will help me judge it.

After investing so many efforts and cost, finally we may think that our work is done. However, the last step of evaluation should not be ignored. It is neccssary to report formative UX evaluation results and convince the project team to fix those problems. For different audience, use different content types. But the basic principles should be followed: consistency, plain language, precision and specificity. Instead of being blunt, the report should be courtesy, likeable and positive. Remember that this is a collaborative team work. Although we are writing some flaws and pitfalls of the baby product, positive and euphemistic words will breed persuasiveness and success. Avoid jargon, avoid acrimony and avoid blaming. By doing so, our UX role will be taken seriously and the problems will be effectively fixed.

Leave a comment

Posted by on November 6, 2012 in Reading Reflections



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: