A Complete Look at Usability Testing
It’s easy as a designer or developer to think that your target market will behave in a certain manner – because it makes sense to you – but as we discovered, this is often not the case!
Designing a site and its workflow is often a “one and done” task. Sure, we sometimes get opinions from our colleagues and even reach out to our friends and family for feedback during the process, but the truth is these casual opinions aren’t really worth much.
Usability testing encompasses a whole range of tests and information gathering in the purest form possible – by having your target market actually use your site.
Gossamer Threads recently had the opportunity to participate and oversee a usability test for our client Wordy.com. Given the novelty of their service, Wordy wanted to see if their site made sense to their target market, and if it was delivering an effective message.
What we discovered was truly enlightening.
About the Test Site
Wordy offers a new way to have your copy, content and documents professionally edited by experts in a variety of industries. Through an online submission process, users can get their work edited and sent back within the hour (and sometimes within minutes!).
Gossamer Threads was tasked with the development of the Wordy site. After having recieved the design mockups from the client, GT went to work implementing the website, from the HTML/CSS templates to the job workflow and payment systems.
Wordy and Gossamer Threads knew that there was something left to be desired about the flow of the website. Though visually attractive, we weren’t sure if certain layouts made sense, or if site exploration had any obstacles. Since the design and development were done by the two companies, these questions couldn’t really be answered in-house.
It became apparent that Usability Testing was needed.
Usability: A Participant’s Point of View
Our Lead Designer, Ian, had a chance to be part of this study as a writer for the Gossamer Blog and various optimization guides on our speed optimization site GTmetrix. Considered part of Wordy’s target market, Ian was asked to look at the dev site to see if it would appeal to him as a content writer.
Ian was one of 8 participants in this usability study, which is the minimum number needed in a usability study. Bigger sites with more exploration have seen a need for up to 25 participants!
Ian was placed in a room with a special computer setup. This setup has the standard keyboard, mouse and monitor one is accustomed to, but also a webcam and eye-tracking device affixed to the top of the monitor. The room also has another camera setup that gives a more complete view of the participant (to observe body language or head movement).
Participant setup with 2 webcams, eye-tracking and screencasting
The goal is to have a familiar environment for Ian so that his natural behaviour shines through when he explores the dev site.
Ian was asked a few questions about his line of work to give him a basic sense of what to expect without actually saying what the site is about.
Calibration also needed to be done to ensure proper function of the eye-tracking feature, which required Ian to follow a bouncing red block with his eyes. Upon completion, he was told by the facilitator the following items:
- They study is testing the site, not you (there are no wrong answers)
- You’re not necessarily looking for technical bugs
- Think aloud as much as possible
- Explore the site as normally as you would any other
The dev site was then revealed and Ian started to explore.
Ian figures out the new Wordy redesign
The facilitator then prompts him when he gets to certain areas of the site, asking him about his immediate thoughts and most importantly, his expectations. If anything was confusing or unclear, Ian told them what content he thought should be there instead, or how he expected a certain function to work.
Ian was also asked what he liked and didn’t like about the site, and his opinions on what the site was trying to communicate.
The Other Side: The Observers Point of View
In the next room, a group of people are monitoring and observing the participants interaction with the website. They take notes, and discuss common issues brought forth. A projector screen is setup that displays the participants screen in real time, the close-up and far perspective video feeds, and an audio feed.
The next room can observe the participants every action
Close observation and copious notes are taken on the each participants comments, and through the live screencast and eye-tracking, common patterns in how users react to elements on the site begin to emerge. Observers can literally see what the participants’ eyes are drawn to first, what areas they linger on, and what text they actually read.
One thing that stood out was the discovery of our participant’s expectations. Common issues and opinions were brought up, prompting us to re-evaluate certain areas of the site. It’s easy as a designer or developer to think that your target market will behave in a certain manner – because it makes sense to you – but as we discovered, this is often not the case!
What was Discovered
As we suspected, Wordy had some usability issues that the participants helped us to identify, but the testing also identified expectations from our target market that we did not forsee.
In regards to workflow, here are just a few examples of what the participants experienced:
- Despite a step-by-step visual guide, users didn’t understand how Wordy worked
A visual element might look nice, but not necessarily communicate a message effectively.
- Users felt uneasy when clicking on “Get Started” immediately presented them with a Sign up form
An example of how the target market’s expectations differed from the designer/developers’.
- Once a plan was chosen, there was no way to repurchase that plan again
An example of misplacement of options for a user that went unnoticed.
- and more…
These issues are almost always difficult to identify in-house. Having your target market verbalize their thoughts on your website is indispensable knowledge.
Heatmaps, Eye-tracking and Other Visual Data
What wasn’t verbalized by our participants was observed by our webcams, eye trackers and screencasts. This gave us even more insight as to how our target market reacted with the site.
1) Heat maps reveal why there was confusion on service offerings
The use of eye tracking gave us heatmaps – a visual representation of what areas of the site were focused on the most, with red being the highest concentrated areas.
For the Pricing page, participants tended only to read the first bits of information, which left them wondering what exactly was included in the purchase. Though more information was there, participants just didn’t get past the first inclusion of content.
Prices and first items only were the main focus on this page.
The main resulting comments after view this page was “I’m looking for a summary of what it really provides for me and I’m not getting that.” – confirming our analysis that participants were fixated on the intial generic overviews and disregarded everything else.
2) Reverse heat maps reveals unread content in Tour page
Despite having a tour and descriptions of the service, our participants still expressed confusion and stated that they were not sure how it worked. Looking at the reverse heat maps, it’s clear to see why.
Reverse heat maps are an alternative visualization of standard heatmaps that overlay an opaque black layer, with highly viewed areas being revealed.
This reverse heat map is an aggregate composition of all participants, that shows us what content was read on this page. Content was merely glanced over and barely read – no wonder people didn’t understand how the service worked!
The Next Steps
A report was compiled and presented to the Wordy and Gossamer team with suggestions and recommendations on how to overcome these usability issues. Having an actual UI expert counselling us on issues and problems was an immense help in polishing our final product.
With this, we go back to the drawing board and address the concerns of our target market. The next Wordy redesign will have an element of market feedback, fine-tuned to speak to them and most importantly, make sense to them.
It’s amazing what you can discover about your market’s expectations versus what your site actually delivers. And with eye-tracking technology, and real-time behaviour logging, usability testing can give you incredible insights on how to talk to your market more effectively.
Interested in Usability Testing?
Let’s talk about how you can reach your target market better »