Paid Website Usability Testers
Kazi Ziaul Islam
Human Computer Interaction
Prof. Kourosh Samia
December 8, 2014
Paid Website Usability Testers
Two attributes determine if a website is useful and they are usability and utility. Utility is whether or not the website provides the required information or features. Whereas, “usability is a quality attribute that assesses how easy user interfaces are to use.” (Nielsen, 2012, para. 1) You cannot have one without the other. Suppose your site is brimming to the top with lots of information and has all the bells and whistles, but if the Interface design (ID) is poor and as a result the User Experience (UX) ...view middle of the document...
This way you can test quickly and be more effective.
Usertesting.com is a website designed to recruit usability study participant for website owners. They give the opportunity for anyone capable of meeting some basic criteria to earn money to perform web testing or mobile testing. Currently, testers can earn $10 for web testing and $15 for mobile testing. While it’s typical for companies to provide monetary incentive to test participants, the rates they pay out can vary greatly depending on the demography and suitability of the user profile for the study. According to a report generated by the Nielsen Norman Group, The “average incentive paid to external users was $64 per hour of test time.” (Sova & Nielsen, n.d., p. 6) Typically high-end professionals can earn almost four times as much as non-professional users.
One of the question that inevitably arises when considering paying website usability testers is how reliable is the information that is gathered from them. “The reliability or consistency of a method is how well it produces the same results on separate occasions under the same circumstances.” (Rogers, Sharp, & Preece, 2011, p. 471) There are numerous studies that been done to discover this issue. The results are still inconclusive. While there are well-known cases that some of data may be skewed based on numerous factors, following good usability evaluation guidelines can mitigate these factors. For example, The National Institute of Standards and Technology (NIST) is an agency of the U.S. Department of Commerce, which provides such guidelines. There is the trend that some users tend to be more negative when online, however this can also mean that if you design the activity or questions properly you can typically elicit a more honest answer. Contrary to what people might think about in the wild studies, testing done through usertesting.com where multiple data points are being captured and analyzed and there can be fine granular control of the demography and profile of testers, the information can be considered pretty reliable.
Usability Evaluation Methods or UEMs were classified info four general categories by Nelsen and Mack: automatic, empirical, formal and informal. (Hasan, Morris, & Proberts, 2012, p. 707) The evaluation method being used by usertesting.com is a user-based UEM, where the user interaction is recorded via a screen recorder while the think-aloud protocol is used to capture user’s verbal comments. Mouse movements, clicks and keystrokes as well as spoken comments while the user is using the website is captured. Once the recording session has been completed, an online questionnaire is filled out by the user to gather reaction to the website and evaluate the user experience. Typically the four questions asked are as follows:
1. What frustrated you the most about this site?
2. If you had a magic wand, how would you improve this site?
3. What did you like about the site?