Summary. One of the most common resources for health information is the Internet. On some websites, users are able to input personal symptoms into a Symptom Checker. The Symptom Checker provides a description of possible conditions based on the described symptoms. This study compared the usability of the Symptom Checker on two popular health websites, WebMD.com and HealthCentral.com, along with general search functionality for medical conditions. Overall participants were more satisfied with WebMD.com than with HealthCentral.com. This was primarily due to the ease of use of the Symptom Checker and clarity of the medical information provided.
One goal of e-health is to provide quality information on the Internet. Many people explore health websites to get information on a specific condition, to see if they need a physicians’ appointment, or to see if they can find some treatments on their own (Gualtieri, 2009). These websites can be used as a first opinion for diagnosis. However, many patients do not report to their physician that they have conducted research online because they feel unsure of the information, confused, or disrespectful if they speak out (Gualtieri, 2009). On the other hand, physicians do not ask patients if they have explored health websites during appointments because of possible confrontation, short appointment times, or because of the awkwardness associated with asking such a question to a patient that is paying them for a diagnosis.
It is estimated that 50-80% of internet users search for health information on the internet (Miller & West, 2009). The Pew Internet and American Life Project (Fox & Jones, 2009) found that 61% of respondents reported using the internet for health information, and that 52% searched for this information for someone other than themselves. With online capabilities rising, an astonishing 57% reported that e-health information influenced their treatment options. Forty-one percent searched for information on news groups and support blogs, while 24% used information provided by physicians and hospitals. This is not to say that physicians are being replaced. In fact, 53% of users reported that their internet search lead them to either inquire more information from a physician or make an appointment.
Despite the surge in e-health participation, websites are often found to be unreliable and lead to consumers being concerned about the quality of information they read on health websites (Oermann & Wilson, 2000). Users also have reported that they felt uncertain whether they should trust e-health material in their decision making (Hesse & Shneiderman, 2007). In the Pew Internet and American Life Project, 89% of respondents reported that they were afraid that the companies sponsoring such sites would sell their demographic information and 86% were afraid that the information was not reliable (Fox & Jones, 2009).
The purpose of this study was to evaluate the usability of the WebMD.com (WebMD) and HealthCentral.com (HealthCentral) (see Figure 1 and Figure 2 respectively) websites for usability, satisfaction, and preference. Specifically, the Symptom Checker and general search capability was examined on each site. Both sites provide an interactive Symptom Checker component to help users diagnose medical conditions. In addition, they both provide information regarding the etiology, prognosis, and treatment options for a wide variety of conditions (see Table 1).
Figure 1. WebMD Symptom Checker.
Figure 2. HealthCentral Symptom Checker.
Table 1. Website Features
|Interactive Symptom Checker||X||X|
|Allows user to select affected area of the body||X||X|
|Asks additional questions, which allows users to clarify the specifics of the symptom.||X|
|Provides information regarding etiology, diagnostic evaluations, and treatment/prevention techniques.||X||X|
|Provides participants with a link to experts and community members.||X|
Participants were 16 volunteers (10 male, 6 female; M Age = 21.87, SD=5.09) from Wichita State University. Participants reported spending an average of 17.5 hours per week on the computer. Of that time, participants reported using the internet to search for health information an average of 12 times per year. Thirty eight percent of participants reported that the health information on the internet did not impact their medical decisions, 31% reported that it did impact their medical decisions, and 31% reported that it had some impact. None of the participants had prior experience with the Symptom Checker of either website.
A single Pentium Core 2 Duo computer running Windows XP Professional with a 17-inch LCD monitor at a 1024 x 768 resolution was used in this study. Participants accessed the WebMD.com and HealthCentral.com websites using Mozilla FireFox, version 3.0.8 during the April-May 2010. A web camera and Morae™ 3.1 were used to record page changes (number of times the screen changed) for each task. Comprehension quizzes covering material the website provided for a specific health condition, satisfaction survey, and preference survey were created and recorded using Google docsTM. The satisfaction survey was made up of selected questions from the SUS (Brooke, 1996).
All participants provided informed consent and answered background and internet usage questions. Participants were randomly administered four different tasks one both websites. The conditions used in the tasks included inputting symptoms for an ACL sprain, meniscus tear, ankle bursitis, and performing a general search for tennis elbow. The order of the websites was counterbalanced across participants. All tasks were written and given to the participants on note cards to read while using the websites. Participants then read aloud four tasks in all of which described a scenario where a person had sustained some injury and were given the related symptoms. They were asked to assume the role of the injured person and place the provided symptoms in either the WebMD or HealthCentral Symptom Checker. Once the symptoms were recorded, they were asked to choose which condition they thought they had based on the website’s list of conditions. Participants read the information provided by the site for the condition associated with each task.
After reading the information, each participant was asked to rate the ease of understanding the information presented on a 5-point Likert scale, (1= very easy, 5= very difficult). Next, participants completed a 4-question comprehension quiz pertaining to the information presented on that website for that particular health condition. After all four scenarios were completed, participants were asked to complete the satisfaction scale pertaining to their interaction with that particular website. This process was repeated for each of the four scenarios on the other website. Once the tasks and surveys were completed for both sites, participants were asked which website they preferred and why. Given the similarity of the information in both sites, comprehension was only measured after the first site exposure.
The Symptom Checker of WebMD was preferred by 15 of the 16 participants. In particular, users liked the manner in which they were able to select the symptoms and further clarify the symptoms on WebMD more than on HealthCentral.
Users reported that they perceived WebMD to be easier to use and more consistent than HealthCentral t(15)=2.15,p<0.05; t(15)=-2.44, p<0.05, respectively. Users also rated WebMD (M=5.38, SD=0.71) overall more “user-friendly” than HealthCentral (M=4.56, SD=1.26), t(15)=2.36, p<0.05. This was based on the statement “Overall, I would rate the user-friendliness of this website as:” with a 7-point scale (1= Worst Imaginable, 2=Awful, 3=Poor, 4=OK, 5=Good, 6=Excellent, and 7=Best Imaginable).
The lack of consistency on HealthCentral was primarily due to the presence of advertisements throughout the web pages. Participants reported that the advertisements were often placed within the informational text area which resulted in the unbalanced use of white space on the page. This impacted their ability to read the information. The layout of the WebMD pages remained consistent throughout the site and did not include advertisements.
The number of webpage changes was examined to determine if there was a difference between the user’s path on WebMD and HealthCentral to reach the correct diagnosis. There were no differences between the two websites on the number of web page changes, except for the task regarding the ACL injury condition. In this task, participants had significantly fewer page changes beyond the optimal path for WebMD (M=0.37, SD=0.88) compared to HealthCentral (M=2.62, SD=1.66), t(15)=-4.70, p<0.001. On WebMD, users were able to scroll through a text box to read the information regarding the condition, while on HealthCentral, users navigated more pages above the optimal path in order to read the information.
An independent samples t-test was conducted to evaluate whether the comprehension of the material presented on WebMD or HealthCentral was different (score out of a possible four on each quiz for a total of 16). There was no difference in comprehension found between WebMD (M=10.10, SD=2.33) and HealthCentral (M=11.17, SD=1.83), t(15)=0.95, p=0.96.
WebMD Usability Issues
The most beneficial feature of WebMD was reported to be the general functionality of the Symptom Checker. Users were able to easily choose symptoms from a list that were relevant to the particular condition and were asked a series of clarifying questions based on their symptoms. Many users reported that this interaction was somewhat like visiting their physician. After all of the relevant symptoms were entered, a list of conditions associated with their symptoms was displayed (see Figure 3).
Figure 3. WebMD allows users to select their symptoms and related conditions.
1. Medical Terminology as Links
One issue that was not positive for users involved the way in which the site described each condition (Figure 4). Many key terms and reference diagrams in the descriptions were represented as embedded links throughout the text. Clicking these links brought up a new browser window. Users reported there were too many links which resulted in too many open windows and that reading across these windows was disruptive to their understanding of the condition.
Figure 4. WebMD provided many links and diagrams to explain terminology.
2. Physician Referral
Users reported that the WebMD site may not enforce the need for the user to still seek advice from a physician for proper diagnosis and treatment. Many reported that the website seemed very professional and appeared as though it was generated by medical professionals. However, they thought that it still left the diagnosis decision with the user, who may not be qualified. Despite this, many users reported that it offered an adequate first look at what their health issue may be and what questions to expect when at their physician’s office. It should be noted that since our study, WebMD has added a warning to users who may need to seek medical attention when clarifying their symptoms (Figure 5).
Figure 5. Since this usability study, WebMD has added a message to users to seek medical attention if a symptom is a result of a recent injury.
3. Broken Links
On WebMD, users encountered several links which lead to an error page on the site. For example, clicking the "bursitis" condition in the list of conditions generated an error. This was frustrating to the users and caused them to abandon their path within the Symptom Checker and use the site search function to find the information.
4. Symptom Checker First Impressions
Users initially had difficulty finding the Symptom Checker from the WebMD homepage. They expected it to be left of the navigation menu of the homepage, however, it was placed on the right (Figure 6). Once found, they also did not realize that they were able to click directly on a body region to start identifying symptoms (Figure 7). They eventually were able to learn this through trial and error but indicated that some initial instructions may be helpful. It should be noted that since this usability study, WebMD has added instructions on how to use the Symptom Checker.
Figure 6. WebMD’s Symptom Checker location (outlined in Red). Users initially had difficulty finding the link to the Symptom Checker (right). Also, most participants did not realize that there was a direct link to the Symptom Checker under the search option.
Figure 7. WebMD allowed users to click on the area of the body and then zoom in on the particular joint within that general area.
5. Information Display
Users were disappointed somewhat with the small area of text used to describe the conditions. Even though there was a link to display a full article in a separate window, users reported that the default display should be larger since it required a lot of scrolling to view its content (Figure 8).
Figure 8. Users would like to see WebMD automatically display the information in full text and also decrease the amount of medical terms with links as they were excessive.
HealthCentral Usability Issues
1. Symptom Checker First Impressions
Compared to WebMD, participants were not as satisfied with the HealthCentral Symptom Checker. They found the follow-up questions to symptoms to be broad and the response choices sometimes confusing (Figure 9). Users wanted the ability to clarify their symptoms further than what the site allowed. Many reported that because of this, they did not feel that their symptoms were fully evaluated by the website. Also, although the Symptom Checker allowed users to select which area of the body to examine, users reported that it was not intuitive initially to use (Figure 10). This was because they were able to click on an area of the body to begin (e.g., leg), but then had to click on a text link to further refine the specific area of the body (e.g., ankle). Most tried to click on the body image again to refine their search.
Figure 9. HealthCentral asked broad questions and gave limited responses.
Figure 10. HealthCentral allowed users to choose the area of the body to examine, but there were few instructions as to how to narrow down the area from a region (leg) to a specific area (ankle).
Users were turned off by the large number of advertisements and unbalanced white space throughout the website (Figure 11). On some pages, the medical information was positioned on the right of the screen with very short line lengths to accommodate the use of advertisements in the middle of the page. Participants reported this to be very distracting.
Figure 11. HealthCentral had too many advertisements and unbalanced white space on their pages.
3. Expert Advice and Community Ties
HealthCentral provides users a way to “Ask a Question” to experts and community members. Users liked this feature and indicated that they would probably use it to seek advice on their own conditions.
The results of this usability test reveal that the interactive features of the WebMD Symptom Checker, ease of selecting body parts and symptoms, ability to clarify symptoms with additional questions, and the breadth of information on various health conditions were more preferred than the HealthCentral Symptom Checker.
Although both websites were interactive, WebMD was found to have higher overall usability, satisfaction, and ease of use for diagnosing specific symptoms. It must be noted that these results reflect first-time usage of the sites. It is believed that as users become more acquainted with these websites that they would become more accustomed to their content and navigation.
Based on these results, the following are recommendations for improving the usability of e-health websites:
Symptom Checker. Interactive Symptom Checkers are very popular among users. The feature should be prominently displayed on a site homepage and very easy to find. It should also be intuitive to launch and provide instructions for first-time users. Follow-up questions
to clarify symptoms are recommended since they make the user feel more comfortable in the proposed diagnosis. At the same time, recommendations for the user to seek medical advice from a physician should be provided.
Medical terminology. Users found the descriptions of various conditions on the WebMD site to be easier to understand than HealthCentral. It is important that the sites use less medical terminology and more layman’s terms when possible. Terms should be described in the text as much as possible and accompanying diagrams should be provided where helpful. Providing this information within a single browser window may reduce the chance of users getting lost within the site and to maintain optimal flow while reading about a condition.
Advertisements. Users were frustrated when ads were positioned within the informational text. Web page content should be designed such that advertisements are positioned separate from the informational text and are visually distinct.
Brooke, J. (1996). SUS: A Quick and Dirty Usability Scale, in P. Jordan, B. Thomas, B.
Weerdmeester, & I. L. McClelland (eds.), Usability evaluation in industry, (pp. 189-94).
London, UK: Taylor & Francis. Retrieved from
Fox, S., & Jones, S. (2009.) The social life of health information: America’s pursuit of health
takes place within a widening network of both online and offline sources. Pew internet. Washington DC. 2-72. Retrieved from http://www.pewinternet.org/Reports/2009/8-The-Social-Life-of-Health-Information.aspx
Gualtieri, L.N. (2009) The Doctor as the Second Opinion and the Internet as the First. In D.R. Olsen Jr., R.B. Arthur, K. Hinckley, M.R. Morris, S.E. Hudson, and S. Greenberg (Eds.). Proceedings of the 27th International Conference on Human Factors in Computing Systems, CHI 2009, Boston, MA. Retrieved from http://lisaneal.wordpress.com/2009/02/16/the-doctor-as-the-second-opinion-and-the-internet-as-the-first/
Hesse, B.W., & Shneiderman, B. (2007.) eHealth research for the user’s perspective. American
Journal of Preventative Medicine, 32(5); 97-103. Retrieved from http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1939873/?tool=pubmed
Oermann, M.H., & Wilson, F.L. (2000.) Quality of care information for consumers on the
internet. Journal of Nursing Care Quality, 14(4); 45-53. Retrieved from http://web.ebscohost.com/ehost/pdfviewer/pdfviewer?vid=4&hid=8&sid=d0e54d78-ab0f-44a2-94da-561f1fcdb530%40sessionmgr12