By Shiva Naidu
Summary: This study examined the usability of educational websites for children. Children ages 7 – 11 performed seven search tasks with one of three websites. Overall, participants, especially those less than 10 years of age were not very successful. Terminology, number and organization of links, location of information above the fold, and length of individual pages all influenced performance on the tasks.
Children between the ages of 6-12 are using the Internet more and more as a resource for finding information for school projects. The UCLA Center for Communication Policy (2003) reports that approximately 75% of children who used the Internet in 2002 went online at their school, an increase of about 12% from 2000. Educational website creators have the challenge of designing for a wide range of ages and abilities. Arsenenault (2004) tested the usability of 24 commercial websites for children elementary-aged children in a private school in Montreal, Quebec. She found that while children preferred an interactive website, too many frames caused confusion and tables with elaborate listings and poorly organized links were cognitively demanding (especially for those under ten years old). The use of icon metaphors, excessive animation, and required plug-ins also were reported to be problematic to this population.
Bilal (2001) carried out an experiment with 17 children in an East Tennessee school using Yahooligans! as a test site. She found that browsing required less cognitive load than keyword searching. Searching by keywords often resulted in failure because of misspellings or inappropriate search terms. Bilal concluded that the children’s limited success on completing the assigned tasks was due to the Yahooligans! design not being based on children’s developmental level and cognitive abilities.
To further understand what aspects of educational websites influence performance and satisfaction, we evaluated childrens’ success, navigational efficiency, and satisfaction with three educational websites* – Enchantedlearning.com®, Factmonster.com™, and Infoplease.com’s Homework Center™ (see Figures 1- 3). EnchantedLearning.com™ is a site designed for children by Enchanted Learning Software. The Infoplease.com Homework Center and FactMonster.com are sites sponsored by Information Please, a division of educational publisher Pearson Education. These sites were chosen because they offered similar content but different interfaces to the users. FactMonster.com was a very colorful and graphical site while The InfoPlease.com Homework Center was primarily text-based. EnchantedLearning.com offered a mix of graphical and textual links and generous areas of whitespace.
Figure 1. EnchantedLearning.com
Figure 2. FactMonster.com
Figure 3. InfoPlease.com Homework Center
A total of 30 elementary-aged children (17 females, 13 males) ranging between 7 and 11 years of age (M = 8.67) volunteered for this study. Participants were recruited from local communities such as churches, scout troops, and after-school programs. All the participants had at least one year of experience using computers and the Internet specifically.
Participants were asked to complete a series of seven informational search tasks on one of the three websites. Age of the participants was balanced across websites. The tasks were as follows:
What is the state capital of Wyoming?
Use the search field to find out in what country The Great Wall is located.
Find a phone number or address which will get you in touch with the people who made the website.
What is the planet Mars named after?
Find a multiplication table that gives you the answer to 3 x 2.
Sir Edmund Hillary was a famous explorer. What was he famous for?
How do you say “Thank You” in Italian?
Tasks were presented in random order and took approximately 30 minutes to complete. After all tasks, participants were asked to complete a modified version of the System Usability Scale satisfaction survey (Brooke, 1986) using a “smiley-face” Likert scale derived from the Wong-Baker pain rating scale (1998) to rate their satisfaction with different aspects of the website. In addition, participants’ success, time on task, and search efficiency were measured. Search efficiency, or lostness, was defined by the number of pages traversed beyond the optimum number of pages to complete a task. Time and efficiency data was gathered by the tracking program Ergobrowser™.
Table 1 shows the percentage of participants that were successful on each task for each site. Factmonster.com had the highest rate of overall success (67.5%) followed closely by Enchantedlearning.com(62.8%) and then the Infoplease.com Homework Center(47.6%).
There were several tasks that seemed to suggest that the level of difficulty varied across website. Finding contact information proved to be extremely difficult on Enchantedlearning.com (10%) and the Infoplease.com Homework Center (33.3%) primarily because this information was located at the very bottom of the page. Participants using the Infoplease.com Homework Center also had trouble finding information on Mars, Sir Edmund Hillary, and the Italian language. The primary problem was the overwhelming number of link choices on this site and terminology that did not match what the participants’ were thinking. For example, the information on how to say thank-you in Italian was not found by any of the participants using this site. This information was located under the “Almanac > Writing and Language > Basic Phrases in Other Languages” links. Most participants looked for it under the “Speaking and Listening” link on the home page rather than “Almanac.” Participants looking for information on Sir Edmund Hillary on FactMonster.com clicked on the “People” link and then “People Fun Facts” instead of the correct link of “Biographies”. Those looking for this information on the InfoPlease.com Homework Center clicked on “Social Studies” or “History” instead of “Biographies”.
Table 1. Success rate for each task by website (bold indicates least successful tasks)
|Find the state capital of Wyoming||100||81.8||66.7|
|Use the search field to find location of Great Wall||80||100||100|
|Find site contact information||10||63.4||33.3|
|Find origin of Mars||70||63.6||33.3|
|Find a multiplication table||70||72.7||77.8|
|Find info on Sir Edmund Hillary||70||36.4||22.2|
|Find how to say “Thank You” in Italian||40||54.5||0|
Results from a one-way ANOVA showed that the participants using FactMonster.com were able to complete all seven tasks the quickest – an average of 7 minutes faster than the InfoPlease.com Homework Center (F(2,27) = 3.76, p<.05) (Figure 4). Time to complete all tasks on EnchantedLearning.com was not significantly different from the other two sites. Average lostness was not significantly different across sites with EnchantedLearning.com having an average of 3.3 pages beyond the optimal path, FactMonster having 4.2, and the InfoPlease.com Homework Center having 4.8 (F(2,27)=.884, p=.42).
Figure 4. Average time to complete all 7 tasks for each site.
Overall, participants rated EnchantedLearning.com, FactMonster.com, and the InfoPlease.com Homework Center equally well (26, 29, 29 respectively out of 44 total. The higher the number, the more satisfied the participants). The only individual satisfaction question that showed some difference across sites was in regard to the colors of the site. Participants using FactMonster.com were more positive about the colors than the other sites. Given that each participant worked with only one site (and had no other sites to compare it to), it was not too surprising that there were no differences in satisfaction. Our experience at SURL has been that children, in general, tend to be positive about a website even when they do not perform well.
Differences by Age
Across all three sites, it appeared as though the older children had an easier time finding the information than the younger children (see Figure 4). This could be due to the fact that they (i) had more experience in school using the Internet and (ii) they had more knowledge of what the correct answer to the questions were and (iii) they had an overall better vocabulary which helped them understand the site’s terminology. Despite this fact, all sites were designed for children in K-12. Educators of children in this age group (especially those less than 10) need to be very cognizant of a site’s content and terminology to make sure it is best suited for their classroom.
Figure 5. Success rate for each task by age
The results obtained suggest several things. Overall, the success rate on each website was low with this population. Factmonster.com was highest at 67.5%, which was better than the 43% with the InfoPlease.com Homework Center, but still not good odds for a child attempting to complete a homework assignment. The following were noted as important features that contributed to the participants’ overall success with the sites:
Page Length: Many of the web pages on all three sites were long. Participants, especially the 7 and 8-year-olds, tended not to scroll down the page even when reminded that they could do so before the start of each task. This was especially evident with the task to find contact information. Both the Infoplease.com Homework Center and EnchantedLearning.com had its “contact us” link at the bottom of the homepage. Factmonster.com’s higher success rate on this task was due to the fact that its Help icon was immediately visible at the top of the screen.
Too Many Choices: The Infoplease.com Homework Center offered many links on each page. For example, on the “Find the origins of Mars” task, once a user clicked on the Science category, there were many topics and sub-topics to choose from ranging from the earth sciences to technology. Users reported it to be “messy” and confusing, regardless of age. By contrast, Factmonster.com had fewer category links and was easier for participants to initially find a keyword related to what they were looking for.
Terminology: Enchantedlearning.com and FactMonster.com used simpler and more descriptive words to identify topics than the InfoPlease.com Homework Center. Factmonster.com used titles like “People” and “Word Wise” which were readily understood by the youngest children. However, they and the InfoPlease.com Homework Center also used terminology that was not as familiar to some children like “Biography” and “Almanac”.
Font size and pictures: The younger children liked the larger font size and pictures offered on Enchantedlearning.com. This was a contrast to the Infoplease.com Homework Center that used a smaller font size and few pictures to guide the children. While EnchantedLearning.com was found to be easy to navigate, the older children commented that the large graphics made the site appear to be “just for little kids.”
Depth of the site: The deeper the participant had to travel to find information, the more lost they became. The youngest children, especially, tended to use the Back button solely for navigation. Most were unfamiliar with the fastpath of a Back-to-Home logo or icon. Therefore, it was easy for them to get lost looking for a particular answer if it was not close to the homepage.
Advertisements: FactMonster.com, in particular, had a lot of animated advertisements on their site. This was very distracting to the participants, especially the 7-8 year olds. Some could not figure out how to close the ads (pop-ups) while others inadvertently clicked on a link within the ad and ended up on a completely different site.
Target Audience: Even though a website may be geared for K-12, this does not guarantee that all children in these age groups will be successful. As Figure 4 shows, the 11-year old children did much better on the tasks than the 7-year olds. This was found to be true across all sites despite the fact that the look-and-feel of FactMonster.com and EnchantedLearning.com appeared to be geared more towards younger children.
This study evaluated the usability of three educational websites. Participants were fastest with FactMonster.com and more successful with FactMonster.com and EnchantedLearning.com than the InfoPlease.com Homework Center. None of the sites proved to be superior in terms of efficiency or satisfaction. Terminology, number and organization of links, location of information above the fold, and length of individual pages all influenced performance on the tasks. Older children (age 11) performed better on all sites than the younger children (age 7-8). Designers of K-12 educational sites should continue to conduct usability testing with a wide range of children to insure ease of use for all students.
*Note: Sites named in this article were tested in April – May 2005. Site design and content may have changed since this evaluation.
Arseneault, C. (2003). Having fun or finding information? Usability for kids sections of Web sites. 7th Museums conference and the Web 2003, North Carolina, USA. Retrieved July 10th, 2005 from http://www.archimuse.com/mw2003/papers/arseneault/arseneault.html
Bilal, D. (2001). Children’s use of Yahooligans! Search engine. II. Cognitive and physical behaviors on research tasks. Journal of the American Society for Information Science and Technology, 52(2), 118-136.
Brooke, J. (1986) SUS: A Quick and Dirty Usability Scale. Retrieved 07/11/05 from http://www.usability.serco.com/trump/methods/satisfaction.htm
UCLA Center for Communication Policy (2003). UCLA Internet Report 2003: Surveying the digital future, Year Three, Los Angeles, 1-89.
Wong, D., & Baker, C. (1998). Pain in children: Comparison of assessment scales. Pediatric Nursing, 14(1), 9017.