By S. Naidu, V. D. Hinkle, & S. Shrestha
Summary. This study evaluated the usability of three websites for Spanish-English Dual Language K-8 schools. Twelve participants (6 parents, 6 teachers) reviewed and performed tasks on the three public school websites. Site usability was determined through both objective and subjective measures, including task completion time, first-click, total number of pages visited, task success, perceived task difficulty, user satisfaction, and overall ranked preference. Results indicated that one site was preferred more than the others by both user groups and resulted in more efficient search behavior. Clear navigation, link terminology, and proper use of both languages were found to be critical factors contributing to the sites’ usability.
McKenzie (1997) says that a well designed school website has four goals:
1. Introduce the school to the visitors and to anyone who wants to know about the mission and objectives of the school.
2. Act as a resource center offering informational links from the World Wide Web that is relevant and educational to the teachers, parents, and students.
3. Act as a platform to publish student work.
4. Provide relevant and informational content locally.
To meet these goals, the needs of the end-users must be understood. This can be complicated for a school website because there are a number of unique user groups including parents, teachers, visitors, and students. While the needs and expectations of these user groups are very different, they all need to be taken into consideration while designing the school site. A dual language school introduces additional challenges because it delivers its curriculum in two languages. The purpose of such schools is to provide an educational curriculum whereby students can graduate proficient in reading, writing, and conversing in both languages. One goal of a dual language school website is to provide school information while at the same time conveying the bilingual nature of the school. This is typically done by presenting the site content in both languages.
The purpose of this study was to assess and compare the usability of three dual language (English/Spanish) school websites found in the United States. We evaluated the school websites for the Horace Mann Dual Language Magnet School (Wichita, KS), the Adelante Spanish Immersion School (Redwood City, CA), and the Amigos School (Cambridge, MA). Figures 1-3 show the homepages for these school websites.
Figure 1. Horace Mann Dual Language School Home page
Figure 2. Adelante Spanish Immersion School Home page
Figure 3. Amigos School Home page
A total of 12 participants ranging between 29 and 63 years of age (M = 41.50) volunteered for this study. Six participants were elementary school teachers and 6 were parents of elementary-aged children. None of the participants were affiliated with any of the schools tested. Participants were recruited from local communities such as elementary schools and the YMCA. Seventy-five percent of the participants reported using the Internet on a daily basis for work and educational purposes.
Participants were asked to complete nine basic tasks on all three of the websites. The tasks were as follows:
- What is a dual language program?
- What types of classes are typically offered in the second language?
- What is the school philosophy or approach to teaching a second language?
- Find a link that provides information to help with a child’s homework.
- Your son/daughter is entering third grade this school year. Find the third grade teachers’ names.
- You would like to know more about parent volunteer opportunities at the school. Find volunteer programs in the school (i.e., a parent-teacher organization).
- Your family is going to Cancun for Christmas this year. Find out when winter recess begins and when is the first day of school next semester.
- You noticed an error in some of the information on the site and would like to report it. Find the person responsible for fixing errors or anything related to the website.
- Find out the name of the school principal(s) and his/her e-mail.
Tasks were presented in random order for each of the three sites, and the order of the sites was counterbalanced across participants. After each task, the participants were asked to provide a difficulty rating (1 = Very Easy and 5 = Very Difficult) for that particular task. In addition, after all tasks were completed, participants were asked to complete a modified version of the System Usability Scale (Brooke, 1996) to evaluate their satisfaction with varying components of the website they had just used. Task success, time on task, and total number of pages visited were also collected. The number of pages visited helped determine the efficiency of use by comparing it to the optimal number of pages necessary to complete the task. Both time on task and the number of pages were gathered using Ergobrowser™ (2001).
Table 1 shows the percentage of participants that were successful on each task for all three websites. Success rates were fairly high overall and did not differ significantly across the sites (F(2,10) = 1.87, p > .05). Finding a link for homework help was the most difficult task across all three sites, especially for the Amigos site (16.7% successful) and the Horace Mann site (41.7%). Participants either expected this information to be behind a parent information link or to be clearly labeled "Homework/School Help". The Horace Mann link for homework help was named "Hawk’s Nest" after the school athletic teams, which none of the participants were familiar with. The Amigos site link was called "Library" which participants found misleading. The relative success (66.7%) on Adelante’s site was due to the fact that this information was behind a link labeled "Useful Links".
Finding the types of classes offered in the second language was difficult on the Adelante site (41.7% successful) primarily because this information was only vaguely mentioned within the text.
The task requesting the users to find information on a parent volunteer group also resulted in lower success on the Adelante site (33.3%). The parent-teacher group was referred to as "UNIDOS" which confused the participants who did not speak Spanish. Participants also had trouble finding the principal’s name and email on the Adelante site (58.3%) because there was no direct link to the principal’s information.
Table 1. Percent of participants successful on the tasks on each website (bold indicates least successful tasks)
|What is a dual language program||83.3||83.3||83.3|
|Types of classes offered in second language||91.7||66.7||41.7|
|School philosophy in teaching second language||66.7||91.7||83.3|
|Link to help with homework||41.7||16.7||66.7|
|Third grade teacher’s name||100||100||91.7|
|Parent volunteer opportunities||83.3||66.7||33.3|
|Start of winter recess and first day of school||91.7||91.7||91.7|
|Contact someone for web help||83.3||91.7||91.7|
|School principal’s name and email||75||91.7||58.3|
Navigation Efficiency (Speed and Number of Pages visited)
While participants took the least amount of time to complete the tasks on the Horace Mann website, there was no statistically significant difference across sites (Figure 4) (F(2,10) = 1.61, p > .05). In addition, participants clicked, on average, 1.68 pages beyond the optimal path on the Horace Mann site, 2.61 pages beyond the optimal path on the Amigos site, and 3.81 pages beyond the optimal path on the Adelante site.
Figure 4. Average time for participants to complete all tasks
Satisfaction and Preference
Overall, Horace Mann was rated the highest in overall satisfaction (79.6 out of 100), followed by a 60.4 rating for Amigos and a 49.2 rating for Adelante (F(2,10) = 4.71, p < .05). When asked which site they preferred the most, most participants chose the Horace Mann site. Parents had unanimous agreement over the most preferred website (100% chose Horace Mann) while teachers showed less consensus (67% chose Horace Mann).
User Group Differences
Observation of the teachers and parents as they worked through the tasks revealed that each user group has specific expectations for where things should be in a website. Most of the teachers in this study had more experience dealing with school websites and showed more flexibility and patience when searching for answers. Parents tended to be less experienced with using school-related websites and became easily frustrated when they could not find the information quickly.
On average, teachers showed a trend for higher success on the tasks across all websites than the parents (Figure 5); although the difference was not statistically significant. In addition, teachers visited fewer overall pages for task completion when compared to parents, F(1,8) = 6.67, p = .03 (Figure 6).
Figure 5. Overall Task Success rate by user group. In general, teachers were more successful than parents.
Figure 6. Average total number of pages visited by participants to complete all tasks.
Results from this study revealed several design issues that impacted both performance and satisfaction of the dual language sites.
Link names that were in a foreign language (e.g., "UNIDOS") were problematic, because the non-Spanish-speaking parents and teachers did not know what they represented. Also, links that were too specific to the school were misunderstood. For example, users had difficulty finding the student homework help. On the Horace Mann site this information was under the link named Hawk’s Nest. While this task may not have been problematic to actual parents of the school (who are familiar with the school mascot), it is important to consider how visitors will use the site. School websites are often used as a marketing tool for future students and teachers in addition to an information repository for current students.
Display of Bilingual Information
All of the sites used in this study displayed the site content in both English and Spanish. While most of the participants in this study were monolingual, they liked having the Spanish information available since the schools promoted dual-language programs. The only time users disliked the use of the foreign language was when it was used as a link name on both the Spanish and English portions of the site. Web designers of dual language school websites cannot assume the parents of children at the school will be bilingual.
Another important consideration of the bilingual content is the translation from one language to the other. The terminology needs to be familiar to the users in either language. Simply translating one word into the other language does not always provide a comprehensible link name. One example of this is the "about us" link. In Spanish, the literal translation ("Sobre Nosotros") sounds awkward to a Spanish-speaker as a link name. The correct term would be "Conocenos." Using the most accepted terminology on the second language ensures that native speakers easily recognize the links and understand what they may be about. A native speaker should be consulted for proper translation or ideally be part of the web design team.
This study examined the usability of three dual language school websites for both teachers and parents. Clear navigation, link terminology, and proper use of both languages were found to be critical factors contributing to the sites’ usability. Parents and teachers differed slightly in their site expectations and satisfaction which reiterates the importance of gathering user requirements for all potential user groups during design.
Brooke, J. (1996). SUS: A Quick and Dirty Usability Scale, in P. Jordan, B. Thomas, B. Weerdmeester, & I. L. McClelland (eds.), Usability evaluation in industry, (pp. 189-94). London, UK: Taylor & Francis.
Ergobrowser™, Ergosoft Laboratories © 2001.
McKenzie, J. (1997). Why in the World Wide Web? (Available online http://www.fno.org/mar97/why.html)