By Barbara Chaparro, T. St. Romain, & R. Hacker
In our last issue of Usability News, we reported on the trials and tribulations of shopping on the web for electronics, discount items, and toys (Chaparro, Childs, Praheswari, & Rappard, Winter/2001). Statistics continue to show staggering numbers of users abandoning their shopping carts before making a purchase – many times due to ‘poor user experience’ (Gordon, 2000). In our continuing quest to further understand how people shop online, we investigated sites from two different domains that are of interest to today’s university students: grocery shopping and student textbooks1.
To do this, we evaluated participants’ user satisfaction, navigational efficiency, and general preference for three sites in each domain. Efficiency was measured by the degree of ‘lostness’, or the ratio of optimum number of pages to complete a task to the actual number of pages traversed by the participants. A lostness ratio of a 1.0 indicated participants traversed the optimal path. A lostness ratio closer to 0 indicated that users traversed many more pages than the optimal path (and therefore would be considered to experience more lostness). This efficiency data was gathered by the tracking program ErgobrowserTM.
After completing the tasks with each site, participants answered a satisfaction questionnaire. After completing the tasks with all three sites, participants ranked the sites in order of preference. A Pentium II based PC computer, with a 60 Hz, 96dpi 17" monitor with a resolution setting of 1024 x 768 pixels was used. The following is a summary of what we found:
Online Grocery Stores (peapod.com, homegrocer.com, albertsons.com)
Eight participants volunteered for a usability study with peapod.com, homegrocer.com, and albertsons.com. All participants were familiar with the web, but had not visited an online grocery store before. Participants were asked to complete three tasks with each site (site order was counterbalanced across participants). The three tasks included finding four grocery items for an Italian-themed dinner party. Once all items were found, the participants were instructed to remove one item from the shopping cart because they had insufficient funds. (This allowed us to observe how the participants worked with the shopping cart.) Participants were also asked to complete the tasks without using the site’s search function so we could better understand how users’ navigate the site’s structure.
Usability Results for Online Grocery Stores
Peapod.com and homegrocer.com were found to be significantly more satisfying than both albertsons.com [F(2, 12) = 11.97, p <.01] (see Figure 1), as well as being the most preferred [Friedman Χ2 (2, N = 6) = 6.33, p < .05]. The navigational efficiency, or lostness, of the three sites were about equal: peapod.com = .627, homegrocer.com = .605, albertsons.com = .526.
Figure 1. Satisfaction scores for the 3 online grocery stores.
Participants stated that the reason they preferred peapod.com and homegrocer.com over albertsons.com was due to the overall grocery item structure – albertsons.com used a long textual menu structure that was "counterintuitive", hard to read (due to a very small font), and required scrolling (see Figure 2). In addition, the menu had no dairy section, which made completion of one of the tasks (i.e., find Parmesian cheese) very difficult. The two other sites used a more ‘intuitive’ graphical menu structure and larger text font that was reported to be easier to read (see Figure 3 for peapod.com).
Figure 2. Albertsons.com Menu Structure was reported to be ‘counterintuitive’ and hard to read.
Figure 3. Peapod.com’s graphical menu structure was ‘intuitive’ and ‘easy’ to navigate to the requested item categories.
Student Textbooks (textbooksatcost.com, varsitybooks.com, ecampus.com)
Eight participants were run in a usability study with textbooksatcost.com, varsitybooks.com, ecampus.com. Participants were asked to complete three tasks with each site (site order was counterbalanced across participants). The three tasks included finding two specified new books by title, finding one specified used book by ISBN, and determining the shipping costs on orders of $50 and greater than $100.
Usability Results for Online Textbook Stores
Ecampus.com was found to be significantly more satisfying than varsitybooks.com [F (2,14) = 6.07, p < .05 ], as well as the most preferred [Friedman Χ2 (2, N = 8) = 6.25, p < .05]. The navigational efficiency, or ‘lostness’, of the three sites were not statistically different, though there seemed to be a trend for ecampus.com and textbooksatcost.com to be more efficient: ecampus.com = .74, textbooksatcost.com = .72, varsitybooks.com = .59.
The reasons for preference of ecampus.com over varsitybooks.com mainly involved the ease with which the users were able to search and find the requested books and shipping information. Both ecampus.com and textbooksatcost.com used ISBN as the default search criteria. Searching by ISBN is far more efficient than title or author searches. Varsitybooks.com used ‘Title’ as the default search criteria, which is what several of the participants initially attempted since it was the default. Results from the title searches yielded a variety of unsuccessful results across all sites. In addition, participants reported that it was difficult to find both used and new versions of the books (or even determine if both were available) on varsitybooks.com. Both ecampus.com and textbooksatcost.com displayed ‘new’ or ‘used’ and corresponding price next to each title. Shipping price information was reported easy to find on both ecampus.com and textbooksatcost.com home page. Several participants had trouble finding the shipping information on varsitybooks.com as it was under a Customer Service FAQ link.
Results from this series of usability studies demonstrated that the dot.com sites tended to outperform their brick-and-mortar competitors. For both the grocery and textbook domains, the clarity and intuitiveness of each site’s home page was critical to participant success. On the albertsons.com site, participants struggled with the task of finding cheese because there was no dairy category identified. On the varistybooks.com site, participants struggled to find the cost of shipping charges because it was not evident on the home page. Other problems reported included small, hard-to-read fonts, counter-intuitive menu structures, and incomplete information. Such problems are not difficult to fix, but could have a profound impact on whether users continue with a site and more importantly return to it again. Usability testing is a fast, cost-efficient method to help developers find these problems. It doesn’t seem plausible that an e-commerce company can afford not to do it.
Chaparro, B., Childs, S., Praheswari, Y., & Rappard, A. (2001) Online Shopping: TV’s, Toasters, and Toys! Oh My! [Online]. http://usabilitynews.org/online-shopping-tvs-toasters-and-toys-oh-my/
Ergobrowser®, Ergosoft Laboratories © 2001
Gordon, S. (2000). Shoppers of the web unite: user experience and ecommerce. [Online] http://www.zdnet.com/devhead
1 All sites were tested in March – April of 2001. Site design may have changed since this time.