The number one complaint reported by Internet users continues to be download speed or taking too long to load web pages (GVU web site, 1998). It has been reported that users are more likely to lose interest in a site if the download time exceeds 10 seconds (Nielson, 1996). Recent studies have investigated how perceptions of websites are influenced by delays in retrieval times (Ramsay et al., 1998; Sears et al., 1997). Pages with longer delays were rated as less interesting and more difficult to find. Thus, users’ subjective evaluations of webpages can be affected by retrieval delays, but what happens to performance on the website?
To explore this question, we examined the effect of delays on user performance on a website. To do this, users located task information on an airline website that had been previously tested and ranked high on preference and performance measures (Selvidge, 1999). The website was downloaded to control the delay lengths of page loading time. The first experiment examined delay lengths by varying the delay time (1, 10, and 20 seconds) before users could view a particular webpage.
This study measured degree of “lostness”, task success, frustration, and task difficulty as a function of delay time. The lostness measure, which is the users’ ability to find specific information, is the ratio of the optimal number of nodes required to complete a task to the actual number of nodes visited while searching for task information (Smith, 1996). The closer the lostness value was to 1.00, the less lost the user was on the website. It was found, however, that delays did not affect lostness, frustration or task difficulty ratings. Moreover, users did not successfully complete the task in the 1 second condition significantly more than the other delay levels (mean values are presented in Tables 1 & 2). These findings suggested that perhaps the delays were not long enough. Consequently, in a second experiment, delay lengths were increased to 1, 30, and 60 seconds and were varied as a within-subjects design. In addition, a cash incentive was introduced in order to reward users with the fastest completion times across tasks. This was done to focus users more on download time, as well as increase the incentive for users to perform at the best of their abilities.
In this experiment our results indicated that longer delays produced greater frustration, with significant differences between 1 and 30 seconds and 1 and 60 seconds. Lostness and task difficulty were not affected by delay length. Longer delays did affect the frequency of task success, with users in the 30 and 60 second delay conditions not successfully completing the task more than those in the 1 second delay condition. In addition, more users quit the task in the 60 second delay condition than 1 or 30 second delays (mean values are shown in Table 3 & 4).
Lostness was not affected by the longer delay conditions, but the results were mixed on task success across the two experiments. Users were frustrated by the 30 and 60 second delays in page loading time, but would tolerate the 20 second delays. Based on these results, further support is provided for designing websites that decrease download time, such as limiting unnecessary graphics. The longer the wait for pages to load, the greater the frustration, which could lead users to quit the task or try another site to find information.
Table 1: Lostness, Frustration, & Task Difficulty Means for Experiment 1
|1 second||10 seconds||20 seconds|
*Lostness equals the number of nodes required to complete a task (optimal path) divided by the number of nodes visited. The closer the value is to 1.0, the less ‘lost’ the user was.
*Frustration & Task difficulty are on a 5-point scale with 1 = no frustration and very easy and 5 = lots of frustration and very difficult.
Table 2: Task Success for Experiment 1
|Number of Participants||1 second||10 seconds||20 seconds|
Table 3: Lostness, Frustration, and Task Difficulty Means for Experiment 2
|1 second||30 seconds||60 seconds|
*Lostness equals the number of nodes required to complete a task (optimal path) divided by the number of nodes visited. The closer the value is to 1.0, the less lost the user was.
*Frustration & Task Difficulty are on a 5-point scale with 1 = no frustration and very easy and 5 = lots of frustration and very difficult.
Table 4: Task Success for Experiment 2
|1 second||30 seconds||60 seconds|
* 2 missing values in 30 second delay condition
Note: A paper based on this work can also be found in the Human Factors and Ergonomics Society 44th (2000) Annual Meeting conference proceedings, San Diego, California.
Georgia Tech Research Corporation (1998). GVU’s 9th WWW User Survey [On-line]. Available http://www.gvu.gatech.edu/ user_surveys/survey-1998-04.
Nielsen, J. (1996) Top ten mistakes in web design. Alertbox, http://www.useit.com.
Ramsay, J., Barbesi, A., & Preece, J. (1998). A psychological investigation of long retrieval times on the World Wide Web. Interacting with Computers, 10, 77-86.
Sears, A., Jacko, J., & Borella, M. (1997) Internet delay effects: How users perceive quality, organization, and ease of use information. CHI 1997 Proceedings, http://www.acm.org/sigchi/chi97/proceedings.
Selvidge, P. (1999). Reservations about the usability of airline web sites: A comparison of seven airline web sites. Poster session presented at the annual meeting of CHI 1999, Pittsburgh, PA.
Smith, P. (1996). Towards a practical measure of hypertext usability. Interacting with Computer, 8 (4), 365-381.