Usability Evaluation of Three Social Networking Sites

Doug Fox & Shiva Naidu

Summary. Social networking sites have quickly become one of the most popular means of online communication. Users can quickly share photos, videos, and communicate to friends, family, and colleagues via a social networking site. This study evaluated the usability of three of the most popular social networking sites (MySpace, Facebook, and Orkut) with first-time users. Results revealed issues related to confusing terminology, inadequate feedback and error messages, and improper link location impacted user performance and satisfaction.

Introduction

Social networking sites (SNSs) have quickly become one of the most popular mediums of online communication. Boyd and Ellison (2007) define SNSs as web-based services that allow users to share a public or private profile with common users and explore connections with others within the site. Using the Internet to chat with friends or meet new people is nothing new. The unique contribution of SNSs is the ability to see others social networks, which can serve as a way to extend one’s own social network.

Owyang (2008) estimated that there were 110 million active users for MySpace in 2008. There is an estimated 175 million users of Facebook (http://www.facebook.com/press/info.php?statistics). In fact MySpace has 50 million mail messages per day (more than Google, Yahoo, or Hotmail), 14 billion comments on the site, and 10 billion friend relationships (Owyang, 2008). This is remarkable growth considering that the Nielsen net ratings from 2006 estimated a total of 68.6 million users of all SNSs, with MySpace users comprising of 38.4 million of the SNSs population. Facebook is equally as popular considering it is the sixth most trafficked site in the United States, and active users double every 6 months (Owyang, 2008).

Unlike most web pages, which are organized by content, SNSs are organized by user interests (Mislove, Marcon, Gummadi, Druschel, & Bhattacharjee, 2007). Within minutes users can sign up and share thoughts, pictures, and videos with their connections (friends, family, co-workers, etc.) based on shared interests. There is a SNS for everyone; from MySpace for more youthful users to Dogster for dog lovers. Not only are SNSs divided by interest, but by cultural differences. While MySpace and Facebook are highly popular in the U.S, Orkut is highly popular in Brazil and India.

Even though the basic premise of each SNS is similar, each SNS has unique applications for profile sharing and applications that may make networking easier or more difficult for users. Assessing the usability of these SNSs is very important. Hart and associates (2008) have demonstrated that SNSs such as Facebook rate very poorly on their adherence to usability guidelines. Based on Nielsen’s ten heuristics, only 2 out of 10 were satisfactory for Facebook. The lack of usability for SNSs could lead to high drop out rates for users. Brandtzaeg and Heim (2008) found that the lack of usability was the third highest reason for SNS abandonment, with 18% of those surveyed reporting it was a significant reason for not returning to a SNS. Some of the usability issues included difficulty changing profile information, unusable interfaces, and difficulty learning the services available (Brandtzaeg and Heim, 2008).

Purpose

The purpose of this study was to evaluate the usability of three of the most popular SNSs*: MySpace, Facebook, and Orkut (see Figures 1-3) with first-time users. All three sites were ranked in the top 10 for SNSs by TopTenReviews (2009): MySpace (#1), Facebook (#2), and Orkut (#6). A comparison usability test was conducted with the three sites to evaluate first-time user satisfaction, navigational efficiency, and general preferences.

Figure 1. MySpace homepage.

Figure 1. MySpace homepage.

Figure 2. Facebook homepage.

Figure 2. Facebook homepage.

Figure 3. Orkut homepage.

Figure 3. Orkut homepage.

Method

Participants

Ten participants (8 females, 2 males) with an average age of 27.1 (SD = 11.5) participated in the study. All participants were first-time users of the three sites. Nine out of 10 users reported that they had never used a SNS.

Materials

A Pentium Core 2 Duo PC with a 96 dpi 17" monitor running at a resolution of 1024 by 768 pixels was used. Both the initial background questionnaire and the post-survey questionnaire were created using SPSS MrInterview, version 3.5. User satisfaction was measured using the System Usability Scale (SUS), which was adapted for web use (Digital Equipment Corporation Limited, 1986). MoraeTM 3.0 was used to record user onscreen activity and navigational efficiency measures such as time on task and number of clicks.

Procedure

Users completed a background questionnaire addressing web use and general demographics. Users then completed 10 tasks across all three sites. The order of presentation for the sites were counterbalanced, and tasks were randomized to prevent ordering effects. Users performed the same tasks across the three sites; however wording was altered to remain consistent with the site. For example, when using Facebook users had to post a "note", while Orkut users posted a "testimonial". Users rated the difficulty of each task on a scale of 1-5 (1=very easy, 5=very difficult). The SUS was completed after all tasks were completed on the site. At the end of the study, users ranked their preference across the three sites. The tasks included the following:

  • Add new information (favorite movies, dating status, etc.) about yourself to your new profile.
  • Upload a picture to your new profile.
  • Write on the wall of your friend, Chris Jones.
  • You have just received a message. Read and reply back to the message.
  • Change your privacy settings so that everyone can view your profile.
  • Change the notifications option for messages to include an alert for new messages.
  • Write a note on your views of the price of gasoline.
  • Make a photo album and upload three pictures to it.
  • Join a group for singles.
  • Find the chat link from your profile.

Results

Overall Success and Difficulty

Table 1 shows the percentage of participants that were successful on each task for all three social networking sites. Success rates did not differ across sites (F (2,18) = 1.36, p > .05). While most of the tasks were completed successfully, several tasks were found to be more difficult. Table 2 shows the difficulty ratings participants gave to each task. Across the three sites the most difficult tasks were uploading a profile picture and changing the notifications option for receiving messages. The easiest tasks were adding information to one’s profile and making a photo album.

Table 1. Percent of participants successful on the tasks for each website (bold indicates less successful tasks)

Task Facebook MySpace Orkut
Add information to profile. 90 90 88.9
Upload a profile picture. 90 20 33.3
Write on wall of friend. 100 100 66.7
Read & Reply to a message. 90 100 66.7
Change privacy settings for profile. 88.9 70 40
Change notifications option for messages. 60 30 60
Write a note/blog/scrap. 88.9 100 90
Make a photo album. 100 90 90
Join a group for singles. 77.8 50 90
Find the chat link. 10 70 90
Average Across Websites 79.56 72 71.56

Table 2. Average difficulty ratings (1=very easy, 5=very difficult) for tasks of each website (standard deviation in parentheses)

Task Facebook MySpace Orkut
Add information to profile. 1.60 (1.07) 2.00(.94) 1.56(.73)
Upload a profile picture. 2.10(1.10) 3.30(1.57) 4.11(1.36)
Write on wall of friend. 1.80(.79) 1.90(.88) 3.33(1.66)
Read & Reply to a message. 2.00 (1.15) 1.60(.97) 2.33(1.41)
Change privacy settings for profile. 3.22(1.30) 2.90(1.66) 2.60(1.51)
Change notifications option for messages. 3.10(1.52) 3.80(1.23) 3.00(1.49)
Write a note/blog/scrap. 1.78(.67) 2.20(1.03) 1.70(1.25)
Make a photo album. 1.60(.97) 2.40(1.58) 2.60(1.35)
Join a group for singles. 2.56(1.59) 3.20(1.48) 2.60(.97)
Find the chat link. 4.30(1.34) 2.80(1.69) 1.30(.67)

Navigation Efficiency (Speed and Number of Pages visited)

Participants took the least amount of time to complete the tasks on Facebook (M = 14.80 minutes, SD = 1.91), compared to Orkut (M = 14.99 minutes, SD = 4.41) and MySpace (M = 18.92 minutes, SD = 3.94). However, there was no statistically significant difference across the sites, F(2,12) = 1.05, p > .05.*

Interestingly, there was a significant difference in number of pages visited, F (2,12) = 12.24, p < .01*. A pairwise comparison showed significantly fewer pages were visited with Facebook (M = 26.86, SD = 4.88), when compared to Orkut (M = 47.86, SD = 11.87) and MySpace (M = 60.43, SD = 25.17). In addition, participants clicked, on average, 2.86 pages beyond the optimal path on the Facebook site, 17.86 pages beyond the optimal path on the Orkut site, and 27.43 pages beyond the optimal path on the MySpace site.

*Data cleaning and transformation were performed to ensure distributions were normal.

Satisfaction and Preference

Overall, Facebook was rated the highest in overall satisfaction (M = 61.7, SD = 15.3), followed by MySpace (M = 54.7, SD = 18.0), and Orkut (M = 50.6, SD = 29.0). However, there was no significant difference found across the sites. All scores are based on a 100-point scale.

When asked which site they preferred the most, there was no clear favorite among the three social network sites. Facebook was preferred by 40% of the participants, while Orkut and MySpace each were preferred by 30% of the particpants.

Usability Issues by Task

Below is a discussion of specific tasks that proved to be difficult for participants.

Changing the Notifications Option Task

Changing the notifications options so that an alert will be given when a new message is received was particularly difficult across all three sites, with MySpace being the worst (30% successful), followed by Facebook (60% successful) and Orkut (60% successful).

Participants initially found the correct link ("My Account") but often failed to notice the horizontal menu just under "Settings" on the MySpace site (see Figure 4). Most participants continued to scroll down the page or go back to the homepage.

On Facebook, finding the "Settings" option proved difficult as the link was in a small font and hidden on the top right hand corner of the screen.

Orkut had an unorthodox approach to changing settings (see Figure 5). Most users were initially unfamiliar with this matrix view and were unsure what each checkbox meant.

Figure 4. "My Account" section on MySpace. Users failed to notice the horizontal menu (not salient enough) just under "Settings".

Figure 4. "My Account" section on MySpace. Users failed to notice the horizontal menu (not salient enough) just under "Settings".

Figure 5. Notifications page on Orkut. Users were unfamiliar with this matrix view of "on/off option" and wree unsure what each checkbox meant.

Figure 5. Notifications page on Orkut. Users were unfamiliar with this matrix view of "on/off option" and were unsure what each checkbox meant.

Uploading a Profile Picture

Uploading a profile picture was reported to be difficult for two of the three sites, MySpace (20% successful) and Orkut (33.3% successful).

On the MySpace site, all users clicked on the "Upload" option next to "Photos" or simply clicked on the space for the profile image. The upload option leads users to the photo album section of the site, while clicking on the image itself sent the user to the homepage. Some users also tried "Edit Profile" but this only allowed users to change their profile information (excluding pictures).

The Orkut site was very particular about the image size that could be uploaded. If users tried to upload an image that was too large of a file, an error appeared at the top of the page. The color and font size of the message blended into the background and was difficult to notice. As a result, most users simply missed it and were left confused as to why their picture would not load.

Figure 6. On MySpace, most users clicked on the "Upload" option next to "Photos" or on the space for the profile image. Neither option allows the user to change their profile picture.

Figure 6. On MySpace, most users clicked on the "Upload" option next to "Photos" or on the space for the profile image. Neither option allowed the user to change their profile picture.

Figure 7. This error message on Orkut was not noticed by many users. As a result, they were unsure why their picture would not load on their profile page.

Figure 7. This error message on Orkut was not noticed by many users. As a result, they were unsure why their picture would not load on their profile page.

Chat Link location

Finding the chat link was very difficult on the Facebook site (10% successful). The chat icon was very small and hidden in the bottom right corner of the screen underneath the advertisement banners (Figure 8). Almost all users missed this icon when performing this task.

Figure 8. The Chat link icon was difficult to find for users of Facebook.

Figure 8. The Chat link icon was difficult to find for users of Facebook.

Joining a Group

Joining a group was a problem for MySpace users (50% successful). This was because the option for groups was located under the "More" horizontal menu option. Users were expecting it to be under the "Profile" option.

Changing the Privacy Settings

Changing the privacy settings was a difficult task for Orkut users (40% successful). This was partially due to the fact that users needed to use the drop-down menu to complete this task. Most users were expecting a radio button option for this feature based on their experience with other web-based applications which allowed privacy options (i.e., email).

Discussion

From a quantitative standpoint, Facebook seemed to be the best SNS. While most of these measures were not significant (number of pages being the exception), there seemed to be strong support for Facebook as the best SNS when considered collectively. First, users experienced the most success on Facebook (79.6% task success). In fact, only one task (finding a chat link) presented problems for users. With that task removed, almost all tasks were completed at a high success rate. It was also the most efficient site. It had significantly fewer number of pages visited and took less time to complete the tasks. The amount of variability for time on task was another indication of the higher efficiency for Facebook, the standard deviation was much smaller for time than MySpace or Orkut. In other words, Facebook users were consistently more efficient when completing tasks. Also, users gave it the highest overall satisfaction rating (61.7 out of 100) when compared to the other sites (MySpace = 54.7 and Orkut = 50.6).

Across the three sites users were able to successfully complete most tasks. However there were some tasks that were more difficult than others. The tasks in which users performed poorly were:

  • Changing the notifications option (all sites)
  • Uploading a profile picture (specific to MySpace and Orkut)
  • Finding a chat link (specific to Facebook)
  • Joining a group (specific to MySpace)

Several usability issues were found to contribute to these task failures:

Terminology

There were several inconsistencies with link terminology on all three sites. MySpace used the link name "My Account" which did not match the page heading "Settings". Each subsequent tab under "My Account" was also labeled "Settings". There was also confusion among participants as to the difference between "Edit Profile" and "Customize Profile". Customizing the profile should really be a sub-component of "Edit Profile" to eliminate confusion. Under "Photos", there is a feature named "Photo Cube" which several users clicked expecting the option to make a photo album. "Photo Cube" is unrelated to making albums (in fact, it is not even an online tool but one that allows users to print pictures instead).

Facebook used the term "Live Feed" on its homepage which most users confused as a chat link. The term "live" gave the impression that one could connect in real time to friends online. Moreover, the term "Boxes" on the profile page is rather cryptic and users were unsure what functionality this link controlled.

Orkut had several terms that were unfamiliar. The terms "Scrapbook", "Testimonials", and "Communities" were unconventional to the users in this study. This may be explained by the fact that Orkut is more popular in Brazil and India than it is in the United States; therefore, Orkut may use terminology more appropriate for that population. Many users commented that the reason for their task failure was due to not understanding the terms used.

Color and Font Use

All three websites employed a dark blue horizontal menu bar at the top of the screen, yet participants only complained about the Facebook layout. Several participants reported that the "Settings" link was "hard to see/find" when they scanned the menu. This issue may be a combination of two things. First, Facebook used a small font size for their menu options. Second, the contrast between the text foreground and the background color was worse on Facebook when compared to the other two sites.

Feedback and Error Messages

Uploading a profile picture proved difficult in part because users did not receive adequate feedback from the website. MySpace did not provide any feedback to users when they were trying to upload a picture as their profile image. MySpace also provided a poor error message to users when they forgot to give their photo album a name and did not draw their attention to the mandatory fields.

Orkut, as stated previously, provided an error message at the top of the page that was not salient. The same problem occurred when users forgot to provide their photo albums with a title. A more prominent error message is needed from Orkut to draw the user’s attention to the required fields.

Login vs. Sign Up

One problem that plagued most participants was initially logging in to their Facebook account. When users were first logging into their account (provided dummy account), most would enter their information in the "sign up" section and not the "login" section of the page. This can be contributed to the design of the page. Due to the prime location of the "sign up" section and the bright green color of the "sign up" button, users’ attention was first drawn to this area of the page. The actual "login" area was not designed to draw attention. The "login" button blends into the background color and it is located at the top corner of the page away from the main content of the page (see Figure 9). This may be a marketing tool by Facebook. Acquiring new users seems to be the major priority of this site, especially since there has been an increase in commercial revenue from sponsors in the site.

Figure 9. Nearly all Facebook users entered their sign-in information in the Sign Up fields instead of the Sign-in fields in the top right of the page.

Figure 9. Nearly all Facebook users entered their sign in information in the sign up fields instead of the login fields in the top right of the page.

Conclusion

It should be noted that only first-time users participated in this study. Therefore, some of the issues that were found may not generalize to more experienced SNS users. However, it is important to assess first time usage because negative first impressions may result in users abandoning the SNS site. About 18% of SNS users recently surveyed revealed that lack of usability was a significant reason for not returning to use the site (Brandtzaeg and Heim, 2008). With this in mind, it is important that SNSs follow usability guidelines so that they retain users.

The following is a list of recommendations on how to improve the overall use of social network sites:

  1. Use consistent and familiar terminology.
  2. Provide a brief explanation for terms that are unique to the site (e.g. PhotoCube on MySpace, Testimonials on Orkut, Boxes on Facebook).
  3. Provide sufficient feedback to the users. Too often the users repeated failed actions simply because they were not sure if the system had performed their initial task.
  4. Improve link placement. Uploading a profile picture, finding the chat link and looking for the Settings option should be easy tasks to perform and should be placed within easy view of the user on the profile homepage.

*It should be noted that this study was conducted in February 2009. Modifications to the SNSs may have occurred since then.

References

Boyd, D. M., & Ellison, N. B. (2007). Social network sites: Definition, history, and scholarship. Journal of Computer-Mediated Communication, 13(1), Retrieved February 8, 2009, from http://jcmc.indiana.edu/vol13/issue1/boyd.ellison.html.

Brandtzaeg, P. B. & Heim, J. (2008). User Loyalty and Online Communities: Why members of online communities are not faithful. In Proceedings of Second International Conference on Intelligent Technologies for Interactive Entertainment, January 8-10 (2008), Cancun, Mexico.

Digital Equipment Corporation Limited (1986). System Usability Scale. Reading, United Kingdom: Digital Equipment Co. Ltd.

Facebook (2009). Press Room. Retrieved March 24, 2009, from http://www.facebook.com/press/info.php?statistics.

Hart, J., Ridly, C., Taher, F, Sas, C., & Dix, A. (2008). Exploring the Facebook experience: A new approach to usability. In Proceedings of NordiCHI Conference, October 22-28 (2008), Lund, Sweden, 471-474.

Mislove, A, Marcon, M, Gummadi, K. P., Druschel, P., & Bhattacharjee, B. Measurement and analysis of online social networks. In Proceedings of the 7th ACM SIGCOMM conference on Internet Measurement, October 24-26 (2007), San Diego, CA, 29-42.

Owyang, J. (2008, January 9). Social network stats: Facebook, Myspace, reunion. Retrieved February 8, 2009, from http://www.web-strategist.com/blog/2008/01/09/social-network-stats-facebook-myspace-reunion-jan-2008/.

Top Ten Reviews. (2009). 2009 Social Networking Websites Product Comparisons. Retrieved February, 8, 2009, from http://social-networking-websites-review.toptenreviews.com/.

Tagged with: , , , , , , , , , , , ,
Posted in Usability News
Subscribe to SURL

Want to receive notifications when SURL has new articles? Please enter your name and email address to subscribe to our website.

Popular Topics in Usability News
Log in/Sign Up
%d bloggers like this: