Trick or Tweet: How Usable is Twitter for First-Time Users?


Justin W. Owens, Kelsi Lenz, and Stephanie Speagle

Summary. Social networking websites have become common tools people use for communication and sharing media such as photos. Currently, one of the most popular social networking websites is Twitter. A usability study was conducted with first-time users to determine ease of use and satisfaction with the site. The evaluation revealed several issues involving signing up for a new account, site terminology, and basic communication.

Introduction

As one of the leading forms of communication, social networking sites (SNSs) are maintaining user popularity worldwide. Boyd and Ellison (2007) define SNSs as web-based services that provide an opportunity for an individual to build and display either public or private profiles. Within a given site, these profiles can be shared amongst users, connecting or networking users based on common interests. SNS users utilize sites such as Facebook, MySpace, and Twitter to communicate information regarding status, mood, activities, location, and other details of their lives (Fox and Lenhart, 2009).

Developed and implemented in 2006, Twitter is one of the newest SNSs (http://twitter.com/about). It limits some of the functions of a typical SNS to include only basic features such as text updates. From February 2008 to February 2009, Twitter experienced over 1200% growth and grew six times faster than Facebook for the same period. As of February 2009, Twitter had over 7 million users.

Public status updates allow users to share messages containing this information with other users in their social network. SNSs like Facebook and MySpace maintain the typical components of a SNS, but allow users more freedom to install various applications to personalize and enhance their profiles with photos, games, etc. Neither Facebook nor MySpace limit status updates nor private messages to a certain numbers of characters. Twitter can be considered a form of microblogging, which has been described as blogging through text updates that are typically limited to 200 characters or less (Finin, Java, Song, & Tseng, 2007). Twitter’s limit of 140 characters per update or private message creates a slightly different user experience.

These updates are created via SMS text messaging, e-mails, instant messages, the web, or other third party applications. Twitter limits all user messages, status updates, replies to status updates, and private messages to 140 characters (Fox and Lenhart, 2009). Given the limited number of characters, many Twitter users have adopted the use of word abbreviations (shorthand) and symbols to make communicating more effective.

The main draw to Twitter seems to be based on the idea of simplicity and ease of use. As noted by Brandtzaeg and Heim (2008), usability issues rank as the third highest reason for abandoning a SNS site. An example of a common usability issue on a SNS is difficulty learning the different features that are available. The importance of uncovering such challenges is to discover the site improvements they inspire that then lead to greater satisfaction among users.

Purpose

The purpose of this study was to evaluate the usability of Twitter.com (Figure 1) with first-time users.

Figure 1 Twitter.com main sign-in screen.

Figure 1. Twitter.com main sign-in screen.

Method

Participants

Thirteen participants (3 male, 10 female) ranging from 18 to 42 years of age participated in this study. Ten of the thirteen participants reported using social networking websites, other than Twitter. Such sites included Facebook and MySpace. Participants were asked whether they used Twitter.com in the past, and none of the participants had used the service before this study. All participants reported having experience on the Internet.

Materials

A single Pentium Core 2 Duo computer running Windows XP Professional at 1024 x 768 resolution on a 17-inch LCD monitor was used to run the study. Both the initial background questionnaire and the post-survey questionnaire were created using SPSS MrInterview, version 3.5. User satisfaction was measured using the System Usability Scale (SUS), which was adapted for web use (Brooke, 1996). Participants accessed Twitter.com* using Mozilla FireFox, version 3.0.8. Participant’s comments and reactions were recorded using a Web camera and MoraeTM 3.0 software. The software captures and combines both the video footage and the on-screen events such as time on task and number of clicks for each task.

A new Twitter account named "joeasmith” was established for the purposes of the study. The account followed or subscribed to the updates of some rather generic and non-offensive Twitter users, of which roughly half followed or subscribed to in return.

Procedure

Users were briefed on the overall purpose of the study and then completed a series of eight tasks. After each task, users were asked to rate its difficulty on a scale of 1-5 (1=very easy, 5=very difficult). Once testing on all tasks was complete, participants completed the background questionnaire and satisfaction survey. Tasks were presented in a fixed order since some were prerequisites to others. The tasks included the following:

  • Create a new Twitter account.
  • Change your name and location on your new Twitter account.
  • Post a tweet (message) on Twitter.
  • Find joeasmith on Twitter.
  • Follow joeasmith with your Twitter account.
  • Send joeasmith a tweet.
  • Respond to a tweet by joeasmith.
  • Delete your Twitter account.

Results

Overall Success and Difficulty

User satisfaction with the website was measured using the (SUS), and the scores are summarized in Table 1. In general, satisfaction ratings for the Twitter.com website were below average (38.46 out of 100). However, variance in the satisfaction scores was high across participants (SD = 24.01). This was primarily due to the large range of satisfaction scores (from 0-80). Participants noted that they found Twitter to be complex and felt they would need to learn quite a bit about it before using it. This reflected their troubles with figuring out how the "What are you doing?” field worked. Moreover, participants demonstrated they had little confidence using Twitter. That said, participants thought they would learn the system quickly, but said they would not want to use Twitter frequently.

Two tasks, in particular, were rated as highly difficult and had low success rates (See Table 2). The tasks were sending joeasmith a tweet (difficulty rating = 4.38; success = 15.4%) and responding to a tweet by joeasmith (difficulty rating = 3.69; success = 38.5%). On both tasks, users had difficulty determining where the option was to send or reply to tweets. This was very frustrating to some users and commonly mentioned as one of the main reasons they would not want to continue using Twitter.

Table 1. Summary of System Usability Scale

Positively-scored items (1 – 5; the higher the number the more satisfied) Average Score (Std Dev)
I think that I would like to use Twitter frequently. 2.15 (1.28)
I thought Twitter was easy to use. 2.46 (1.39)
I found the various functions of Twitter were well integrated. 2.31 (1.03)
I would imagine that most people would learn to use Twitter very quickly. 2.54 (1.45)
I felt very confident using Twitter. 2.00 (1.22)
Reverse-scored items (1 – 5; the lower the number the more satisfied)  
I found Twitter unnecessarily complex. 3.38 (1.12)
I think that I would need the support of a technical person to be able to use Twitter. 2.38 (1.50)
I thought there was too much inconsistency in Twitter. 3.38 (1.04)
I found Twitter very cumbersome to use. 3.38 (1.04)
I needed to learn a lot of things before I could get going with Twitter. 3.54 (1.13)
Average Overall Satisfaction Score (Ranges from 0-100) 38.46 (24.01)

Table 2. Task difficulty ratings and task success. Most difficult/least successful in bold.

Task Average Difficulty

1=Very Easy 5=Very Difficult
Success Rate (%)
Create a new Twitter Account. 1.92 100.0%
Change your name and location on your new Twitter account. 2.00 100.0%
Post a tweet (message) on Twitter. 1.54 100.0%
Find joeasmith on Twitter. 1.23 92.3%
Follow joeasmith with your Twitter account. 1.31 92.3%
Send joeasmith a tweet. 4.38 15.4%
Respond to a tweet by joeasmith. 3.69 38.5%
Delete your Twitter account. 1.92 84.6%

Usability Issues

The following is a breakdown of the usability issues found on Twitter.com. Each issue contributed to users’ perceived level of difficulty and satisfaction with the site.

1. Site Terminology and Codes

Site Codes and Terminology

Figure 2. Twitter.com main user page with areas of confusing language highlighted in red.

Most participants had difficulty learning the "language” that is unique to Twitter. Many participants found the "followers" and "following" links confusing. It was not apparent that "following" refers to the people that an individual wishes to receive updates and messages from, while "followers" are other Twitter users that wish to receive updates and messages from one’s self. In some cases, there was not a distinct difference between multiple terms. For instance, the difference between "Profile" and "Settings" was not always clear. Many users were unsure of whether to choose "Profile" or "Settings" to edit their information. The difference between "Direct Messages" and "What are you doing?" was also unclear. Some participants initially thought "Direct Messages" would provide the functionality to post a tweet. In reality, the functionality is located on the main screen for Twitter. Participants were unsure whether posting a message or tweet was the same as posting "What are you doing?". They commented that they were guessing or making assumptions that using the "What are you doing?" textbox was correct.
Figure 2 highlights several links and labels with confusing language.

Moreover, status updates and messages are commonly referred to as tweets. Commonly found during messaging are shortcuts, which include RT (retweeting) for reposting a message from another user. Confusing site codes include the use of the "@" symbol to indicate usernames in tweets, DM (direct messaging) for sending private messages, and the use of hashtags or octothorpes (#) to indicate topic tags for messages.

2. Similarities with Facebook

Similarities with Facebook

Figure 3. Facebook.com "What’s on your mind?" text field.

Similarities with Facebook

Figure 4. Twitter.com "What are you doing?" text field.

Facebook is currently one of the most popular social networking website with 250 million users (Facebook, 2009). Participants made several comments about wishing the Twitter interface was more like Facebook, explaining that they expected it to behave similarly. Some comments referenced the Facebook status area, stating that the Twitter interface should have buttons like the Facebook interface. Interestingly, there are several similarities between the verbiage of the update textbox/button and the style of the textbox/button between Twitter and Facebook (See Figures 3 and 4). On appearance, the function is similar between both websites in that they post messages. However, Twitter has hidden functionality that allows replying to tweets from other users and sending public and private messages to other users. This was reported to be confusing by some users in this study.

3. Deciphering Captchas

Deciphering Captchas

Figure 5. Example of Captcha located on the new account screen on Twitter.com with the Captcha text enclosed in the red box.

In order to reduce spam accounts, Twitter incorporates the use of Captchas. Captchas are used to insure a human (and not a computer) is completing the form. Many users found the incorporated Captcha difficult to decipher and in some instances, they required multiple attempts to complete. Participants questioned whether it needed to be capitalized or have spaces between words and made comments regarding its poor legibility.
Figure 5 illustrates the type of Captcha used on Twitter.com.

4. Sending and replying to messages.

Sending and replying to messages.

Figure 6. Twitter.com interface highlighting the "What are you doing?" label.

Both tasks that involved sending a tweet to joeasmith and responding to a tweet by joeasmith proved to be difficult to users. Users update their status, send public and private messages to other users, and respond to other’s messages through the same interface (See Figure 6). The lone exception is a dedicated Direct Message interface, which resembles the general, multi-purpose interface. For example, when posting a message, a user can input text in the "What are you doing?" textbox and click the "update" button. However, there is no visual change to the textbox, button, or label to indicate the intended use of the box to distinguish it from other functions.

If sending a message to a user, a public message is preceded by "@username", where "@username" is the intended recipient of the message. A public message also can be sent to a user by pressing the reply icon on another user’s tweet. This is the same functionality to reply to a tweet, which was the goal of the task to respond to a tweet by joeasmith. In both cases, the only update on the web page is the "What are you doing?" label changing to "Reply to username", where the username is the intended recipient. There is no distinction between replying to a tweet and sending a public message to another user. Moreover, there is no distinction as to which tweet is the target of the reply. None of the participants identified the reply icon located on their "following" list, which is a link that may be used to send a public message to another Twitter user.

Private messages require the intended recipient to be a follower of the person sending the private message. Private messages are preceded by "DM @username" if using the "What are you doing?" text field on the Twitter home page. A private message also can be sent to another user via the "Direct Messages" utility. If using the "What are you doing?" text field, the only visual update is the label changes to "Direct Message". While some users did notice this change, they were still unclear as to what the text field would do.

Due to the lack of a salient visual change, users did not know the current state of the form. This appeared to be a contributing factor to the very low success rate and high difficulty ratings for both tasks. Another contributing factor was the lack of a "Reply” icon when a user wished to reply to a tweet that was not in a mouseover state. Given this, it was not apparent to the users that tweets could be replied to in this manner. Even participants who were successful in sending joeasmith a tweet were unsure if they were successful due to the fact the tweet appeared on their homepage, and not joeasmiths’. Several said they felt stupid after seeing this response, feeling they had incorrectly completed the task. Most participants expected it to function more similarly to Facebook, where their message to him would appear on his page, not theirs.

Discussion

The results of the usability study show that first-time users have difficulties with certain tasks on Twitter. On the sign-up page, the Captcha, a type of Turing test used to prevent spam accounts, was described as illegible several times. Users had a difficult time using Twitter to communicate. Their most successful tasks were creating a new Twitter account, changing their name and location, and posting a simple update. These tasks were 100% successful, and rated fairly low on difficulty. The other two tasks related to sending messages and replying to other Twitter users had very poor success rates, 15.4% and 38.5%, respectively and high difficulty ratings. This difficulty had a few potential sources. The form used for posting updates also is used to send messages to other users and to reply to other users’ messages. The form lacked salient differences between modes and little information about the state of the form. When replying to another message, the reply icon was hidden unless the message of interest was moused-over. Twitter.com rated below average on the SUS overall. Participants found the service to be complex and felt they would need to learn quite a bit before using it. Moreover, participants reported that they would not use the service often.

Conclusion

The goal of this usability test was to assess user-friendliness, user satisfaction, and perceptions of Twitter by first-time users. It is anticipated that through subsequent and recurring use of Twitter, user performance would improve over the results found within this study. It should be noted though, that first impressions are vital to whether users will use the website in the future. Issues found within this usability test may discourage novice users from returning to Twitter and using its services.

The following is a list of recommendations on how to improve the usability of Twitter for first-time users:

  • Make the site language more intuitive. For instance, users were often confused with the meaning of the profile, settings, @username, username links, followers, following, and tweets.
  • Have more distinct visual changes between messaging modes. Currently, the only difference between posting an update, posting a reply, and sending a public message to another user is a label change on the web form.
  • Provide additional hints to first-time users. There are features and terms in Twitter that do not make their function or purpose clear. Hints should be applied to the messaging form to assist users in determining the current mode of communication. They should also be applied to usage of @username terminology and links such as Profile, Settings, etc.
  • Replace or use simpler Captchas to ward off spammers. Many participants had difficulty with the Captcha on the sign-up form. It could be improved upon by using one word instead of two, eliminating potentially illegible words, or switching to another type of Captcha.

References

Boyd, D. M., & Ellison, N. B. (2007). Social network sites: Definition, history, and scholarship. Journal of Computer-Mediated Communication, 13(1). Retrieved August 9, 2009, from http://jcmc.indiana.edu/vol13/issue1/boyd.ellison.html.

Brandtzaeg, P. B. & Heim, J. (2008). User Loyalty and Online Communities: Why members of online communities are not faithful. In Proceedings of Second International Conference on Intelligent Technologies for Interactive Entertainment, January 8-10 (2008), Cancun, Mexico.

Brooke, J. (1996). SUS: A Quick and Dirty Usability Scale, in P. Jordan, B. Thomas, B. Weerdmeester, & I. L. McClelland (eds.), Usability evaluation in industry, (pp. 189-94). London, UK: Taylor & Francis.

Facebook Statistics. (2009). Retrieved September15, 2009, from Facebook.com:
http://www.facebook.com/press/info.php?statistics.

Finin, T., Java, A., Song, X., & Tseng, B. (2007). Why We Twitter: Understanding microblogging, usage, and communities. Joint 9th WEBKDD and 1st SNA-KDD Workshop ’07 , August 12 (2007 ),San Jose, California , USA .

Lenhart, A., & Fox, S. (2009, February 12). Twitter and Status Updating. Retrieved August 8, 2009, from http://www.pewinternet.org/Reports/2009/Twitter-and-status-updating.aspx.

Tagged with: , , , , , , , , ,
Posted in Usability News
Subscribe to SURL

Want to receive notifications when SURL has new articles? Please enter your name and email address to subscribe to our website.

Popular Topics in Usability News
Log in/Sign Up
%d bloggers like this: