Using Repertory Grid Interviews to Capture First Impressions of Homepages


Veronica Hinkle

Summary. This article explains the Repertory Grid Interview (RGI) method and provides an example of how to use it to capture users’ first impressions of website homepages. Six website homepages selling high-end food products were used for this study. RGI results helped define the features of homepages that generate first impressions, both positive and negative first impressions. In our example with e-commerce sites, features related to positive first impressions of these homepages included the aesthetic appeal of the homepage, availability of promotional products, and ease of online browsing and shopping. Homepages that scored high on these features typically were perceived more positively than those with low scores.

Introduction

The Repertory Grid Interview (RGI) is an interview-based method originally developed by George Kelly (1955) to explore personal constructs about interpersonal relationships in a clinical context, i.e., (psychotherapy). The technique is based on Personal Construct Theory (PCT), which assumes that people make mental representations of the world and use these representations to make and test hypotheses about reality and the environment. The underlying idea is that people represent their experiences by placing alternative constructs upon them (Kelly, 1955). That is, people interpret reality and their environments based on how they perceived the world around them.

RGIs provide a wealth of information, all of which is generated from the interviewee. More importantly, this information can be qualitatively or quantitatively analyzed. Training time is minimal, unless the interviewer needs to learn probing and laddering techniques. Extensive time is required to analyze the results and the researcher should be familiar with appropriate quantitative, e.g. (clustering methods), and qualitative procedures, e.g. (content analysis), to get the most value out of the data.

RGIs can be used when the ultimate goal of a study is to discover people’s perceptions of products, concepts, or systems or establish comparisons between these stimuli. Constructs generated during RGIs can be used to develop scales of attitude dimensions for a product, concept, or system (Brinkman & Love, 2006).

Based on the existing research literature, it is safe to say that RGI has not been widely used as a knowledge elicitation technique in user interface design. However, its use is starting to gain in popularity in recent years (Alexander & Van Loggerenber, 2005). The following are examples for the type of user perceptions that can be gathered with RGIs:

  • products, systems, and devices (Stanton, Salmon, Walker, Baber, & Jenkins, 2005)
  • factors that influence user acceptance of interface design (Muller et al., 2008)
  • social norms that influence user acceptance of interface design (Brinkman & Love, 2006)
  • websites (Hassenzahl & Trautmann, 2001; Verlinden & Coenders, 2000)
  • virtual environments (Steed & McDonnell, 2003)
  • design perspectives from design professionals (Tan & Tung, 2003,
  • text types (Dillon & McKnight, 1990)
  • team members in a design team (Young, Edwards, McDonald, & Thompson, 2005)

Important Features of RGIs

The RGI method requires a minimum of six stimuli and can be divided into two phases: construct elicitation phase and ratings phase. During the construct elicitation phase, participants compare three stimuli at a time, also known as triads. They are told to find two stimuli that share something in common (construct), and then explain why the third is different (contrast).

For example, they could be shown three banking websites (Figure 1). They need to find something two of these three websites have in common and name it. Once the similarity is established, participants are asked to describe what the remaining website did not have in common with the other two. The participant may say that Wells Fargo and Bank of America have pictures of people at the top of their page and that Citibank differs from the other two in that it has a picture of a laptop. In this case, the construct is “picture of people” and the contrast is “picture of computers.” After comparing every possible triad combination of the websites (which is why there must be a minimum of six stimuli) the construction elicitation phase is completed.

Websites presented during a triad.

Figure 1. Websites presented during a triad.

At this point the rating phase begins by asking the participants to rate all the websites on each construct. After rating all the stimuli based on the constructs elicited during the interview, the participant is thanked for their participation and the interview is over. Figure 2 shows a grid from one of the participants in our study.

A grid of a participant’s constructs, contrasts and ratings.

Figure 2. A grid of a participant’s constructs, contrasts and ratings.

Triading

Triading is used to elicit constructs. Triads consist of showing three stimuli at a time. For the construct elicitation phase, every possible combination of at least six stimuli is generated. Dillon and McKnight (1990) recommend following a predefined sequence so no combination of elements are repeated. Although the assumption is that participants will generate one construct per triad, sometimes participants do not produce any constructs or may come up with more than one construct. If participants fail to produce a construct, they should be given the opportunity to continue on to the next triad.

Laddering

Laddering is a way to gather in-depth information about the constructs and contrasts generated during the interview (Tan and Tung, 2003). Young et. al. (2005) use laddering either to probe for more information regarding a construct and how it affects a stimuli or to confirm what the construct means to the participant. Laddering leads to more specific answers or clarification of something the participant said.

Semantic Differentials

RGI was originally developed to use strict Boolean techniques with semantic differentials (Good vs. Bad, Expensive vs. Cheap), but now it has extended by using rating scales where each pole represents either a construct or its contrast. The latter is recommended because it allows greater distance between the poles, thus increasing granularity for the scale. Upon completion of the construct elicitation phase, participants rate all elements based on the constructs/contrasts they created. Young et. al. (2003) explain that it is better to ask participants to generate a contrast to the construct rather than simply use a literal opposite because it provides more information about how they perceive the construct in regards to the stimulus. They also make a good point about the advantage of RGI’s bipolar scales and explain that “a general interview requires the interviewee to give a single in-depth answer to a question, whereas the bipolarity of repertory grid gives a rating on all the questions against all the answers to the questions” (Young et. al., 2005, p. 3). Figure 3 demonstrates a typical repertory grid scale.

 



Figure 3. Typical repertory grid scale.

Elements/Constructs

The basic output is a grid, in the form of n (constructs in rows) and m (stimuli in columns), to organize each participant’s ratings using a 5 or 7 point scale. Choosing a scale depends on the desired level of granularity for the ratings. The grid contains stimuli, which can be specific concrete examples or a set of observations represented as columns, and constructs represented as rows. The constructs are typically based on similarities when they compare two stimuli from a triad. The opposite or contrast to the construct is then elicited based on how participants perceive the third element to be different from the other two in the triad.

Purpose of this Study

To explore the potential of using RGI methods, a pilot study was conducted to determine user perceptions of homepages of websites for high-end food products. The goal of the study was to determine the features of a homepage that generate first impressions.

Methods

Participants

Eight participants, 6 females and 2 males, (M age = 31.13 years) were recruited from Wichita State University to participate in the study. All participants were graduate students in psychology. In terms of Internet experience, all participants visited websites on a daily basis and spent at least 10 to 20 hours a week using the internet. Seven participants reported liking to shop online, while one reported that sometimes they looked but not actually buy anything. The web sites selected for this study were all commercial sites that sell high-end products for cooking and specialty food items. They were selected because they are the main competitors in their industry. To ensure there was a level of interest and involvement in the domain, participants were asked if they liked to cook. Seven participants declared that they enjoyed cooking, and 1 participant said she was neutral about cooking.

Materials

A Pentium IV-based PC computer with a 19-inch monitor was used. The stimuli selected were 6 websites (see list) that sold high-end consumer products and were considered main competitors in the domain according to a Google search.

We also used a rating scale of 1-5, where 1 represented the construct and 5 represented the contrast of the construct, but not its literal opposite. A grid form template created in MS Excel was used to record constructs and contrasts generated during triads and ratings for all elements. Participants were provided with index cards displaying the rating scale in which they were to write their constructs and contrasts generated during the triads. The experimenter used the grid form to record ratings for all elements. The grid contained six columns which represented the websites, while two additional columns were added into the grid for writing the construct and its contrast after the interview was completed to facilitate data analysis (See Figure 2 for an example).

Procedure

The experimenter recorded all constructs and contrasts generated by all participants. Participants were allowed 2-3 minutes to look at the homepages of all the different websites before the interview began. They were instructed to stay within the homepage and not follow any links, but were allowed to scroll vertically and horizontally.

At the start of the interview, the experimenter briefed the participant on what they were about to do and provided an example of how to compare two websites to generate a construct and use the third website to generate a contrast. The participant was guided through an example of what they were expected to do during a triad, and explained that at the end of triading they would rate all the websites based on the constructs and contrasts they generated. The triading sequence was determined by considering all the potential combinations of the six websites.

Participants were also told that it was acceptable if they could not think of a new construct during a triad. In such cases, the experimenter moved on to the next triad. In the case of repeated constructs, laddering was used to clarify the meaning of the constructs and determine whether it was the same as a previously elicited construct. At the end of the interview, participants rated the six websites based on each of the constructs and contrasts elicited during their interviews.

Results

Grid results were interpreted to develop a classification theme for the constructs using a grounded theory approach as explained by B. Dick (2005). The classification theme was modeled after Tan and Tung’s (2003) classification theme. Interview time ranged from 30 minutes to 60 minutes, with an average of 40 minutes. The interviews yielded a total of 112 constructs and contrasts. Constructs elicited during interviews ranged from 10 to 19, with an average of 14 constructs per interview. Constructs elicited during the interview seemed consistent across interviews and facilitated using this type of analysis on the results. Contrasts tended to be conceptual opposites of the constructs, with a few being actual literal opposites. Figure 4 shows the constructs generated across all participants.

Constructs generated across all participants.

Figure 4. Constructs generated across all participants.

To create the classification theme, classes were created for all the constructs and contrasts generated during the RGI. After designating classes for all of the constructs and contrasts, metacategories were assigned to the classes. There were five predominant metacategories for the constructs and contrasts elicited during the interviews.

These metacategories can be used to define features of a homepage that generate positive and negative first impressions. Table 1 shows the metacategories, the construct classes, and examples of constructs for each class.

Table 1. Classification Theme of Interview Results

Metacategories Construct Classes Construct Examples
Aesthetics Colors, pictures, font styles, page layout, seasonality colors and font professionally done and pleasing, seasonal themes are predominant, colors blend better and are softer, pictures interest you in products
Functionality Shopping carts, navigation menus, seasonal promotions and ads, gifts and registries shopping carts, top and left navigations, offers recipes in prominent place, clickable inside ads, pictures/text convey information
Information Quality Easy to read, pictures and text match, easy to find, conveys an aesthetic message more information about the site, clever page layout, clean impression (good white space)
Products quality of products, gift potential of products, personal shopping, type of products high end products, options about products, food products, indoor/outdoor living, accessories, decorate home
Company Image* perceived qualities of the company, web site reflection of the company, online shopping image high end, sophisticated , online shopping oriented

* Company image was reported to be the overall perception generated for the website.

The metacategories from the classification theme were used as general features for the homepages. The ratings for all the constructs within each metacategory were used to determine whether these features were present for each of the websites’ homepages. First, each rating was converted to a score that depicted how much the rating differed from neutral. Then, the mean of the difference scores for each metacategory was calculated. Table 2 shows the values used during the conversion process. Table 3 shows the means and standard deviations for each of the metacategory scores. Table 4 shows the total score for each website’s homepage.

Table 2. Rating conversion process

Construct/Contrast scale Score Conversion
1 -2
2 -1
3 0
4 1
5 2

Table 3. Means (SD) for each of the metacategory scores.

  Aesthetics Functionality Information quality Products Company Image
Dean & DeLuca -2.50 (5.58) 0.20 (1.92) -1.0 (2.24) 3.33 (6.53) -0.50 (1.22)
Williams-Sonoma 3.83 (5.04) 2.80 (1.92) 3.20 (2.77) 0.83 (6.55) 0.33 (2.25)
Balducci’s 2.83 (5.00) 0.20 (3.11) 0.20 (4.15) 0.17 (6.79) 1.17 (0.41)
Crate & Barrel 3.17 (4.01) 2.60 (3.06) 2.0 (4.26) -3.67 (4.67) -0.83 (0.98)
Zabar’s -1.67 (5.13) 0.40 (1.79) 0.60 (4.02) 1.50 (5.91) -1.50 (0.28)
Bloomingdale’s 2.83 (3.58) 1.40 (2.49) 0.80 (3.94) -5.83 (4.94) -2.50 (1.27)

Table 4. Total score for each website’s homepage.

Participant Dean & DeLuca Williams-Sonoma Balducci’s Crate & Barrel Zabar’s Bloomingdale’s
p1 0.08 4.2 1.8 1.2 -0.6 -2.6
p2 -0.25 3.6 -1.6 1.2 1.6 -0.4
p3 2.75 1.0 1.2 1.4 1.0 -0.2
p4 -4.00 8.5 4.5 8.5 -5.5 9.0
p5 -4.50 2.5 7.0 3.0 0.05 3.0
p6 3.0 -5.0 -5.0 -5.0 -2.0 -3.5
p7 3.0 -5.0 -5.0 -5.0 -2.0 -3.5
p8 2.25 2.0 0.4 0.8 -0.8 -2.4
Total Score 2.33 11.8 3.3 6.1 -8.25 -0.6

A subjective ranking of preference was established based on the total scores for each website’s homepage. Interview notes also corroborate the subjective ranking of preference established by the total scores. Based on their metacategory scores and total scores for the website’s homepages, it seems that participants preferred the websites in the following order:

  1. Williams-Sonoma
  2. Crate and Barrel
  3. Balducci’s
  4. Dean and DeLuca
  5. Bloomingdale’s
  6. Zabar’s

Discussion

Interview results show that people take into account many aspects of a website to form an opinion about the quality of the products they offer or the company’s image. Surprisingly, brand name recognition did not influence how participants perceived the websites. What seemed to matter most was the aesthetic appeal of the homepage, availability of promotional products, and the perceived ease of online browsing and shopping.

User perceptions of the homepages were influenced greatly by the pictures and the layout of information. For example, many participants commented feeling “hungry” or wanting to “buy figs” (one of the sites had a very eye-catching picture of fresh figs) or “eat a bagel” (one of the sites had a beautiful picture of bagels) because the pictures were attractive to them. When they felt positive about a website, they wanted to browse through the site after the study. Clearly, aesthetics and layout were two big components of positive first impressions of homepages.

To gather more anecdotal data regarding how the constructs were generated, it was vital to use laddering to investigate how participants created their constructs. Participants explained to the interviewer why they felt a certain way about a construct or contrast, but in some cases probing was necessary to determine whether a construct was being duplicated.

Although websites used in this study can be ranked in order of appeal based on metacategory scores and total scores for the website’s homepages, in the future it would be best to ask participants to rank the websites in order of preference and compare this rating to the total scores. This additional information would help corroborate the ranking established by the metacategory scores and total scores fo website’s homepages for each of the sites.

Conclusion

RGIs can be used to gather users’ perceptions of website homepages and use these perceptions to determine important features that affect first impressions of homepages. The results from this pilot study demonstrate the importance of aesthetic appeal and usability of homepages in terms of layout and the quality of the information. High scores in these features could lead to further exploration of the web site and therefore a new customer. Low scores in these features may indicate low generation of appeal and therefore fail to incite further exploration of the website. RGI results can help determine important elements for specific domains based on the users’ reactions to the websites. Although there are other ways to capture first impressions of websites, RGIs provide in-depth qualitative and quantitative data that show the important features for potential users of the site. This in-depth information about important features allows researchers to create benchmarking goals for the first impressions of a homepage.

References

Alexander, P. M., & Van Loggerenber, J. J. (2005). The repertory grid: discovering a 50-year-old research technique. Proceedings of SAICSIT 2005, 192 – 199.

Brinkman, W. & Love, S. (2006). Developing an Instrument to Assess the Impact of Attitude and Social Norms on User Selection of an Interface Design: A Repertory Grid Approach. Proceedings of the 13th European conference on Cognitive ergonomics: trust and control in complex socio-technical systems. ACM International Conference Proceeding Series; Vol. 250, 129-136.

Dillon, A. & McKnight, C. (1990). Toward A Classification of Text Types: A Repertory Grid Approach International Journal of Man-Machine Studies. 33, 623-636.

Dick, B. (2005). Grounded theory: a thumbnail sketch. [On line] Available at http://www.scu.edu.au/schools/gcm/ar/arp/grounded.html.

Kelly, G. A. (1955). The psychology of personal constructs. Vol. 1 and 2. New York, NY: Norton.

Muller, J.; Jentsch, M.; Kray, C.; & Kruger, A. (2008). Exploring Factors that Influence the Combined Use of Mobile Devices and Public Displays for Pedestrian Navigation. Proceedings of NordiCHI, 308-317.

Stanton, N., Salmon, P., Walker, G., Baber, C. & Jenkins, D. (2005). Human Factors Methods: A practical guide for engineering and design. Burlington, VT: Ashgate.

Steed, A. & McDonnell, J. (2003). Experiences with repertory grid analysis for investigating effectiveness of virtual environments. Proceedings of the 6th International Workshop on Presence, Denmark.

Hassenzahl, M. & Trautmann, T. (2001). Analysis of web sites with the Repertory Grid Technique. Proceedings from CHI, USA, 167-168.

Tan, F. B. & Tung, L. L. (2003). Exploring website evaluation criteria using the repertory grid technique: A web designers’ perspective. Proceedings of the Second Annual Workshop on HCI Research, USA, 65-69.

Verlinden, J. C. & Coenders, J. J. (2000). Qualitative usability measurement of websites by employing repertory grid technique. Conference on Human Factors in Computing Systems archive.

Young, S. M., Edwards, H. M., McDonald S. J., Thompson, B. (2005). Personality characteristics in an XP team: A repertory grid study. Proceedings from Human and Social Factors of Software Engineering (HSSE), USA, 1-7.

Tagged with: , , , , , , , ,
Posted in Usability News
Subscribe to SURL

Want to receive notifications when SURL has new articles? Please enter your name and email address to subscribe to our website.

Popular Topics in Usability News
Log in/Sign Up
%d bloggers like this: