User Loyalty Articles

During the two cycles of the iterative design process the use of appropriate design that suits the topic, focus on the topic, delivery of content and the creation of a positive atmosphere were identified as the most essential aspects for a valuable user experience. The prerequisite for these aspects to become the most essential was that the lower levels of user needs, usability and availability, were fulfilled.

Four pages had personalisation features implemented, the start page, the news page, the idea-rating page and the start page of the tutorial. While the tutorial did not have enough visitors to be of interest in this study, the personalisation system of the idea-rating page lacked content and therefore could not work as planned. The findings show that the news page and the start page were better perceived by users that had the personalisation features enabled (B+users). The accuracy and coverage of the news page personalisation system could be confirmed. B+users perceived the website as more trustworthy, were more interested in new features of the site and were more likely to recommend the site. This indication of increased user loyalty was confirmed by data that showed that B+users were visiting the site more frequently. Therefore, the hypothesis, that it is possible by utilising dynamic personalisation to increase user loyalty to a site, is confirmed.

Start page

Even though in the overall category B+users rated the start page better than B-users, the categories of usability and importance were rated better by B-users, while structure was rated equally by both groups.

News page

The news page was rated overall and in structure and usability better by B+users, while B-users rated the importance slightly better.

Ideas page

B-users rated the ideas page as better in all four categories.

The survey further showed that all five B+users ‘always find something interesting in the news’, while B-users where less likely to find something of interest. B+users and B-users are both equally worried about their privacy on the Internet and B+users responded slightly more positive that the site was credible and personal. More B+users responded positively to the two questions that indicated their loyalty. B+users also were able to identify with the site slightly better. Expectations were equally met, even though the rating shows that they were not exceeded. While B-users rate the content of the site slightly better, B+users perceive the site as more trustworthy than do B-users.

The key findings are that pages which rely heavily on personalisation (start page, news page) are perceived better by B+users. The ideas page could not use the personalisation system that was implemented as it was planned because of its low amount of content (44 ideas at the time of the survey) distributed over 16 categories. Therefore user could not create profiles that represent their preferences.

The sum of all ratings of the first three sections show that the news page was the best-perceived section of the site followed by the start page and then the ideas page.

The impact of the news page personalisation on users is confirmed by the positive rating that B+ users ‘always find something interesting in the news’.

During the two months of the field study, the sum of all visits to the news page by users in both groups 396. The difference is that B+users visited 279 links to news while B-users visited only 159. This data confirms the statement that B+users were more likely to find something interesting in the news.

The effectiveness of the news page personalisation system was confirmed by the fact that four out of the five B+users named the same category of news as their favourite, according to their user profiles. An interesting aspect is that only one of the B-users named the category that was the highest ranked in their user profile. This could mean that the delivery of the most favoured news as the first news every time a B+user came back to the site has helped them to develop a concrete preference.

The fact that B+users could identify slightly better with the site and that the news page personalisation system delivered content similar to their preferences explains why the site is clearly more trustworthy in the eyes of B+users (Fogg, 2003, p. 123)

The importance of trust for loyalty (Koufaris et al.2002; Vatanasombut et al. 2004; Kim 2000) might have played a role in that more B+users responded positively to the two questions that indicated their loyalty to the site (Stratigos, pers. com. Feb. 19, 2005; Heater, pers. com. Feb. 19, 2005).

The evaluation of the question about how users would describe the atmosphere showed that the atmosphere was perceived as very positive.

Both questions, the question about the atmosphere and the question about what users miss on the site, were also used to evaluate certain aspects of the site.

Two B+users characterised the atmosphere of the site as ‘personal’ , two others said it was ‘dynamic’ and two said they enjoyed the feeling of being in a group, even though one mentioned that the start page should have a better structure. On the question of whether there was something missing, one B+user would like to see several improvements in the ideas section. Three B-users characterised the atmosphere as ‘pleasant’ and one emphasised that he enjoyed the community on the site.

On the question of whether something was missing, two B-users would like to see improvements in the ideas section while one would like to see more structure on the start page. A look at the results of the survey in general shows that the atmosphere on the site is related to the use of warm colours. Of 21 members who gave a comment on it, nine members mentioned the positive impact of the chosen colours on the atmosphere.

Five members said the atmosphere of the website was friendly, while four members said that the atmosphere was personal and they enjoyed the feeling of being in a community.

15 members responded to the question of whether there was something missing.

While eight members said they did not miss anything, five members would like to see improvements in the ideas section. One member said the start page was too unstructured, while two members mentioned this in the question about the atmosphere of the site. Therefore, this issue is not solved and will have to be addressed in the future.

An interesting finding is that users in the B+users and B-users group tended to associate atmosphere more with community and a personal touch, while other users, who used the site over a shorter time associated atmosphere more often with the impression they got from the colors and the visual design. Of all 10 members in the B+ and B- user group only one member mentioned the positive effect of colour.

For the general evaluation of tracking data only data from members who visited the site more than once was considered as valid. This was done to exclude the data of members who joined for the first survey and did not return.

The tracking data showed that B+users with two or more visits looked at an average of 1.47 news items each time they visited while B-users looked at 0.97 news items every visit. Over all, in the time from December 15 to February 15 B+users visited 439 news links and B-users 208. That was done during 299 visits of B+users and 215 by B-users (fig1). Over all, B+users were more likely to find something interesting in their news than B-users.

The run of the curve that shows the frequency of visits is similar between B+users and B-users, but a B+user was more likely to return after three visits.

On average, during these two months, B+users visited 28.09% times more often than B-users.

Following the Software Usability Measurement Inventory (SUMI) (Kirakowski 1994), which is a method of measuring software quality from the user’s point of view, a survey was prepared. The goal was to get more detailed information about the user experience the site provided and some indications of what impact that had on loyalty. The questions about usability (q. no. 3, 4) were implemented to reassure that no usability problems existed, because these problems would have direct impact on user satisfaction (Froekjmr et al. 2000).

Furthermore, the survey had questions which could be categorised into satisfaction, loyalty, website characteristics and background information. The survey was promoted in several forums and one had to become a member and visit every personalised page at least twice on the website before the link to the survey became visible. 64 members participated in the survey but only 10 of these were site members before the survey was announced. Therefore, the overall findings of the survey represent an extended first impression. They could only deliver an indication of the impact of personalisation features because even though new members had to visit every site a second time, these features provide valuable results for users over a longer period of time.

The survey confirmed the previous findings that the site was perceived well by users. Overall users with personalisation features enabled rated the site slightly higher than users without personalisation features. Of all participants, 69.2% were in the age of the target audience and nearly the half (31 participants) were using the internet more than 15 hours every week and were therefore considered as internet-savvy.

The response to the question (q. no. 7) in which participants had to characterise the site showed that only 5.4% named attributes that were considered as not intended by the designer and therefore negative. 94,6% of the selected attributes described the site as affectionate, personal, pleasant, unusual, exciting, bizarre, structured, warm and beautiful. This shows that aesthetic fidelity has been achieved. A surprising finding was that 23.4% rated ‘Atmosphere’ below ‘Content’ with 39.1%, as the most liked aspect of the site (q. no. . This result is positive because atmosphere describes ‘an interesting, often exotic, effect’ (Webster’s New World Dictionary 1988) and is considered a general emotional quality (Webster’s New World Thesaurus 1997) that can have an impact on the mood of the viewer (The Hutchinson Encyclopaedia 2003). That shows that the site can make an impact on users’ feelings. Even though it does not show what atmosphere the site creates. The high rating on ‘Content’ is positive because content is the ‘classic way to increase loyalty on the web’ (Nielsen 1997, 1999) and a way to attract new customers to a site (Wurster et al. 1999). This high rating confirmed that the chosen content for the website was appropriate.

A negative finding was that there was a gap between the overall impression and the perceived value a user got from visiting the site (q. no. 1,2). Even though the ‘overall impression’ is a mix of various elements and ‘what the site provides’ can be understood as related to content, it showed that the site generated expectations which it did not completely meet. This was repeated in the two questions about usability of the site (q. no.3,4). The structure of the site was rated better than how the users rated how they could navigate on the site.

Despite these results, more than half of the participants voted either good or very good in all four questions.

The question (q. no. of whether a user would return to the site was also answered positively by 31.5% of participants who said they would definitely return and 57.4% who thought they would return. The data of their user profiles showed that 33.3% of members who said they would definitely return and 80.6% of users who said they thought they would return did not return and log in. Therefore this question seemed to be unnecessary in the second survey.

The survey showed that users could be considered as overall satisfied. The usability and structure of the site was well perceived and aesthetic fidelity has been achieved. Questions that needed further investigation were about the kind of atmosphere that the site creates and if the site meets the user’s expectations. Furthermore the impact of personalisation features on the site had to be examined in detail.

Due to the low amount of user participation and, as a result, the relatively small amount of content, it was decided to use content personalisation in this research. Collaborative filtering would have been the wrong choice as it finds a profile of another user that is most similar to the users, but without many members it is unlikely to find peers that are similar enough to provide valuable results (Adomavicius et al. 2003). Furthermore, the news section has only a small amount of new information every day, which reduces the options for success of collaborative filtering even more. Another reason for using content personalisation were the findings of Shepherd et al. (2001) that users do not enjoy too detailed information filtering, therefore the news is only filtered according to news categories.

The personalisation features were implemented in the start page, the news page and the page on which the user rates ideas. Every member was identified by his or her login and password. Because A/B testing was used, every second member that registered surfed the site with personalisation features enabled (B+user) while the rest surfed the site without them (B-user).

The implicit data for the personalisation system derived from predefined PHP variables (PHP Group, 2003), which save user data that is sent to the server, and the tracking of users’ browsing behaviour. The system is based on a user profile in which all data is saved and every site that has personalised content on it is controlled by the user profile. The user profile consists of data for identification (Identification Profile), system data such as ip address or id (System Profile), self-categorised data such as age or gender (Socio-economic Profile) and data about the actions of the user on the site (Interaction Profile)(Schubert et al. 2000)(refer to Appendix J). Every time a user loads a page his user profile is updated in the appropriate category of interest. The start page is excluded because it shows all categories of content in five modules and therefore does not represent a specific interest. Over a period of time a profile of the users’ interest in the site is created. According to this profile the ratio of each content category of the start page is adjusted. If one category was not visited yet a link is shown with a call to visit this category while B-users did not see this feature. [P4] This is done to create awareness of existing features on the site and thereby increase the user experience (McMullin 2004). Furthermore, the start page shows special features and articles according to the operating system of the user and every user is greeted according to the time of the day and the time he has not[P5] visited the site. (pic1)

On the news page the news posts are sorted by starting with news from the category the user preferred most in the past. B-users see the news items according to the order in which they were published. The news is categorised according to the five news groups that were determined in the user-perception test. These five groups have subgroups with more specific topics (refer to Appendix I). When users click on the link to the news source their profile is updated in the corresponding category. To prevent the profile being updated several times when the news has more than one link, the user profile logs every news item a user has already read and compares this data with the incoming data from every update. Only the news that was posted since the user’s last visit or in the past six hours is sorted according to the user’s profile. This is done because changing all news in real time could confuse a user and harm the user experience. To sort the news the ratio of the viewed news of a user to the total amount of news in this category was calculated. This had to be done because not every category had the same amount of news and therefore the pure data in the user profile did not have to represent the user’s interest.

The user rank was also derived from their activity on the site and favourite news category. The activity is the sum of all values of the content categories in the user profile. (pic user profile structure)

The user rank feature is the only personalisation feature that uses explicit data to a minimal degree. The gender that users assign to themselves is used to give the title the right grammatical form.

Ideas are also categorised (refer to Appendix K). On the idea-rating page a B+ user will see an idea from a category in which he rated an idea before if he did not already rate all ideas of this category. These 100%, which represent all ratings he made, are divided according to the sum of all ratings in each category. Therefore if a user rates ideas from a particular [EB6] category highly, it is more likely that they will get more ideas from this category to rate next. In this personalisation feature no ratio is necessary because the ratings control the level of interest.

The personalisation features have multiple goals. In the short term they should provide users with a better user experience on the website. The long-term goal is to give the website a personal touch and thereby establish an emotional bond between the user and the website, which results in increased loyalty.

Following these findings the focus of the next version of the website shifted from establishing an online community to becoming a source for news and knowledge about the topic ideas and creativity. Therefore, the experience design focused on enhancing the usefulness of the news section and the visual design was redone to give the website a more desirable look. The low amount of user participation also influenced the choice of the personalisation systems which were used. To focus more on the topic of the web site and to satisfy the demand for background information the knowledge and the idea section were added.

The first step to enhance the news section was to give users another option to see the latest news. Therefore an RSS-feed was implemented. Furthermore, a user-perception test, with different icons for news categories, was conducted (Syarief et al. n. d.),. Eight participants in the target audience had to group icons in four groups and then assign category names to each of them. Every test was conducted separately. The evaluation showed that the category icons for the categories Design and Technology were inappropriate. Furthermore, two comments were made that the icon of the category Mobile ‘looks like an oven’, indicating that this category icon should also be changed. Another finding was that instead of four groups as was planned, five groups were better for the news categories. In a repeat test all of the three participants identified all icons properly (refer to Appendix H). A new idea section in which user can rate ideas or submit their own ideas offered a more concrete possibility than the forum for user to engage creatively.

To increase the credibility of the website, a privacy policy according to the ‘Online-Datenschutz-Prinzipien (transl: Online-Privacy-Principles) and an ‘Impressum’, which is a duty for commercial sites and recommended for non-profit sites under the German law, were written (Teledienstgesetz 2001, §6). Furthermore, detailed information about the background and people who run the site was added (Fogg, 2003, p.173). Content for the knowledge section and gallery was added and the news administration section was enhanced, making it easier to include pictures. Moreover, a section with a new citation and a new picture every day was added to the start page to give users another reason, besides the daily news, to visit the site on a regular basis.

The visual design of the beta site was build in contrast to the alpha site, using a spectrum of harmonic warm colours (Crüger 2004) with strong hues to make the site look exciting and active (Mahnke 1996, p.71). Another reason for this choice of colour was that warm colours increase brain activity (Birren 1984, p. 80) and Birren claims that extroverted people prefer warm colours over cold colours (Birren 1984, p. 119). The monk in the banner should give the site a face and communicate happiness. Faces are good ways to communicate emotions because these emotions are understood by everyone regardless of their background (Morand, 2001) and faces play a ‘major role in human interaction and nonverbal communication’ (Lien 1998). To establish a better brand recognition a logo was created. The first letter of idea symbolises a temple and the Asian font Wonton, together with the monk, give the site an Asian touch. The angular font for ‘temple’ is a strong contrast in shape to ‘idea’ and symbolises a foundation for ideas. The playful Asian font suits the topic ideas because ideas are not tangible (Boas 2003).The layout was changed from three-column to two-column layout because the site should focus more on the usefulness of its main features, which would be the news and the ideas section. An example of successful design for usefulness is the search engine Google. Because of its focus on usefulness it is one of the leaders in providing a valuable user experience (Hurst 2002).

After the site was put on line it was promoted in several forums(refer to Appendix E) which were selected according to the target audience. In the forums a thread was opened in the appropriate subcategory starting with a short description of the site and a call for feedback about the site.

28 people responded with their opinion about the site (refer to Appendix F). The first and most important finding was that the concept was considered a ‘good concept’ or ‘unusual’, which are both positive outcomes. The second finding was that five people stated that they like the design, but also five people said that it was not appropriate for the content of the site. Even though only five people supported this point of view, it was the only issue, besides the positive feedback, that was supported by several people.

The first poll in which 17 members participated, confirmed that the site was well received (refer to Appendix G). The second poll, in which 15 members participated, showed a demand for background information about ideas and that the site should deal more with the subject ‘ideas’ overall (7 votes). The second poll also confirmed that the design of the site did not suit the subject (6 votes).

Even though the majority of the feedback was not very detailed, the findings indicated that the visual design of the site was considered inappropriate. The site should also become more focussed on the topic ‘ideas’ by providing background information.

Despite the positive feedback, the number of members increased more slowly than expected.

First time visits to the site reached their peak usually on the day or the day after a forum thread was opened (refer to Appendix E).

A logfile analysis showed that the most visited page of the alpha site was the start page, which was also the news page. It was discovered that the news was the reason for users to visit the site again. That the news was the most popular part of the site was confirmed in November when the go.php page was implemented. This page decrypts encrypted links to articles that were posted in the news and became the most visited in the site.

In the competitor analysis (refer to Appendix B), websites with similar concepts were rated according to different criteria. The first key finding was that the main competitor of ideatemple was ideen-box.de because it was the only website of that kind in German and it had a high search-engine ranking (Google first result when people search for the German word for ideas, ‘ideen’). Another benefit of the site is its large amount of content in the form of ideas and the possibility of asking a lawyer copyright-related questions. The second key finding was that all examined sites did not have community features, with the exception of globalideasbank.org , but its forum lacked categories and had only a very small group of visitors. The third key finding was that no website provided background information or tutorials that were related to ideas except ideafinder.com, which lacked the possibility of submitting one’s own ideas. The fourth key finding was that the aesthetic and creative aspect was not considered in the websites that were compared.

While ideatemple did not have the resources to compete with the rich content of ideen-box.de, in the short term it was possible to implement community features, background information about ideas and develop an attractive and easy-to-use design that suited the subject.

The key findings of the literature review and the competitor analysis form the foundation of this research and the concept of the first version of the site is based on these findings.

As the internet is an active medium (Chak 2003, p.29) in which customers decide how they interact, it was important to give users various ways to contact the organisation behind the website and meet the demand for user-to-user interaction. The concept of the website included the possibility of getting assistance by email, using the chat feature to get live support, submit their own articles, submit their own news, submit their own ideas, or use the chat or forum to enter into a dialogue with other users. The news section also offered the possibility of writing comments to a news post. These features were included to adhere to the concept of establishing an online community (Rheingold 1993; Preece et al. 2003). A community offers the possibility of establishing relationships and improving the value of information (Tartaglia et al. 2005) and can generate much of the content of a site (Kotha 1998).

In addition creation of content by customers increases their involvement and enhances the credibility of a website (Kotha 1998). Another beneficial aspect of communities is that they produce loyal visitors. Studies show that active community members visit a site up to nine times more frequently (Sterne 2002, p. 180).

Online communities can be categorised as communities of relationship, communities of interest, communities of transaction, and communities of fantasy (Armstrong et al. 1996). The community that was created in this research was primarily a community of interest, in which people who are interested in the topic, ideas and creativity, can meet and exchange. A secondary goal was to establish a community of relationship because increased social interactions among users generate a ‘switching cost’ (Ginsburg et al. 2004) or barrier. Positive interpersonal relationships are viewed as positive switching barriers (Julander et al. 2003) because users would have to leave these interactions behind if they were to defect to another website.

The open source CMS PHP-Fusion was selected because it had several features like a forum, options to submit news and submit articles already implemented. The reason for using open-source software was that it is free to anyone to do modifications on the source code. PHP-Fusion is under the GNU Public Licence (1991) and therefore everyone is permitted to edit the source code.

The content for the site had to be interesting, unusual and related to ideas and innovations, therefore the weblog awards winner of 2004, engadget.com, which is a technology weblog and its sources, were used to gather news for the site. Using the format of weblogs as a news source had several advantages. The main advantage is that they are written by amateurs and not much effort is needed to create a weblog and therefore they can deal with much more specific content than traditional journalists who have to focus on attracting a crowd of people (Rosen 2003). The growing popularity of weblogs shows in a study that in 2004 the weblog readership increased by 58% in the US and 27% of ‘on-line Americans’ read weblogs (BBC 2005).

An example of a weblog that demonstrates the variety of content that is available is the weblog, ‘A star from Mosul’, in which a 16-year-old girl writes about her daily life in post-Saddam Iraq (Abdullah, 2005).

Attractive graphic design is a key element in making interfaces more enjoyable to use (Shneiderman 2004), which enhances the user experience. The importance of consistency (Brannon et al. 2000) and design is crucial for the user experience (Tedeschi 1999) (Lynch et al. 2004). Before the final design was developed several other possibilities were considered (refer to Appendix C). In order to unify the design (Graham 2001, p.84) a three-column grid was used (Graham 2001, p. 112). The decision to use ‘cold colours’ was made because of their relaxing effect on the viewer (Birren 1984, p81). Blue also stands for ‘a sense of balance, calm, and security’ (Lu, 2004), which are attributes that seemed appropriate, as ideas can be very personal (WorldNet, Definition ‘idea’). The minimal use of controls in the top navigation with the only options to choose ‘Home’ or ‘Forum’ emphasised the community features (Hearst 2005). A system for news categories was implemented which should become personalised in the beta phase. Additional fields to add links to news were implemented in the news administration to increase the functionality. To give guests a reason to become a member, the link and the download section were filled with content and became a members-only section. Furthermore, the news arena panel, with additional news for members, was implemented.

The initial concept was focused on creating an aesthetically designed website (sight experience), establishing an online community to provide users with the opportunity of a co-experience and to create switching barriers through social interactions. Furthermore, the website should motivate users to engage creatively (think experience) in one of the seven forum categories (refer to Appendix D). The main goal was to establish a loyal user base for the field study.

Figure 3: First version of the website.