User Loyalty Articles

During the two cycles of the iterative design process the use of appropriate design that suits the topic, focus on the topic, delivery of content and the creation of a positive atmosphere were identified as the most essential aspects for a valuable user experience. The prerequisite for these aspects to become the most essential was that the lower levels of user needs, usability and availability, were fulfilled.

Four pages had personalisation features implemented, the start page, the news page, the idea-rating page and the start page of the tutorial. While the tutorial did not have enough visitors to be of interest in this study, the personalisation system of the idea-rating page lacked content and therefore could not work as planned. The findings show that the news page and the start page were better perceived by users that had the personalisation features enabled (B+users). The accuracy and coverage of the news page personalisation system could be confirmed. B+users perceived the website as more trustworthy, were more interested in new features of the site and were more likely to recommend the site. This indication of increased user loyalty was confirmed by data that showed that B+users were visiting the site more frequently. Therefore, the hypothesis, that it is possible by utilising dynamic personalisation to increase user loyalty to a site, is confirmed.

Discussion and future work

The findings after the first cycle of the iterative design process depicted that users demanded an appropriate visual design that suits the topic and the findings of the first survey that 23.4% of the users liked the site because of the atmosphere having parts, which overlapped from the visual design. This was shown by the descriptions of atmosphere in the second survey in which 41% of users related the atmosphere to the visual design. The reason that the visual design made an impact was because aesthetic fidelity was achieved.

After the first cycle of the iterative design process it became clear that the participation of users would be less than expected and the concept had to be changed. Even though, the site already had the main features such as a forum and the possibility to submit own articles, the understanding of the audience was missing, which is crucial for developing an online community (Preece et al. 2004).

Therefore the goal of the website shifted to providing news and knowledge about the topic. The development was concentrated on creating more focus on the topic by providing background information and on developing an appropriate visual design. The main problem at this stage of the process was the lack of contribution of users to the content of the site. The idea section should bring more focus on the topic and give users another possibility to add content while the tutorial section was developed to guide users through the process of idea creation.

Feedback from forums in which problems with the structure of the start page of the second version of the site were mentioned were addressed by a test in which the aesthetic aspect and the scanability was tested. The test, with participants who were new to the interface, was conducted with the assumption that if users of a forum who were new to the interface were able to identify problems with the structure of the start page, the participants would be able to do that as well. The findings suggested that a more structured start page would decrease the aesthetic aspect of the page and therefore have a negative impact on the user experience while the scanability was only slightly faster on a better-structured site. During the progress of this research, comments of long-term users about the lack of structure on the start page showed that the test did not produce valid results for long-term users. It confirmed Dillon’s et al. (2001) findings that ‘evaluations based on single experiences with technology do not provide stable estimates of long-term (or even medium-term) usability.”

Certain limitations of this research must be acknowledged. Due to the complexity of user experience (Olsen, 2003) the essential aspects for the user experience, which were found, might not apply in general, especially given the specific topic and the resulting target audience. The focus on the German speaking countries, and as a result, the cultural background of users might have had an impact on the findings as well.

Second, the period of time for the field study, two months, might also be a limitation. The short period of time did not allow to establish an online community of any size in which personalised user rankings and identity played a significant role. Even though, the descriptions of the atmosphere of long-term users confirmed that it was possible to establish a “sense of community ” (Blanchard et al. 2001).

Third, because the final survey relies on data from a small group of people, five b+ user and five b- user, their personal interests in the website could have played a role in their judgements.

The result that four out of five B+ user could name the same favourite news category as in their user profile, while only one of the five B- user could, suggests that personalisation helped to create personal preferences or helped to identify them. The knowledge of a certain preference could make a user more sensitive to changes on a site and therefore have a negative impact on their loyalty.

Furthermore, the delivery of personalised content could limit the users perception of the variety of information on the site and therefore have a negative long-term effect on loyalty. Thus, future researchers should examine whether that the findings of dynamic personalisation can have an impact on user loyalty is consistent over a longer period of time. The creation of a personalisation lifecycle, which begins by creating a user profile and ends in de-personalisation, could be the framework for this future research.

The results of the final survey show that B+user consider the website as more trustworthy. Similarity, or in the case of this research content similar to the users preference can increase trust. Fogg (2003) identified credibility, which consists of trust and expertise, as a key factor for persuasive webdesign. The findings suggest that the use of personalisation can enhance the impact of persuasive webdesign. While persuasive webdesign only played a role in establishing credibility of the website. Future research should focus on the impact that personalisation can have in combination with persuasive webdesign.

The second survey was conducted three weeks after the first survey.

The second-survey questions about the loyalty of users were derived from indices used in the industry (Heater, pers. com. Feb. 19, 2005; Stratigos, pers. com. Feb. 19, 2005). To understand how efficiently the personalisation system works one has to measure the accuracy and the coverage (Mobasher et al. 2001)(Sheperd et al. 2001). Accuracy measures how precisely the personalisation system can deliver content that the user likes and coverage measures how likely the user is to accept the content.

Questions about the credibility of the site were important because without credibility a website cannot ‘persuade users to change their attitudes or behaviours’ (Fogg 2003, p. 148).

Unlike the first survey, the second survey was not promoted, except on the site itself and through emails to members. This was done because this survey was designed for long-term users of the site.

Data from the five most active B+users and the five most active B-users who completed the survey were evaluated, because this group of users experienced the site over a longer period of time. Every participant was a site member for at least 15 days, had at least seven visits to the site and read at least six news items distributed on different visits. During the time they were members the sum of all visits of the B+user group was 166 while the B-users visited 152 times.

In the first section of the survey participants had to rate aspects of every site in which personalisation features were included. The part of the survey in which users had to rate the tutorial was not evaluated because an insufficient number of users did the tutorial.

Due to the low amount of user participation and, as a result, the relatively small amount of content, it was decided to use content personalisation in this research. Collaborative filtering would have been the wrong choice as it finds a profile of another user that is most similar to the users, but without many members it is unlikely to find peers that are similar enough to provide valuable results (Adomavicius et al. 2003). Furthermore, the news section has only a small amount of new information every day, which reduces the options for success of collaborative filtering even more. Another reason for using content personalisation were the findings of Shepherd et al. (2001) that users do not enjoy too detailed information filtering, therefore the news is only filtered according to news categories.

The personalisation features were implemented in the start page, the news page and the page on which the user rates ideas. Every member was identified by his or her login and password. Because A/B testing was used, every second member that registered surfed the site with personalisation features enabled (B+user) while the rest surfed the site without them (B-user).

The implicit data for the personalisation system derived from predefined PHP variables (PHP Group, 2003), which save user data that is sent to the server, and the tracking of users’ browsing behaviour. The system is based on a user profile in which all data is saved and every site that has personalised content on it is controlled by the user profile. The user profile consists of data for identification (Identification Profile), system data such as ip address or id (System Profile), self-categorised data such as age or gender (Socio-economic Profile) and data about the actions of the user on the site (Interaction Profile)(Schubert et al. 2000)(refer to Appendix J). Every time a user loads a page his user profile is updated in the appropriate category of interest. The start page is excluded because it shows all categories of content in five modules and therefore does not represent a specific interest. Over a period of time a profile of the users’ interest in the site is created. According to this profile the ratio of each content category of the start page is adjusted. If one category was not visited yet a link is shown with a call to visit this category while B-users did not see this feature. [P4] This is done to create awareness of existing features on the site and thereby increase the user experience (McMullin 2004). Furthermore, the start page shows special features and articles according to the operating system of the user and every user is greeted according to the time of the day and the time he has not[P5] visited the site. (pic1)

On the news page the news posts are sorted by starting with news from the category the user preferred most in the past. B-users see the news items according to the order in which they were published. The news is categorised according to the five news groups that were determined in the user-perception test. These five groups have subgroups with more specific topics (refer to Appendix I). When users click on the link to the news source their profile is updated in the corresponding category. To prevent the profile being updated several times when the news has more than one link, the user profile logs every news item a user has already read and compares this data with the incoming data from every update. Only the news that was posted since the user’s last visit or in the past six hours is sorted according to the user’s profile. This is done because changing all news in real time could confuse a user and harm the user experience. To sort the news the ratio of the viewed news of a user to the total amount of news in this category was calculated. This had to be done because not every category had the same amount of news and therefore the pure data in the user profile did not have to represent the user’s interest.

The user rank was also derived from their activity on the site and favourite news category. The activity is the sum of all values of the content categories in the user profile. (pic user profile structure)

The user rank feature is the only personalisation feature that uses explicit data to a minimal degree. The gender that users assign to themselves is used to give the title the right grammatical form.

Ideas are also categorised (refer to Appendix K). On the idea-rating page a B+ user will see an idea from a category in which he rated an idea before if he did not already rate all ideas of this category. These 100%, which represent all ratings he made, are divided according to the sum of all ratings in each category. Therefore if a user rates ideas from a particular [EB6] category highly, it is more likely that they will get more ideas from this category to rate next. In this personalisation feature no ratio is necessary because the ratings control the level of interest.

The personalisation features have multiple goals. In the short term they should provide users with a better user experience on the website. The long-term goal is to give the website a personal touch and thereby establish an emotional bond between the user and the website, which results in increased loyalty.

After the site was put on line it was promoted in several forums(refer to Appendix E) which were selected according to the target audience. In the forums a thread was opened in the appropriate subcategory starting with a short description of the site and a call for feedback about the site.

28 people responded with their opinion about the site (refer to Appendix F). The first and most important finding was that the concept was considered a ‘good concept’ or ‘unusual’, which are both positive outcomes. The second finding was that five people stated that they like the design, but also five people said that it was not appropriate for the content of the site. Even though only five people supported this point of view, it was the only issue, besides the positive feedback, that was supported by several people.

The first poll in which 17 members participated, confirmed that the site was well received (refer to Appendix G). The second poll, in which 15 members participated, showed a demand for background information about ideas and that the site should deal more with the subject ‘ideas’ overall (7 votes). The second poll also confirmed that the design of the site did not suit the subject (6 votes).

Even though the majority of the feedback was not very detailed, the findings indicated that the visual design of the site was considered inappropriate. The site should also become more focussed on the topic ‘ideas’ by providing background information.

Despite the positive feedback, the number of members increased more slowly than expected.

First time visits to the site reached their peak usually on the day or the day after a forum thread was opened (refer to Appendix E).

A logfile analysis showed that the most visited page of the alpha site was the start page, which was also the news page. It was discovered that the news was the reason for users to visit the site again. That the news was the most popular part of the site was confirmed in November when the go.php page was implemented. This page decrypts encrypted links to articles that were posted in the news and became the most visited in the site.

In the competitor analysis (refer to Appendix B), websites with similar concepts were rated according to different criteria. The first key finding was that the main competitor of ideatemple was ideen-box.de because it was the only website of that kind in German and it had a high search-engine ranking (Google first result when people search for the German word for ideas, ‘ideen’). Another benefit of the site is its large amount of content in the form of ideas and the possibility of asking a lawyer copyright-related questions. The second key finding was that all examined sites did not have community features, with the exception of globalideasbank.org , but its forum lacked categories and had only a very small group of visitors. The third key finding was that no website provided background information or tutorials that were related to ideas except ideafinder.com, which lacked the possibility of submitting one’s own ideas. The fourth key finding was that the aesthetic and creative aspect was not considered in the websites that were compared.

While ideatemple did not have the resources to compete with the rich content of ideen-box.de, in the short term it was possible to implement community features, background information about ideas and develop an attractive and easy-to-use design that suited the subject.

The key findings of the literature review and the competitor analysis form the foundation of this research and the concept of the first version of the site is based on these findings.

As the internet is an active medium (Chak 2003, p.29) in which customers decide how they interact, it was important to give users various ways to contact the organisation behind the website and meet the demand for user-to-user interaction. The concept of the website included the possibility of getting assistance by email, using the chat feature to get live support, submit their own articles, submit their own news, submit their own ideas, or use the chat or forum to enter into a dialogue with other users. The news section also offered the possibility of writing comments to a news post. These features were included to adhere to the concept of establishing an online community (Rheingold 1993; Preece et al. 2003). A community offers the possibility of establishing relationships and improving the value of information (Tartaglia et al. 2005) and can generate much of the content of a site (Kotha 1998).

In addition creation of content by customers increases their involvement and enhances the credibility of a website (Kotha 1998). Another beneficial aspect of communities is that they produce loyal visitors. Studies show that active community members visit a site up to nine times more frequently (Sterne 2002, p. 180).

Online communities can be categorised as communities of relationship, communities of interest, communities of transaction, and communities of fantasy (Armstrong et al. 1996). The community that was created in this research was primarily a community of interest, in which people who are interested in the topic, ideas and creativity, can meet and exchange. A secondary goal was to establish a community of relationship because increased social interactions among users generate a ‘switching cost’ (Ginsburg et al. 2004) or barrier. Positive interpersonal relationships are viewed as positive switching barriers (Julander et al. 2003) because users would have to leave these interactions behind if they were to defect to another website.

The open source CMS PHP-Fusion was selected because it had several features like a forum, options to submit news and submit articles already implemented. The reason for using open-source software was that it is free to anyone to do modifications on the source code. PHP-Fusion is under the GNU Public Licence (1991) and therefore everyone is permitted to edit the source code.

The content for the site had to be interesting, unusual and related to ideas and innovations, therefore the weblog awards winner of 2004, engadget.com, which is a technology weblog and its sources, were used to gather news for the site. Using the format of weblogs as a news source had several advantages. The main advantage is that they are written by amateurs and not much effort is needed to create a weblog and therefore they can deal with much more specific content than traditional journalists who have to focus on attracting a crowd of people (Rosen 2003). The growing popularity of weblogs shows in a study that in 2004 the weblog readership increased by 58% in the US and 27% of ‘on-line Americans’ read weblogs (BBC 2005).

An example of a weblog that demonstrates the variety of content that is available is the weblog, ‘A star from Mosul’, in which a 16-year-old girl writes about her daily life in post-Saddam Iraq (Abdullah, 2005).

Attractive graphic design is a key element in making interfaces more enjoyable to use (Shneiderman 2004), which enhances the user experience. The importance of consistency (Brannon et al. 2000) and design is crucial for the user experience (Tedeschi 1999) (Lynch et al. 2004). Before the final design was developed several other possibilities were considered (refer to Appendix C). In order to unify the design (Graham 2001, p.84) a three-column grid was used (Graham 2001, p. 112). The decision to use ‘cold colours’ was made because of their relaxing effect on the viewer (Birren 1984, p81). Blue also stands for ‘a sense of balance, calm, and security’ (Lu, 2004), which are attributes that seemed appropriate, as ideas can be very personal (WorldNet, Definition ‘idea’). The minimal use of controls in the top navigation with the only options to choose ‘Home’ or ‘Forum’ emphasised the community features (Hearst 2005). A system for news categories was implemented which should become personalised in the beta phase. Additional fields to add links to news were implemented in the news administration to increase the functionality. To give guests a reason to become a member, the link and the download section were filled with content and became a members-only section. Furthermore, the news arena panel, with additional news for members, was implemented.

The initial concept was focused on creating an aesthetically designed website (sight experience), establishing an online community to provide users with the opportunity of a co-experience and to create switching barriers through social interactions. Furthermore, the website should motivate users to engage creatively (think experience) in one of the seven forum categories (refer to Appendix D). The main goal was to establish a loyal user base for the field study.

Figure 3: First version of the website.

A successful user experience combines several types of experience (Schmitt, 2004). Expectations play an important role in user experience and also for user satisfaction. The goal in order to establish loyalty must be that the value of the user experience exceeds the user’s expectations. The result is a positive experience and satisfaction. Personalisation can deliver benefits in the form of saved time and effort or personal treatment, but a personalisation system has to utilise the right features and form of presentation to work efficiently and create value for the user.

User experience includes several aspects such as branding, content usability and functionality (Rubinoff 2004) and the multi-layered design process (Olsen 2003) is usually supported by a team of experts (McBride 2004).

Previous research in the field of user experience showed that website navigability and relationship services, such as virtual community building and site personalisation, improve the experience and attract users to visit a website (Kotha et al. 2001). The core of user experience is multifaceted and several models have been developed.

Dillon et al. (2001) proposed that user experience is the sum of action plus result plus emotions. Kankainen (2002) in his model describes user experience as the ‘result of motivated action in a certain context’. The previous experiences influence the present experience and the present experience will shape the expectations for future experience.

According to Wright (2003) an experience consists of four threads.

The compositional thread describes the structure of an experience, the sensual thread is the ‘look and feel’ of the experience, the emotional thread consists of feeling involved and the spatio-temporal thread describes the time and space in which the experience takes place. He concludes that the expectations a user has in the next experience are as important as the environment that is provided for the experience.

Budd et al. (2003) describes the term ‘experience design’ with the goal of creating positive user experiences. He demands that, because user experience can be experience from a variety of experience types, practitioners from the different areas must work together in the future.

Schmitt (2004) describes different types of experiences. The sense experience appeals to the five senses, sight, sound, smell, taste and touch. In the feel experience value is created by creating positive emotions. The think experience creates value by engaging user creatively. The act experience creates value by showing users alternative ways of doing something and the relate experience creates value by giving users a feeling that they belong to a group. Batterbee et al. (2003) names the relate experience ‘co-experience’, which is ‘ the experience that users themselves create together in social interaction’.