User Loyalty Articles

Discussion and future work

The findings after the first cycle of the iterative design process depicted that users demanded an appropriate visual design that suits the topic and the findings of the first survey that 23.4% of the users liked the site because of the atmosphere having parts, which overlapped from the visual design. This was shown by the descriptions of atmosphere in the second survey in which 41% of users related the atmosphere to the visual design. The reason that the visual design made an impact was because aesthetic fidelity was achieved.

After the first cycle of the iterative design process it became clear that the participation of users would be less than expected and the concept had to be changed. Even though, the site already had the main features such as a forum and the possibility to submit own articles, the understanding of the audience was missing, which is crucial for developing an online community (Preece et al. 2004).

Therefore the goal of the website shifted to providing news and knowledge about the topic. The development was concentrated on creating more focus on the topic by providing background information and on developing an appropriate visual design. The main problem at this stage of the process was the lack of contribution of users to the content of the site. The idea section should bring more focus on the topic and give users another possibility to add content while the tutorial section was developed to guide users through the process of idea creation.

Feedback from forums in which problems with the structure of the start page of the second version of the site were mentioned were addressed by a test in which the aesthetic aspect and the scanability was tested. The test, with participants who were new to the interface, was conducted with the assumption that if users of a forum who were new to the interface were able to identify problems with the structure of the start page, the participants would be able to do that as well. The findings suggested that a more structured start page would decrease the aesthetic aspect of the page and therefore have a negative impact on the user experience while the scanability was only slightly faster on a better-structured site. During the progress of this research, comments of long-term users about the lack of structure on the start page showed that the test did not produce valid results for long-term users. It confirmed Dillon’s et al. (2001) findings that ‘evaluations based on single experiences with technology do not provide stable estimates of long-term (or even medium-term) usability.”

Certain limitations of this research must be acknowledged. Due to the complexity of user experience (Olsen, 2003) the essential aspects for the user experience, which were found, might not apply in general, especially given the specific topic and the resulting target audience. The focus on the German speaking countries, and as a result, the cultural background of users might have had an impact on the findings as well.

Second, the period of time for the field study, two months, might also be a limitation. The short period of time did not allow to establish an online community of any size in which personalised user rankings and identity played a significant role. Even though, the descriptions of the atmosphere of long-term users confirmed that it was possible to establish a “sense of community ” (Blanchard et al. 2001).

Third, because the final survey relies on data from a small group of people, five b+ user and five b- user, their personal interests in the website could have played a role in their judgements.

The result that four out of five B+ user could name the same favourite news category as in their user profile, while only one of the five B- user could, suggests that personalisation helped to create personal preferences or helped to identify them. The knowledge of a certain preference could make a user more sensitive to changes on a site and therefore have a negative impact on their loyalty.

Furthermore, the delivery of personalised content could limit the users perception of the variety of information on the site and therefore have a negative long-term effect on loyalty. Thus, future researchers should examine whether that the findings of dynamic personalisation can have an impact on user loyalty is consistent over a longer period of time. The creation of a personalisation lifecycle, which begins by creating a user profile and ends in de-personalisation, could be the framework for this future research.

The results of the final survey show that B+user consider the website as more trustworthy. Similarity, or in the case of this research content similar to the users preference can increase trust. Fogg (2003) identified credibility, which consists of trust and expertise, as a key factor for persuasive webdesign. The findings suggest that the use of personalisation can enhance the impact of persuasive webdesign. While persuasive webdesign only played a role in establishing credibility of the website. Future research should focus on the impact that personalisation can have in combination with persuasive webdesign.

Start page

Even though in the overall category B+users rated the start page better than B-users, the categories of usability and importance were rated better by B-users, while structure was rated equally by both groups.

News page

The news page was rated overall and in structure and usability better by B+users, while B-users rated the importance slightly better.

Ideas page

B-users rated the ideas page as better in all four categories.

The survey further showed that all five B+users ‘always find something interesting in the news’, while B-users where less likely to find something of interest. B+users and B-users are both equally worried about their privacy on the Internet and B+users responded slightly more positive that the site was credible and personal. More B+users responded positively to the two questions that indicated their loyalty. B+users also were able to identify with the site slightly better. Expectations were equally met, even though the rating shows that they were not exceeded. While B-users rate the content of the site slightly better, B+users perceive the site as more trustworthy than do B-users.

The key findings are that pages which rely heavily on personalisation (start page, news page) are perceived better by B+users. The ideas page could not use the personalisation system that was implemented as it was planned because of its low amount of content (44 ideas at the time of the survey) distributed over 16 categories. Therefore user could not create profiles that represent their preferences.

The sum of all ratings of the first three sections show that the news page was the best-perceived section of the site followed by the start page and then the ideas page.

The impact of the news page personalisation on users is confirmed by the positive rating that B+ users ‘always find something interesting in the news’.

During the two months of the field study, the sum of all visits to the news page by users in both groups 396. The difference is that B+users visited 279 links to news while B-users visited only 159. This data confirms the statement that B+users were more likely to find something interesting in the news.

The effectiveness of the news page personalisation system was confirmed by the fact that four out of the five B+users named the same category of news as their favourite, according to their user profiles. An interesting aspect is that only one of the B-users named the category that was the highest ranked in their user profile. This could mean that the delivery of the most favoured news as the first news every time a B+user came back to the site has helped them to develop a concrete preference.

The fact that B+users could identify slightly better with the site and that the news page personalisation system delivered content similar to their preferences explains why the site is clearly more trustworthy in the eyes of B+users (Fogg, 2003, p. 123)

The importance of trust for loyalty (Koufaris et al.2002; Vatanasombut et al. 2004; Kim 2000) might have played a role in that more B+users responded positively to the two questions that indicated their loyalty to the site (Stratigos, pers. com. Feb. 19, 2005; Heater, pers. com. Feb. 19, 2005).

The evaluation of the question about how users would describe the atmosphere showed that the atmosphere was perceived as very positive.

Both questions, the question about the atmosphere and the question about what users miss on the site, were also used to evaluate certain aspects of the site.

Two B+users characterised the atmosphere of the site as ‘personal’ , two others said it was ‘dynamic’ and two said they enjoyed the feeling of being in a group, even though one mentioned that the start page should have a better structure. On the question of whether there was something missing, one B+user would like to see several improvements in the ideas section. Three B-users characterised the atmosphere as ‘pleasant’ and one emphasised that he enjoyed the community on the site.

On the question of whether something was missing, two B-users would like to see improvements in the ideas section while one would like to see more structure on the start page. A look at the results of the survey in general shows that the atmosphere on the site is related to the use of warm colours. Of 21 members who gave a comment on it, nine members mentioned the positive impact of the chosen colours on the atmosphere.

Five members said the atmosphere of the website was friendly, while four members said that the atmosphere was personal and they enjoyed the feeling of being in a community.

15 members responded to the question of whether there was something missing.

While eight members said they did not miss anything, five members would like to see improvements in the ideas section. One member said the start page was too unstructured, while two members mentioned this in the question about the atmosphere of the site. Therefore, this issue is not solved and will have to be addressed in the future.

An interesting finding is that users in the B+users and B-users group tended to associate atmosphere more with community and a personal touch, while other users, who used the site over a shorter time associated atmosphere more often with the impression they got from the colors and the visual design. Of all 10 members in the B+ and B- user group only one member mentioned the positive effect of colour.

For the general evaluation of tracking data only data from members who visited the site more than once was considered as valid. This was done to exclude the data of members who joined for the first survey and did not return.

The tracking data showed that B+users with two or more visits looked at an average of 1.47 news items each time they visited while B-users looked at 0.97 news items every visit. Over all, in the time from December 15 to February 15 B+users visited 439 news links and B-users 208. That was done during 299 visits of B+users and 215 by B-users (fig1). Over all, B+users were more likely to find something interesting in their news than B-users.

The run of the curve that shows the frequency of visits is similar between B+users and B-users, but a B+user was more likely to return after three visits.

On average, during these two months, B+users visited 28.09% times more often than B-users.

Following the Software Usability Measurement Inventory (SUMI) (Kirakowski 1994), which is a method of measuring software quality from the user’s point of view, a survey was prepared. The goal was to get more detailed information about the user experience the site provided and some indications of what impact that had on loyalty. The questions about usability (q. no. 3, 4) were implemented to reassure that no usability problems existed, because these problems would have direct impact on user satisfaction (Froekjmr et al. 2000).

Furthermore, the survey had questions which could be categorised into satisfaction, loyalty, website characteristics and background information. The survey was promoted in several forums and one had to become a member and visit every personalised page at least twice on the website before the link to the survey became visible. 64 members participated in the survey but only 10 of these were site members before the survey was announced. Therefore, the overall findings of the survey represent an extended first impression. They could only deliver an indication of the impact of personalisation features because even though new members had to visit every site a second time, these features provide valuable results for users over a longer period of time.

The survey confirmed the previous findings that the site was perceived well by users. Overall users with personalisation features enabled rated the site slightly higher than users without personalisation features. Of all participants, 69.2% were in the age of the target audience and nearly the half (31 participants) were using the internet more than 15 hours every week and were therefore considered as internet-savvy.

The response to the question (q. no. 7) in which participants had to characterise the site showed that only 5.4% named attributes that were considered as not intended by the designer and therefore negative. 94,6% of the selected attributes described the site as affectionate, personal, pleasant, unusual, exciting, bizarre, structured, warm and beautiful. This shows that aesthetic fidelity has been achieved. A surprising finding was that 23.4% rated ‘Atmosphere’ below ‘Content’ with 39.1%, as the most liked aspect of the site (q. no. . This result is positive because atmosphere describes ‘an interesting, often exotic, effect’ (Webster’s New World Dictionary 1988) and is considered a general emotional quality (Webster’s New World Thesaurus 1997) that can have an impact on the mood of the viewer (The Hutchinson Encyclopaedia 2003). That shows that the site can make an impact on users’ feelings. Even though it does not show what atmosphere the site creates. The high rating on ‘Content’ is positive because content is the ‘classic way to increase loyalty on the web’ (Nielsen 1997, 1999) and a way to attract new customers to a site (Wurster et al. 1999). This high rating confirmed that the chosen content for the website was appropriate.

A negative finding was that there was a gap between the overall impression and the perceived value a user got from visiting the site (q. no. 1,2). Even though the ‘overall impression’ is a mix of various elements and ‘what the site provides’ can be understood as related to content, it showed that the site generated expectations which it did not completely meet. This was repeated in the two questions about usability of the site (q. no.3,4). The structure of the site was rated better than how the users rated how they could navigate on the site.

Despite these results, more than half of the participants voted either good or very good in all four questions.

The question (q. no. of whether a user would return to the site was also answered positively by 31.5% of participants who said they would definitely return and 57.4% who thought they would return. The data of their user profiles showed that 33.3% of members who said they would definitely return and 80.6% of users who said they thought they would return did not return and log in. Therefore this question seemed to be unnecessary in the second survey.

The survey showed that users could be considered as overall satisfied. The usability and structure of the site was well perceived and aesthetic fidelity has been achieved. Questions that needed further investigation were about the kind of atmosphere that the site creates and if the site meets the user’s expectations. Furthermore the impact of personalisation features on the site had to be examined in detail.

User experience includes several aspects such as branding, content usability and functionality (Rubinoff 2004) and the multi-layered design process (Olsen 2003) is usually supported by a team of experts (McBride 2004).

Previous research in the field of user experience showed that website navigability and relationship services, such as virtual community building and site personalisation, improve the experience and attract users to visit a website (Kotha et al. 2001). The core of user experience is multifaceted and several models have been developed.

Dillon et al. (2001) proposed that user experience is the sum of action plus result plus emotions. Kankainen (2002) in his model describes user experience as the ‘result of motivated action in a certain context’. The previous experiences influence the present experience and the present experience will shape the expectations for future experience.

According to Wright (2003) an experience consists of four threads.

The compositional thread describes the structure of an experience, the sensual thread is the ‘look and feel’ of the experience, the emotional thread consists of feeling involved and the spatio-temporal thread describes the time and space in which the experience takes place. He concludes that the expectations a user has in the next experience are as important as the environment that is provided for the experience.

Budd et al. (2003) describes the term ‘experience design’ with the goal of creating positive user experiences. He demands that, because user experience can be experience from a variety of experience types, practitioners from the different areas must work together in the future.

Schmitt (2004) describes different types of experiences. The sense experience appeals to the five senses, sight, sound, smell, taste and touch. In the feel experience value is created by creating positive emotions. The think experience creates value by engaging user creatively. The act experience creates value by showing users alternative ways of doing something and the relate experience creates value by giving users a feeling that they belong to a group. Batterbee et al. (2003) names the relate experience ‘co-experience’, which is ‘ the experience that users themselves create together in social interaction’.

Previous studies suggest there is a link between customer satisfaction and customer loyalty, even though they differ.

Meyer et al. (2001) assume, on the basis of a comparison of satisfaction levels of customers and rebuying behaviour, a progressive relationship between customer loyalty and customer satisfaction, while Müller et al. (1991) prove with empiric results that the relationship has a plateau phase in which customer loyalty remains the same even with increased customer satisfaction.

Homburg (2003) identified in his construct of customer satisfaction the connection between expectations that had been met, expectations that had been exceeded and unfulfilled expectations. The construct shows that exceeded expectations have enthusiastic customers as a result. Bitner (1998) and Costabile (1998) found evidence of a close connection between satisfaction over time and trust. They found that loyalty and trust were a result of the experience of satisfaction over a period of time.

The link between satisfaction, trust and loyalty could not always be confirmed. (Jacoby et al. 1978). Singh et al. (2000) findings show that trust is not the most important factor for creating loyalty. He finds that customers want extra value, which is the optimal combination of time, effort and cost savings. The different findings show the need for further research in this area, but in general, one can see a loose connection between customer satisfaction and customer loyalty. However, highly satisfied customers do not have to be highly loyal customers and vice versa.

Another driving force for loyal behaviour can be barriers. People might stay loyal to a company because of the high switching cost. In 2004 a study, conducted by InterUnity Group, showed that the IT company SAP had the most loyal customers, even though it might not have the most satisfied. In this case 30% of the respondents said they would feel trapped because of the high cost of change (Bailor 2004; McCue 2004).

Online loyalty involves special aspects. Chau et al. (2002) finds that the design of websites strongly determines the motivation of a user to use them.

Murray et al. (2003) suggests that the interface of a website attracts new users when it can be handled with skills a user already has. To establish long-term interface loyalty one has to encourage the development of non-transferable user skills.

Koufaris (et al. 2002) found that positive experiences with a website lead to increased trust in the company that runs the website and this has a positive impact on customer retention and intention to buy. Vatanasombut et al. (2004) suggests that retention initiatives for internet-savvy users ‘should focus on measures that create commitment and trust in the relationship’. Possibilities to do that include ‘implementing effective and proactive communications and ensuring perceived security’. Kim (2000) findings show similar results as he proposes that ‘comprehensive information, shared value, and diverse communications affect the level of trust, which in turn influence customer loyalty’.