Brief 122749 Evaluate the role of Mystery Shoppers in Delivering Quality Service

Introduction The use of participant observation, also called situation research and mystery-shopping, where the researcher interacts with the subject or subjects being observed stems from the field of cultural anthropology (Wilson and Gutmann, 1998). Anthropologists would take part in a tribe's daily life in order to understand the norms, attitudes and behaviours that were neither documented nor communicable via language. Anthropology tended to use unstructured observation with researchers immersing themselves in a situation and discovering 'reality'. Covert participant observation, where the observation is undertaken in a concealed manner to ensure that the experience is natural and not contrived for the sake of the observer, has also been used in the social sciences, for example in the study of race and sex discrimination in recruitment procedures (Esmail & Everington, 1993). Additionally, the Consumers' Association in the UK have been using covert participant observation in their tests of service performance since 1960 (Brown, et al., 1997). Even Government Departments have used the technique to monitor areas such as postal deliveries or the testing undertaken in MOT test centres (garages which test the roadworthiness of cars, COT, (1996)). In a services context, observational methods are able to provide information of the service experience as it unfolds (Grove and Fisk, 1992). In particular, participant observation helps to develop a richer knowledge of the experimental nature of services. The participant can identify dimensions of the service encounter unlikely to be discerned by a distant or non-participant observer. Concealment of this observation, although raising ethical issues can ensure that the experience is natural and not contrived for the sake of the observer. The need for specific performance information stems from the increasing emphasis on service performance by service mangers. Whilst head office staff and senior management invariably set service standards, the task of delivering these standards falls to individual staff member who face customers. Variations in service performance can have a major impact on customer satisfaction. This is emphasised by Bateson (1995) who suggested that the customer's experience during the delivery of a service is as important to customer satisfaction as is the benefit that the service provides. Grnroos, (1983) in his paper on 'interactive marketing' also stressed that 'how' you perform a service is often as important as 'what' you perform. Emerging academic and other literature shows that mystery-shopping measurements tend to be used

for one or more of the following purposes: 1. To act as a diagnostic tool identifying failings and weak points in an organisation's service delivery (Wilson, 1998; Wilson and Gutmann, 1998). As a diagnostic tool, respondents use mystery-shopping to track key elements of their organisation's service delivery process and help identify where capital, technical and human resource requirements to be allocated. 2. To encourage, develop and motivate service teams by linking with appraisal, training and reward mechanisms (Dorman, 1994). Definitive information from mystery-shopping is also seen as being essential for appraising, developing, motivating and rewarding service teams and potentially individual service personnel. In terms of motivation, mystery-shopping results can be used to reward those service teams who are performing well against the set service standards. Financial rewards and incentives are seen as becoming more common in almost all sectors but particularly in the financial services sector (Wilson,1998; Dorman, 1994) 3. To assess the competitiveness of an organisation's service provision by benchmarking it against the offerings of others in an industry (Wilson, 1998; Wilson and Gutmann, 1998). Mystery shopping can also be used as a bench mark comparison with the service performance of competitors. Where the research is undertaken on a syndicated basis, this is relatively straightforward but where an organisation commissions a mystery-shopping programme which includes an assessment of the competitors, certain ethical issues become important (Dawson and Hillier, 1995). The Market Research Society (MRS) Code of Conduct specifies that mystery-shopping should not involve an unreasonable amount of time or expense on behalf of the organisation being researched. Participant observation has tended to be used by each of these bodies as it allows the researcher to overcome some of the potential weaknesses of interviewing and survey research. These are highlighted by Fried.richs & Ludtke (1975). 1. There is often a discrepancy between real and reported behaviour. Occasionally

statements are made in interviews which are not in accordance with the factual behaviour of the interviewed persons. 2. Often facts are brought to light by means of natural settings only. The interviewee is

not conscious of them and they are therefore not easy to get at by questioning. 3. The verbal capabilities of the interviewed person can limit the quality and quantity of information gathered.

Comparative Approaches Mystery shopping became established during the 1 970s, with approximately 25 percent to 35 percent of all banks with over $300 million in deposits conducting some type of mystery-shopping program (Leeds, 1995). What prompted the growing interest in mystery-shopping in the '70s was the realization by marketers and bankers of the importance of developing a sales culture (Reed and Miles, 1995). Because sales professionalism became increasingly important, a device had to be developed to monitor sales skills, as well as changes in service behaviours in the sales culture. Mystery shopping began to be used as a monitoring device for sales culture development, specifically for tracking sales behaviours and skills (Leeds, 1995; Reed and Miles, 1995). This phenomenon then led to the use of mystery-shopping to not only monitor, but to motivate performance, set goals or standards and reward performance. Some of the more progressive and sales-oriented banks began rewarding employees based upon the sales achieved (Dorman, 1994). In the 1980s, the financial industry's new catch phrase was "service quality" and, once again, mystery-shopping (along with consumer and customer satisfaction surveys) was used extensively for evaluating, monitoring and motivating performance. It was the combination of these two research methodologies that changed the basic mystery-shopping methodology to one of a predictor of customer satisfaction. By determining what customers' wants and needs are and what satisfies customers most, checking for and reinforcing specific behaviours can be built into the mystery shopper program (Leeds, 1995; Dwek, 1996) While the proponents of the methodology were deriving benefit from the mystery shopping technique, it was not without critics. Dwek (1996) states that mystery-shopping used to be regarded as corporate spying. It was seen as an underhand activity, probably prompted by management accountants advising clients to axe hundreds or thousands of jobs. It was, in short, an excuse for firing people (Dwek, 1996) In a services context, observational methods are able to provide information of the service experience as it unfolds (Grove and Fisk, 1992). In particular, participant observation helps to develop a richer knowledge of the experimental nature of services. The participant can identify dimensions of the service encounter unlikely to be discerned by a distant or non-participant observer. Concealment of

this observation, although raising ethical issues can ensure that the experience is natural and not contrived for the sake of the observer. In terms of ethics, observing people without their knowledge may violate their rights to privacy and freedom from exploitation (Jorgensen, 1989). However, services are often performed in public settings where their delivery can often be observed by members of the public other than the specific obligation to the service provider in a concealed observation than in normal everyday life circumstances. Mystery shopping is a clear example of concealed participant observation in a public setting. However, it differs from original anthropological approach to observation in terms of its structured and systematic format. Cultural anthropology tended to use unstructured observation with researchers immersing themselves in a situation and discovering 'reality'. Whereas mystery-shopping uses a structured approach of checklists and codes to gather and measure specific information about service performance in everyday conditions (Grove and Fisk, 1992). The need for specific performance information stems from the increasing emphasis on service performance by service mangers. Whilst head office staff and senior management invariably set service standards, the task of delivering these standards falls to individual staff member who face customers. Variations in service performance can have a major impact on customer satisfaction. This is emphasised by Bateson (1995) who suggested that the customer's experience during the delivery of a service is as important to customer satisfaction as is the benefit that the service provides. Grnroos, (1983) in his paper on 'interactive marketing' also stressed that 'how' you perform a service is often as important as 'what' you perform. Theories and Models

In the UK, mystery-shopping is the smallest area of specialisation covered by major research companies (Dawson and Hillier, 1995). However, in a survey conducted by the MRS in 1995, it was found that the use of mystery-shopping as a research methodology is gaining widespread acceptance (Dawson and Hillier, 1995). There are few published figures on the scale of mystery-shopping

activity, Wilson (1998) gave an estimate of between 20-3Omillion per annum and Dawson and Hillier (1995) identified 187 agencies providing this form of research. Apart from using mysteryshopping as a part of performance monitoring programmes, competitor mystery-shopping is relatively widespread (Dawson and Hillier, 1995). However there are also a number of areas that lead to tension among companies with regard to competitor mystery-shopping. In a survey carried out by the MRS in 1995, the following were cited by the respondents to be problematic issues in competitor mystery shopping: mystery shoppers take up too much time, should not do anything out of ordinary, should make a purchase, it is ethically wrong to place orders and then cancel, the taping of conversations is not acceptable unless prior permission is gained, etc. (Dawson and Hillier, 1995). As a response to these problems, and also to reduce the costs of research for individual organisations, syndicated studies involving a number of clients are also in operation. The syndicated studies enable the client organisation to assess their performance relative to industry norms and comprise groups of organisations. The financial services in particular have been engaged in a syndicated shopping since 1992. Subscribers such as the Midland, NatWest, Royal Bank of Scotland, Bank of Scotland, Bradford and Bingley, Nationwide, Yorkshire Building Society and Leeds Permanent get comparisons on their own and competitors' performances on a whole range of services (Miles, 1993). Mystery Shopping is a technique increasingly used in service organizations to measure intangible service experiences. Since a service experience exists only as a memory in the person who had the experience, measuring how good it was is a challenge for service organizations. Instead of being able to rely on the quality inspectors that manufacturing rrms use to ensure product standards are met, services must invent other measurement techniques to assess the quality of their customers service experience. Of the many techniques service organizations might use, one of the most useful is the mystery shopper. Mystery shoppers go to an organization to experience a service incognito. They act as a typical customer to assess predetermined service standards of service quality are being met by employees delivering the service. The mystery shopper visits a specic organization at a predetermined time and then completes a systematic assessment form detailing all aspects of that service experience. Rather than relying on unpredictable customers willingness to ll out comment cards, their vague memories when responding to random surveys, or a supervisors chance opportunities to observe customer-employee interactions, a mystery shopper can catch the experience while it is happening. Whether it is a bank, health care facility, restaurant, hotel or any other organization providing an intangible service experience, a mystery shopper provides an assessment of both objective and subjective aspects of the

quality and value of a service. The assessment forms are designed to measure an organizations specic service standards. Because of their comparatively high cost, mystery shoppers typically visit organizational locations infrequently. Mystery shopper programs using only one visit per unit per year spread across several hundred organizational locations as would be typical for a bank,restaurant, or hotel, provide management with an overall assessment of the organizations ability to meet its quality standards. Analysing these data over time allows managers to discover whether service quality delivery is improving or not for the overall organization. Because of this limited frequency of shopper visits, the data are generally considered useful only for assessing organizational level performance and not for individual units or a specic employee. Benets of Frequent Mystery Shops Advantages of mystery-shopping can be summarised as follows: 1. In contrast to attitudinal surveys of customers, the mystery-shopping approach is being used to measure the process rather than the outcomes of a service encounter (Wilson, 1998). The emphasis is on the service as it unfolds, looking at which activities and procedures do or do not happen rather than gathering opinion about the service experience (Miles, 1993). 2. The literature review reveals that consumers are more interested in functional qualities, as opposed to technical qualities. Hence mystery-shopping is a very suitable methodology as it provides more objective measures of service delivery.

Weaknesses of mystery-shopping in Internet banking services can be summarised as follows. 1. The entire outcome of the exercise depends on the quality of the data collection instrument. Careful design is required to give comprehensive coverage. The omission of a crucial factor may yield spurious results. 2. Because the methodology does not solicit views from a sample of respondents, it lacks the diversity of open-ended answers and suggestions.

3. Inevitably, some measures in a mystery-shopping exercise produce subjective evaluations. While it is impossible to remove the subjective element, it can be reduced by formulating appropriate measures that have objective underpinnings. Because of the data limitations of the typical mystery shopper program, some organizations have increased their frequency to gain better feedback. Cheesecake Factory and Restaurant Partners Incorporated, for example, employ mystery shoppers a minimum of ten times monthly for each location. These data offer several important advantages for organizations willing to spend the money to obtain them. First, mystery shoppers can make a relatively objective assessment of all aspects of the service experience, especially an employees performance in the customer encounter. Because the mystery shopper is asked to assess the experience against preset service standards, a related benet is that using a mystery shopper requires organizations to take the time to identify exactly what they expect to deliver to customers in their service offering. There is considerable value for the organization in dening what the service experience should be for its customers by identifying the aspects of the service experience that are key to its customers satisfaction, repatronage, and positive word of mouth. These key drivers can be used to ensure that the service standards assess delivery of the service experience expected by the customer. Second, once service standards are dened, a mystery shopper can use a customized form to assess the organizations effectiveness. Every shopper uses the same measures for every shop. These data also allow the organization to make performance comparisons across time periods, employees, and organizational units. While mystery shoppers are effective in assessing the more subjective standards of service, such as employee friendliness or helpfulness, they are especially effective in evaluating the objective aspects of the experience, such as the cleanliness of the facility or the number of seconds before being greeted at a particular location. A third, and perhaps the most important but underutilized, advantage of this technique is that it provides third-party data for managers to use in coaching their employees. Feedback from a mystery shopper provides a relatively objective basis for a manager to praise good and correct poor performance. between the employee and the assessor are minimized, as both are strangers to one another. And, the employee is unaware that his or her performance is being formally assessed. A large private school group, for example, uses mystery phone calls to individual school ofces to reveal how enrolment inquiries are handled. An international resort organization uses mystery audits of travel agents to learn how often agents recommend their properties. In each case, the feedback is used to coach individuals about their performance.

A fourth advantage of more frequent use of mystery shoppers is that the data they provide remove much of the perceived subjectivity of the appraisal process. Because mystery shop feedback is provided anonymously on the performance of a particular server, mystery shop data are less subject to traditional appraisal errors made by supervisors than the typical performance review process. The manager can offer employees specic feedback anchored to specic performance standards without appearing to be making ad hominen attacks. Fifth, most mystery shop programs can provide rapid, usually overnight, feedback. This information can be shared with an employee while the customer encounter assessed by the mystery shopper is still fresh in the employees mind. A sixth benet of using a mystery shopper is based on the proven psychological value of knowledge of ones performance. Knowledge of results almost always leads to improved employee performance because it improves learning, as well as motivation through subsequent goal setting. Finally, because employees do not know when their performance will be assessed by a mystery shopper, the feedback provided allows managers to coach on a variable interval (VI) schedule. Since employees, cant predict when or who is assessing their performance, they cant engage in impression management by dressing up their performance for an evaluator. While employees know that both the performance observation and the coaching feedback will happen sometime, they dont know when. Feedback is expected but unpredictable. Mystery shopper programs allow managers to gain the power of variable interval schedules in ways that have not previously been possible outside the laboratory. Weaknesses of Mystery-Shopping Weaknesses of mystery-shopping in can be summarised as follows; 1. The entire outcome of the exercise depends on the quality of the data collection instrument. Careful design is required to give comprehensive coverage. The omission of a crucial factor may yield spurious results. 2. Because the methodology does not solicit views from a sample of respondents, it lacks the diversity of open-ended answers and suggestions. 3. Inevitably, some measures in a mystery-shopping exercise produce subjective evaluations. While it is impossible to remove the subjective element, it can be reduced by formulating appropriate measures that have objective underpinnings.

Conclusion A mystery shopper intervention accomplishes at least three important aspects of any performance appraisal system. First, it requires the specication of the standards of employee behaviour in clear and measurable terms. Second, it provides performance data that can be used to coach employees. Third, the performance assessments are based on observations from a relatively dispassionate third party. This means that the conversation in a coaching session starts off with the comment you were evaluated, and here are the results rather than here is what I think of your performance. It is less stressful in an evaluation process when the data used come from a neutral source. This enables the manager to focus on the implications of the feedback, and coach improvements in performance instead of focusing on the impact the feedback might have on the emotions and of the person being evaluated. It is impractical to constantly coach all employees on desired behaviour. Hence, the obvious lesson learned from our research is that mystery shopping is a powerful technique for managers to use in providing employee third-party feedback on a variable interval schedule in a service organization.

Bibliography and References Bateson, J. (1977), Do We Need Service Marketing?, Marketing Consumer Services: New Insights, Report 77-115, Marketing Science Institute, Boston Brown, C., Sopp, L. and Gould, 5. (1997), Consumers' Association Goes Undercover, Consumer Policy Review, 7(1), pp. 2-6 Dawson, J. and Hilliar, J. (1995), Competitor Mystery Shopping: Methodological Considerations and Implications for the MRS Code of Conduct, Journal of the Market Research Society, (October), pp. 4

17-427 Dorman, K. G. (1994), Results Can Shape Your Future, Bank Marketing, (August), pp. 17-21 Dwek, R. (1996), Magic of Mystery Shopping, Marketing, 17(October), pp. 41-44 Esmail, A. and Everington, S. (1993), Racial Discrimination Against Doctors From Ethnic Minorities, British Medical Journal, 306, 13(March), pp. 69 1-692 Friedrichs, J. and Ludtke, H. (1975), Participant Observation, Lexington, MA, USA Grnroos, C. (1993), Toward a Third Phase in Service Quality Research: Challenges and Future Directions, in Teresa A. Swartz, David E. Bowen, and Stephen W. Brown (Eds.), Advances in Services Marketing and Management: Research and Practice, Vol. 2, pp. 49-64, JAI Press, Greenwich, CT Grove, S.J. and Fisk, R. (1992), Observational Data Collection Methods for Services Marketing: An Overview, Journal of the Academy of Marketing Science, 20(3), pp. 217-24 Jorgensen, D. L. (1989), Participant Observation: A Methodology for Human Studies, Sage Publications, Newbury Park, CA Leeds, B. (1995), Mystery Shopping: From Novelty to Necessity, Bank Marketing, (June), pp. 17-23 Reed, K. and Miles, L.R. (1995), How Mystery Shoppers Support 'The Brenton Way' of Creating and Defining a Service Culture, Journal of Retail Banking Services, Winter, 17(4), pp. 21-28 Wilson, A and Gutman, J. (1998), Public Transport: The Role of Mystery Shopping in Investment Decisions, Journal of the Market Research Society, Oct. 40(4), pp.285-95 Wilson, A. (1998), The Use of Mystery Shopping in the Measurement of Service Delivery, Service Industries Journal, 18(3), pp. 148-163