How can organizations that rely upon volunteers be assured that the quality of tours given tor the public is of an appropriate caliber? How can tour evaluations be accomplished with a limited staff? How will docents react; and how will their concerns and anxieties be addressed?
In the summer of 1996, the docent coordinator of the Freer Gallery of Art and the Arthur M. Sackler Gallery raised the issue of evaluating docent tours. The docents at the gallery had never been systematically evaluated, and the questions stated above were of deep and immediate concern.
As in most institutions, staff was limited and each staff member had heavy a workload. Could peer evaluators be the answer? Docents were skeptical about the idea of evaluation, feeling it was unnecessary and disruptive. Would evaluations be lair? Would people be “weeded out” by evaluation? Would another docent be qualified to judge a peer? Evaluation by a staff member seemed bad enough, but a peer evaluation could be much worse.
With both staff and docent issues in mind, a collaborative approach was proposed. Roca Harding, the docent coordinator, organized a project team made up of two staff members (herself and the tour scheduler) and eight docents. The team, in a series of monthly meetings over nine months, worked together to research and develop a tour evaluation system. After researching twelve institutions nationwide, we found that many had an evaluation process on paper, but never found the time to implement it without the use of peer reviewers.
Our project team agreed on some major features of the evaluation process.
- There would be a mixed system of tour review with both staff and docent evaluators.
- Only staff familiar with both the collections and touring techniques would review tours (i.e. – the docent coordinator, but not the tour scheduler).
- There would be an evaluation of an entire adult tour with museum visitors. It would not be a tour for fellow docents or a 15-minute tour segment.
- Docents would be allowed to select the type of tour to be evaluated (walk-in, reserved group, thematic tour). School tour evaluations were deferred for a future time.
- The goal was to assess the touring skill of each docent to ensure quality and professionalism.
- This was not a process to “weed-out” anyone, but instead, to identify performance problems that could be corrected with training, enabling each docent to give the best possible tour.
Developing Methods and Materials
Members of the project team spent a great deal of time discussing ideas and issues of tour evaluation methodology and materials development. A “Tour Self- Evaluation Checklist” covering every aspect of touring was developed for docent use. It was not intended as a blueprint for any single tour, but was a reminder of all aspects of tour-giving.
A “Tour Evaluation Form,” to be used by the evaluators, was less detailed than the Checklist, but included major headings such as accuracy of information, group management, touring techniques, tour organization, and communication skills. It provided space for evaluators to write comments rather than to check off items.
The “Tour Evaluation Methodology Sheet” was a thoughtful and useful document that set forth evaluation criteria and rules. This document went a long way toward reassuring docents concerned about this process. One docent commented, “I was against the whole idea of tour evaluation, especially the use of docent peer evaluators. But, when I read your Tour Evaluation Methodology, my fears were laid to rest because you had thought through every contingency.”
What were the concerns of docents that had been addressed?
- All evaluations would be confidential and conducted in a professional and sensitive manner.
- Names of peer reviewers would be solicited from the docent corps, but the docent coordinator would make the final selection based on both touring and interpersonal skills. Before a candidate became a peer reviewer, he or she gave a tour that was evaluated by staff
- A docent could ask for a staff, rather than a peer, reviewer. Alternately, a docent could decline to be evaluated by a particular peer reviewer.
- Each docent was informed of the evaluation date, time, and name of the reviewer at least three weeks in advance.
- If a docent felt there were extenuating circumstances, or if fewer than four visitors were interested in the tour, the docent had the choice of deferring the evaluation to another day. Alternately, the docent could decide to go ahead with the evaluation with a minimum of two visitors. Illness or discomfort on that day was also grounds for rescheduling the review. This was limited to one time.
- If, after the tour was finished, the docent felt that the group dynamics did not go well, or the docent was not satisfied with this particular tour, the docent could inform the evaluator and the tour would not count. The docent would then be rescheduled to give another tour, which would count.
- If a reviewer determined that a re-evaluation was necessary, the docent was to give a second tour for review, incorporating the suggestions recommended by the reviewer. This was to be accomplished within two months of the first tour and would be evaluated by two staff members, the docent coordinator, and another education department staff member.
- If a third and final evaluation was required, the docent coordinator and another education department staff member would review the docent’s performance again. Based on that review, the docent would return to active status or could be offered other volunteer opportunities in the Gallery or within the institution.
Informing and Preparing Volunteers for Evaluation
Docents were informed at every stage of the planning process, and as a result, by the time we were ready to implement the evaluation, docent anxieties had lessened. The docent corps learned about the upcoming evaluation through our docent newsletter, memoranda from staff, docent meetings, and a binder that contained all the pertinent information.
To refresh touring skills and build confidence, docents were given the opportunity to attend an optional Evaluation Training Workshop that focused on the major components, of a successful tour. A large majority of the docents attended this workshop and found it useful, not only in reinforcing skills, but also in reducing anxiety.
Preparing the Peer Reviewers
By contrast, docent peer evaluators were required to attend a Coaching Tips Workshop on Assessment and Interpersonal Skills. This two-day workshop covered topics such as: making docents feel at ease; how to be a good listener; how to offer suggestions positively; how to deflect negativism; how to take notes discretely; and how to handle potential problems. Reviewers were instructed to pass serious problems on to the docent coordinator.
Implementing the Tour Evaluation
From March through May 1997, a majority of evaluations was completed, with the few remaining evaluations accomplished over the summer. Five peer reviewers and one staff reviewer evaluated forty-three (43) docents. All but one docent completed a successful tour on the first attempt. Staff reviewed the one docent that was asked to give a second tour. Another docent needed some improvement in touring techniques and was asked to work with a touring coach.
In the post-evaluation survey, 91% rated the actual tour evaluation as excellent or very good; 94% felt the objectivity and sensitivity of the evaluators was excellent; and 98% rated the preparatory handouts and workshops, as well as the evaluation methodology, to be excellent or very good.
- Collaboration between staff and docents when developing methodology for evaluation gave everyone a sense of fairness and helped allay fears and anxieties. Both docents and staff gained deeper insights into the problems and issues of concern to the other.
- After some discussion, we decided to use the term “tour evaluation,” rather than “observation,” “assessment,” “appraisal,” or “performance analysis.”
- For any organization planning an evaluation of volunteers, it is important to think through and articulate the objectives, the methodology and the criteria used in evaluating touring skills and to communicate that information well in advance to the volunteers.
- It is useful to prepare for as many contingencies as possible. Lack of visitors or too few visitors, illness, inclement weather — these are some of the things the must be foreseen. An advance call by the evaluator was critical, as often the docent to be reviewed had questions, misunderstandings, or some confusion about the process. The docent could not ask the evaluator to listen to or comment on the tour ahead of time. The docent could ask advice from a touring coach.
- The two-day Coaching Tips Workshop for Peer Evaluators was time well spent. The peer evaluators were taught to be positive and tactful, to hold post-tour discussions in a low-keyed atmosphere, and to focus on no more than two negative points. As a result, docents found the post-tour evaluation discussion and comment form to be helpful, not threatening.
In any project involving volunteers, it is critical to develop a well-thought out project plan and to bring docents “on board.” An entirely staff-directed process of tour evaluation might have been quicker, but could very well have presented serious morale problems. The collaboration of staff and docents in sharing research, exchanging ideas and concerns, developing methodologies, and evaluating tours produced an extremely successful evaluation process.
We feel that there are three keys to the success of our evaluation process:
- a collaborative and cooperative approach,
- an effective and sensitive project manager, and
- the motivation and dedication of all those involved.
Roca L. Harding has been docent Coordinator for the Freer Gallery of Art and the Arthur M. Sackler Gallery of the Smithsonian Institution in Washington, DC since 1992. Ms. Harding has contributed several articles previously to The Docent Educator and has been an advisor to international and American museums and organizations on the utilization and management of docent programs. Ms. Harding can be reached by e-mail at: firstname.lastname@example.org for services as a consultant or for training on topics related to docent issues, touring and communication skills, and volunteer management.
Marlyse Kennedy has been a docent at the Freer Gallery of Art and the Arthur M. Sackler Gallery since 1992. She was chairman of the 1999-2000 Docent Council.
Lois Raphling has been a docent at the Freer Gallery of Art and the Arthur M. Sackler Gallery since 1989. She was chairman of the 1994-1995 Docent Council. Marlyse Kennedy and Lois Raphling presented this topic at the 1999 National Docent Symposium, which was held in Philadelphia.
Harding, Roca L., Marlyse Kennedy & Lois Raphling. “Tour Evaluations–Taking Collaborative Approach to Following Up,” The Docent Educator 10.2 (Winter 2000-01): 14-16.