Guidance on the use of focus groups for evaluation of public involvement programs at contaminated sediment sites (project report)
Santos, S. Danielson, S, Tuler, S. 2007. Guidance on the use of focus groups for evaluation of public involvement programs at contaminated sediment sites. SERI Research Report. Greenfield, MA: Social and Environmental Research Institute.

Publication Abstract

 This report provides guidance for using focus groups to:
• evaluate public involvement during remediation of contaminated sites;
• conduct both formative and summative evaluations; and
• conduct both process and outcome evaluations.

Why evaluate public involvement?
On June 6, 2003, EPA issued its new Public Involvement Policy. The Policy’s overall goal is for excellent public involvement to become an integral part of EPA’s culture, thus supporting more effective Agency actions. The Policy outlines seven steps to effective involvement, including evaluation.
Evaluations of public involvement help improve how we define, measure, and conduct public involvement programs. Evaluating how well an involvement activity or process worked can help you make those processes more effective for EPA and participants.
Specifically evaluation helps to identify:
1) needs and expectations for remediation or public involvement from public involvement participants;
2) strengths and weaknesses of EPA involvement activities;
3) ideas or suggestions for improvement of EPA involvement activities from people who participate in them;
4) barriers to achieving improved public involvement processes and activities;
5) changes in what and how we measure the success of public involvement activities and processes; and
6) common performance measures that can be used to fulfill requirements of the Government Performance and Results Act of 1993.

When to evaluate?
Evaluation is often thought of as something to do at the end of a project to gauge whether goals were met or whether the community involvement process was “successful.” This is called summative evaluation. Summative evaluations help to validate the effort and can be particularly important for transferring “lessons learned” to new sites.
Evaluations can also be done during a project. If the results of evaluation done during a project are used to revise the community involvement process or the project itself, then this is called formative evaluation. Formative evaluation gives you feedback throughout your process so that you can make changes and improve as you go along. This feedback can be used during all phases of work at sites requiring remediation by agency staff and stakeholders to improve public involvement efforts and positively influence clean-up efforts.

What to evaluate?
You can evaluate the results of a public involvement process, including things such as: participant satisfaction with outcomes, level of understanding about remediation actions, trust in EPA, etc. This is often referred to as outcome evaluation.
Evaluations may also explore how public involvement activities take place, including things such as: who is involved in a process, adequacy of meeting notifications, access to EPA staff, fairness of the facilitator, etc. This is often referred to as process evaluation.

Publication File