Example Material: Evaluation Instruments for Events

Explore two different evaluation surveys designed to be used across multiple events and then take a moment today to draft your own.

Katherine Hollins

September 25, 2019

Evaluation is key to improving outreach and programming, and to communicating the value of your work. And while evaluation is important throughout the process of landowner engagement, one aspect that is relatively simple to obtain and that can be particularly useful for providing insights is event evaluation. It is even more useful if you have the same or similar events repeatedly.

Below are two evaluation templates that were developed to be used across events, and which can serve as a useful starting point for your own event evaluation.

Learn-and-Burn Events

Full Survey Instrument developed by Jennifer Fawcett and Laurel Kays, NC State University Extension, in collaboration with the TELE team.

Goals

The creators’ main goal was to develop an evaluation instrument that could be used throughout the region to assess Learn-and-Burn events hosted by different partners, so the events could be compared, even if they weren’t exactly the same. They also wanted to assess the learning of the participants and any changes in (intended) actions. Finally, they wanted to get a sense of barriers to action so they could pursue opportunities to reduce those barriers in the future.

Highlights

  • The creators focused on questions that would be applicable across different events to allow them to identify themes and track trends across programs and over time. They maintain metadata about events so they can compare half-day versus full-day events, or events that include demonstrations versus those that don’t.
  • Using a template across partners’ events makes it easier to process the data and produce reports that are helpful to other local organizers, and it makes it easy for folks who are less familiar evaluation.
  • Reframing and reorganizing some questions made it easier for the event participants to understand and answer them. The organizers now get fewer skipped questions, and fewer folks asking, “What do you mean by this?”

 

Women Owning Woodland (WOW) Events

Full Survey Template developed by Dr. Emily Huff of Michigan State University, in collaboration with the WOWnetPro community.

Goals

The creator’s main goal was to develop an evaluation instrument that could be used nationwide to assess events hosted by any agency, organization, or individual working to engage women in land conservation and stewardship. Aggregating data in this way will help to bring attention to best practices and identify areas for improvement, as well as show the impact of the work being done to engage this audience.

Highlights

  • The evaluation is relatively simple and is intended to be personalized to meet the needs of the specific event it is being used for.
  • The survey reflects questions posed by the WOW community of natural resource professionals as those that would help them improve their programming.
  • Metadata about the different events, such as length, type of activities, topics covered, and whether or not any of the speakers were women, will allow comparisons nationwide. Insights will be shared with the community of practitioners who submitted their own data.  (Interested in participating in this effort? More info on the WOWnet website under “Evaluation Resources”)

 

Ready to Get Started?

 

Use the examples above and the tips below to take a moment today to draft an evaluation template for you and your organization so you can figure out what’s really working with your events. Here are a few things to keep in mind.

Tip #1 - Focus on what you really need to know.

While, “it would be interesting to know” can feel like a good reason to ask a question, if you don’t have plans (or the ability) to do something with the information you gather, you should cut the question. Each “extra” question you keep takes up the valuable time of your participants and may mean they don’t answer another question that is more practically useful to you. Think about your learning goals and let those goals guide your survey.

For example, the following two questions are interesting, but were cut from the Learn-and-Burn survey since they did not contribute to the ability of the hots to assess the event or plan for future programming.

Tip #2 - Make your survey intuitive to complete.

Once you develop your survey instrument, run through it and make sure your scales are consistent across questions, the format is easy to read, and you’re using clear language. Most folks aren’t going to spend extra time to understand a confusing question and they won’t pay attention if you’re using a different scale later in the survey. Some easy changes made these questions much more clear in the Learn-and-Burn survey.

Below, simplifying the language in the column headers reduced clutter and helped participants quickly understand their choices.

Before

After

Changing the orientation of the question below made it easier to read, and consistent with the orientation of questions in the rest of the survey, making it much more intuitive for the reader.

Before

After

Tip #3 - Simplify

The example below showcases a number of ways to simplify a question. They have reduced the overall text and put the question at the end, eliminating the need for a reader to go back and remind themselves what they’re supposed to be answering. Finally, the range of answers is easier to parse because they are aligned in a more intuitive way.

Before

After

Tip #4 - One question per question, one answer per answer

It’s easy to accidentally let more than one idea sneak into a question or answer choice, but if you ask yourself “What will I know if they answer this Yes (or A, etc.)?” you’ll get an idea of some questions that need to be clarified or divided.

In the below example, pulled from our very own TELE workshop evaluation, you will see that if someone answers No I won’t know if it’s because the project wasn’t real or if the project was real, but the respondent will not continue to work on it. In this instance, the benefit of getting more clarity did not outweigh the cost of another question, but it’s important to consciously make those choices.

For more discussion of why and how to do evaluation, check out Chapter 8 of the Landowner Engagement Guide. Or, if you’re interested in diving into the world of survey best-practices, you may enjoy Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method by Dillman, Smyth, and Christian.

Add new comment