Evaluation: 3 Common Misconceptions

Evaluation often gets a bad rap. It can be seen as stodgy and unemotional, expensive, even irrelevant. This is far from the truth, however. Here are a few common misconceptions debunked.

Misconception 1: Evaluation work is dehumanizing because it just boils people down to numbers.

There is some truth to this; but it is not the whole picture. Human stories are and always will be an important part of the evaluation and reporting processes. They give us a glimpse of what the full, amazing potential of your organization is. However, stories tend to only tell us about the most exceptional cases in an organization’s work. They don’t necessarily tell us what the average experience is. Stories also don’t tell us anything about why you succeeded or, in the times that it happens, why you failed. It is hard to learn from a mistake or replicate a success when we don’t know what went right or wrong.

Good evaluation work fills in the rest of the picture- it helps to tell the whole story. We do this through numbers, yes, that can seem a bit un-human at times. However, a good evaluator can use good data to tell a story far richer than just one person’s life-changing experience. You see, good evaluation doesn’t just look at the average experience or total number of people served. If you have collected the right data, a good evaluator can dig under the surface and start to look at who benefitted most from the program and who benefitted least. They can identify personnel that get more return customers than average or help to spot modules in trainings that resonated versus ones that did not. In short, we can start to see relationships and trends that are impossible for the human observer to spot.

Misconception 2: It is impossible to do evaluation with the kind of work we do.

If you feel that way, it may be because you are thinking of particular kind of evaluation. As Bill Bacon from the Duke Endowment articulates, there are two kinds of evaluation. One type is about proving your organization’s work is effective (impact evaluation) while the other is about improving the quality of the work you do (monitoring evaluation). Impact evaluation is difficult to do in the best of times, costly (10’s of thousands of dollars), and usually reserved for state or nationwide interventions.

The monitoring evaluation that is used to track and improve organization programs any organization can benefit from and is much less costly. This is the kind of evaluation that I focus on here at theGOODevaluator.com. Moreover, it is the chief goal of theGOODevaluator.com to make basic monitoring evaluation affordable for the majority of organizations. I do this by breaking the process down, focussing first on making sure you have good data collection systems in place. You can then plan for the future the necessary time and money to analyze and use that data when it is most convenient for you to do so.

Misconception 3: Evaluation is a waste of money and energy. We have more important things to do.

If it is done poorly, then this is absolutely true. For example, there is no point in asking your constituents about their experience if you are not asking the right questions or the questions are written poorly. The resulting data won’t be reflective of anything meaningful or might not even have accurately captured people’s opinions and experiences. As a result, any decision made based on that data will be made on a lark. In the best case scenario, no harm is done besides the wasted effort. In the worst case, you compromise your program because of bad data.

Done effectively, however, evaluation is fully worth the effort. Every nonprofit- in fact, the whole industry, is heavily resource constrained. In non-economist terms, that means we have no money! We are all trying to maximize the amount of good we can do on very tight budgets and, often times, the problem we are trying to tackle is way beyond our organization’s capacity to fully solve it. While we may set our sights on feeding every hungry child, raising every homeless woman up, saving every cat, or giving every nonprofit affordable evaluation services, we can often only ever really hope to affect a fraction of the problem.

And this is why evaluation is so important: If we are doing it, and doing it right, it helps to make your organization and the industry as a whole more efficient. Maybe the two thousand dollars you spent this year only helps you benefit a handful more people next year (on the same shoestring budget). But when you multiply that out across the thousands of organizations in the state, each organization’s handful slowly transforms into a village, then a town, then a small city. That’s a difference you can measure.

Conclusion

Far from being the stodgy, unemotional, expensive, irrelevant offspring of efficiency nuts, evaluation is, in fact, a powerful tool for telling stories, affordable, and vital to the health and future success of any nonprofit and the sector, as a whole. Click “Hire Me” to set up a time to talk about how theGOODevaluator.com can help your organization can make evaluation a part of its daily life.