In a presentation at the Social for Social Innovation at Stanford University (the audio file is available here), Dr. Alana Conner argued for fewer and better evaluations of social service agencies. Her argument hinges on the fact that good evaluations are hard to do. She points out the importance (and difficulty) of finding a control group for an agency's clients. That is, a group of individuals who are as similar as possible to the clients but who do not receive services from the agency. Comparing the difference in outcomes between the agency's clients and the control group yields the impact of the agency's work. Rick Aubry of Rubicon Programs, one of her co-presenters at the lecture, goes on to point out that evaluations require a serious commitment at all levels of the agency, as intakes and follow-ups must be designed to collect data for the evaluation.Conner's conclusion, then, is that evaluations should only be done on programs that seem deserving (i.e., programs that clients seem to like or other agencies are interested in copying). While this follows logically from her and her colleagues' arguments that evaluations are difficult, the thesis seems unsatisfying. Even if a program does not seem deserving of an evaluation, how undeserving is it? Could it be improved with just a few tweaks? Is it an agency's worst program (should it be cut) or is it somewhere in the middle?
On the other hand, if we do want to do large numbers of evaluations, we need to address her points and make evaluations easier. How can we do this?
Posts
