When doing a Case Management system setup or a data analysis project, we encourage our customers to think about tracking "outcomes" variables. Outcomes are the effects your services (outputs) have on clients. Since many of our customers provide services to low-income individuals and families, we often recommend tracking changes in a client's income. As income changes over time an agency can aggregate those changes and look at their program services versus changes in their clients' poverty statuses.While poverty is a good outcomes variable, on its own it can be imperfect. Let's say we wanted to compare the outcomes of two programs offered by the same agency. One program is a men's shelter, and the other a family shelter. In the family shelter the data tells us the poverty statuses of those clients increased more rapidly, and to greater nominal heights, than in the men's shelter. So is the family shelter program more effective? Not necessarily.
While change in poverty status is a good measure, it has to be seen in context. In our example, the clients in the men's shelter might deal with more chronic issues like mental health and drug addiction. The clients in the family shelter may be more employable and have stronger social skills and networks. Therefore, simply comparing the changes in poverty levels of the two programs would unduly favor the family shelter.
There is no definitive method for how to compare across different types of programs. In some cases you may try to control for the differences between the two client types, alternatively you may apply some type of multiplier that better equates the two variables. Ultimately, the use of data is both art and technique. While it is never a good idea to ignore data in favor of anecdotes, data must always be understood in its context.
(Photo by Heather Brandon)
Posts
