Module 3 - Unit 6: Client Data & Program Evaluation
What is program evaluation?
Program evaluation is systematic way of assessing:
- What is done in the program
- How it is done, and
- How effective the program is
- According to the CDC, program evaluation also helps contribute to continuous program improvement
Why is program evaluation important?
According to The W.K. Kellogg Foundation, evaluation is important because it:
- Strengthens your program – this can lead to more dollars and resources for the program.
- Can be designed to address real issues – this helps build relationships among your constituents.
- Builds the knowledge and skills of staff and participants – this can lead to sustainable resources and staff.
Considerations for Program Evaluation
Program evaluation takes time and money. Some programs have a dedicated staff member who can work on program evaluation. Some programs hire a third party evaluator such as a university or private firm.
There are many resources available to complete a program evaluation internally. Click the link for the 2017 W.K. Kellogg Step-by-Step Guide for Evaluation.
Strong evaluations should ideally start at the same time an initiative begins. A good way to plan and think about programs, projects, and evaluation is to use a logic model and theory of change.
Logic Model
A logic model is a picture of how your program works. It is a systematic way of identifying:
- What resources are invested into a program.
- What activities and services are provided and who they are provided to.
- What is the result of the activities and services.
- What the expected outcomes are including short term, mid-range, and long term impact.
Theory of Change
Every program, project, or initiative has a purpose. It was created for a reason. Describing that reason can be difficult, but being able to describe it well makes a big difference to funders/investors.
According to W.K. Kellogg Foundation, “a theory of change is a narrative that explains the links between program strategies or activities and outcomes, and how and why the desired change is expected to come about” (p. 106; https://wkkf.issuelab.org/resources/27300/27300.pdf).
That narrative may look something like this:
If we do ____________, then ___________ will happen because ___________
Theory of Change + Logic Model
Together the theory of change and the logic model provide a comprehensive description of how your program works. They also provide the tools needed to continuously assess how your program is working and if it is working well.
Click on any of the links below to practice developing your logic model and theory of change.
Theory of Change + Logic Model
Like program evaluation, personal data that is collected from residents should have a purpose.
- What data you decide to collect should be informed by the program evaluation and vice versa.
- The data is often collected and housed within an electronic health record or EHR system.
More on EHR Systems:
- Provides a comprehensive picture of who the person is; what are their goals, strengths, needs, and areas for growth.
- Holds important administrative information such as an intake form, emergency contact, and primary care provider.
- Stores important programmatic information such as fees collected, days attending recovery meetings, progress notes, and infractions.
For program evaluation, these data points can provide useful insight into how the program design improves or impedes resident progress.
Client Data Collection
- Type of data recovery homes might collect
- Basic biographical (name, DOB, relationship status, etc.) and demographic information (race, age, sexual orientation, etc.)
- Health / mental health conditions and medications
- Basic biographical (name, DOB, relationship status, etc.) and demographic information (race, age, sexual orientation, etc.)
- Data safety, privacy, and protection considerations:
- HIPAA
- Client confidentiality
- PHI
- Next, we will provide high level options for collecting and managing data – e.g., an Excel Spreadsheet to an EHR record (example EHR records on the next slide)
Example EHR Tools
Behave
*Disclaimer: CoRH does not endorse either system. There are many EHR systems that could be suitable for your needs. Please be sure to do your research to find a system that works the best for your home.
