3 Ways to Use Evaluation Beyond the APR
3 Ways to Use Evaluation Beyond the APR
The Annual Performance Report (APR) for federally funded grants is ever-present in our minds. Gathering, cleaning, and submitting the APR takes a lot of work and a lot of data! However, once you have all that data what else can you do with it? How else can you evaluate the effectiveness of your program beyond just submitting your APR?
We have three tips for making the most out of your APR data and leveling up your game.
1. Leverage Evaluation Data to Improve Services
One of the most powerful things you can do with your data is to leverage it to improve your services. Most of what is reported on the APR is a simple count of the number of students who participated in your program. This data is great to understand the reach of your program, but it doesn’t tell you how you can improve your services or the overall impact your services have on participants’ lives. Just because students come to one of your services doesn’t mean the service actually made an impact on their lives. A potential way you can move beyond simply counting students is to use data analyses to understand the association between your services and student outcomes.
One way to do this combining participation records and academic data to understand the relationship between your services and students’ academic outcomes.
This can easily be done using program participation records and the academic records provided by your partnering school (school district if you’re a pre-college program or college if you’re a post-secondary program). Try running a simple correlation to see if there is a relationship between the number of services the participant received and their GPA.
If statistics make your heart start to the race, it feels too overwhelming, or you really don’t feel comfortable with this option, you can still leverage evaluation data to improve your program services. Another approach could be to implement a survey that asks students what types of professional development workshops they need or want to help support their educational pursuits. This qualitative feedback can provide critical insight into how your program can support your students and is data that is responsive to their needs.
2. Engage Local Partners
Evaluation can demonstrate the value your program provides to various partners and contributors, and there are many ways you can share your findings. The key to effectively engaging your partners is to develop dissemination materials that document the key findings most relevant to their position.
For example, the superintendent of a large public school district that has a TRIO Talent Search Program does not have time to read a long technical evaluation report. They might just be interested in things like:
You can quickly and effectively highlight your program’s impact in a 1-page evaluation summary. This brief evaluation report should highlight the successes of your program including the student achievement and engagement from your participants who attend school within that district.
For partners that need to know more about the technicalities of your program, you can develop an annual evaluation report that documents the implementation process of your program as well as the outcomes that were achieved.
Depending on your audience you may want to consider either a traditional written evaluation report or perhaps a facilitated dialogue with partners would be more beneficial for the target audience.
Don’t forget that your participants and families of your program are also partners who want to know about the impact of your program! To engage this audience, you might consider weaving evaluation data into your social media outlets. Well-designed social media posts can extend the reach of your data and showcase the impact of your services with a wide-reaching audience.
3. Advocate for Program by Sharing Your Evaluation Findings
The third outlet we encourage you to pursue is scholarly publications and blog posts. Both outlets can serve as a source to help you document the services that your program provides and the student successes that follow.
The focus on programming and student success are great ways to provide evidence that our college access programs are effective. Publishing your evaluation findings also provides other grantees with an up-to-date reference that can be used for future grants. This is becoming an increasingly important endeavor as the US Department of Education expects grantees to use evidence to inform the development of their college access programs.
Let’s make sure our evaluation findings are at the center of this call to action from the U.S. Department of Education and we are getting our data published in peer-reviewed journals, so we can provide the required evidence that our college access programs are effective!
The bottom line
If you want your program to excel beyond the APR, you must make it a goal of your program to leverage evaluation beyond the basic data elements required on the APR. The data you collect for your program should help you generate insight into what’s working and what’s not working.
Taking this approach to evaluation will help you plan services for future years, and help you modify your programming to best meet the needs of your students and meet grant objectives!
Contributed By Sabrina Gregersen
Sabrina Gregerson, Ph.D., is a former TRIO McNair Scholar at the University of Kansas and previously worked with the Research, Evaluation & Dissemination Department in the Center for Educational Opportunity Programs.
Contributed By Meghan Ecker-Lyster
Meghan Ecker-Lyster, Ph.D., is the director of Research, Evaluation & Dissemination for the University of Kansas Center for Educational Opportunity Programs (CEOP). She currently oversees and manages the evaluation portfolio for CEOP’s federally funded college access programs, as well as the external evaluator for other equity-focused programs, including GEAR UP, TRIO, and other educational access programs.
Follow @CEOPmedia on Twitter to learn more about how our Research, Evaluation, and Dissemination team leverages data and strategic dissemination to improve program outcomes while improving the visibility of college access programs.