The evaluation team uses an approach that emphasizes evaluation activities designed to engage and address the needs of stakeholders. The team works closely with stakeholders to understand their diverse information needs and to engage them in the evaluation. Evaluation planning activities include the development of questions, logic models, priority areas and strategies.
Valid and reliable tools are essential for assessing program delivery and impact. The evaluation team has designed several tools specifically for educational opportunity programs, including a college knowledge survey for parents, a financial literacy assessment for students, an interview protocol for principals and a postsecondary follow-up survey. Dr. Karin Chang has extensive experience in educational measurement and has developed more than 30 tools for local, state and national agencies.
Data Analysis and Reporting
Data analysis should be conducted and presented in a clear and informative way. All evaluation team members have doctoral-level training in statistical methods, including structural equations modeling, hierarchical linear modeling and factor analysis. Additionally, the team has extensive experience translating research findings into actionable data. Current reporting mechanisms include data dashboards, briefs, interactive visualizations, online tracking reports, student profiles and executive reports. Evaluation Team members work closely with clients to ensure that data analysis meets the information needs of their diverse stakeholders and is presented in a clear and timely fashion.
Performance measurement is an accountability system designed to help programs improve their outcomes. Optimal performance measurement involves establishing specific goals and objectives and monitoring progress toward those goals throughout the life of the program. The evaluation team has worked with several federally-funded programs, including GEAR UP, TRIO Upward Bound and 21st CCLC to develop research-based, locally-driven performance benchmarks. The team has also developed real-time monitoring systems and dashboards to facilitate easy review and tracking of performance data.
Increasingly, educational opportunity programs are asked to conduct in-depth research to assess the effectiveness of their projects. There are several design options for effectiveness research, and the Evaluation Team has more than 10 years of experience working with programs to select the best methodology to meet their needs. All Evaluation Team members have master’s-level training in research methods and have implemented effectiveness studies from start to finish.
Current or past work includes:
- Assessment and evaluation of reading programs provided by the Mid-Continent Public Library – multi-site study completed for the Mid-Continent Public Library in 2016
- 2015 report on KC STEM Alliance projects – study completed for the KC STEM Alliance and the Kauffman Foundation in 2015
- College Outcomes and Experiences of Hispanic Development Fund Scholarship Recipients, 1985-2015 – study completed for the Greater Kansas City Community Foundation in 2015
- Elements of High School Performance Associated with College Enrollment and Persistence – study completed for the Kansas City, Kansas, Public Schools in 2015
- Impact study of KU GEAR UP on 9th grade achievement – study completed for the KU GEAR UP program in 2013
- Impact study of Kansas Project LAUNCH on kindergarten readiness – study completed for The Substance Abuse Mental Health Service Administration in 2015
- Longitudinal study of KU GEAR UP Cohort 2 (2002–2009) and Cohort 3 (2004–2011) – study completed for RTI International in 2012