Making Sense of ESSA’s Evidence Standards

Making Sense of ESSA’s Evidence Standards

ESSA...  IES... WWC... Do these acronyms feel more like alphabet soup rather than helpful resources? If so, don’t worry, we are going to break down what each of these acronyms means and why they matter for your college access program.

But first, I want to ask you a few questions:

• How do you decide which activities you are going to provide students who participate in your college access program?
• How confident are you that those selected activities are going to lead to your desired results?
• If someone asked you to provide evidence that backs up the rigor of your activities, what would you show them?

If you are unsure how to answer some of these questions, we can turn to that alphabet soup of acronyms for help!

Ok, let’s dive in and see if we can make sense of these acronyms.

First up, ESSA stands for the Every Student Succeeds Act. This is federal education law that was enacted in 2015 that encourages the use of evidence-based interventions. You may be thinking wait hold up, my program provides activities, not interventions, what does this have to do with my program. Let me explain.

The term intervention is used broadly to refer to educational practices, strategies, activities, policies, or even entire program models. Because college access programs often include a wide variety of services you may have several “interventions” embedded within your program. This means you are likely pulled in a lot of different directions trying to make sure you offer as many different services as possible to help your students.

Given this demand on your time, wouldn’t it be nice if you knew that the activities you were offering were going to make a difference and you weren’t wasting your time providing ineffective services? The process for ensuring you are selecting activities that are likely to lead to desired outcomes starts with consulting the research literature and finding a study that examines a similar strategy that you are hoping to implement with your program.

This is where ESSA comes in to help. ESSA provides guidance to help you evaluate the quality of a research study, so you are selecting studies that are conducted in a way that will give you confidence that you can trust the results. Under ESSA there are four tiers of evidence: Strong, Moderate, Promising, and Demonstrates a Rationale.

Evidence ratings are assigned to a research study based on a variety of factors related to the methodology and analytic approach that was used (e.g., study design, sample size). A full discussion of the intricacies of assigning an ESSA rating to a study is beyond the scope of this blog post. Just know there are a lot of helpful resources available if you want to know more about what is going on behind the scenes. For the sake of our conversation, we are going to keep it super simple.

Tier 1 studies use a randomized controlled trial approach to studying the intervention. Students (or classrooms, schools, etc.) are randomly assigned to a treatment condition where they get the intervention or a control condition where they do not.

Tier 2 studies use an analytic technique to statistically create a matched sample of students, which would be comprised of students who received the intervention and students who did not.

Tier 3 studies apply a correlational design with statistical controls for selection bias to study the effects on an intervention.

Tier 4 is where things start to take a slight turn. Unlike the other tiers of evidence that are focused on the methodology and analytic approach used by a study, Tier 4 is more concerned with the conceptual framework used to develop the intervention.

As the lowest tier in the evidence hierarchy, this tier is simply asking that programs develop a rationale for why their program will produce their desired outcomes, and this rationale is depicted through a logic model.

There is so much more we could say about the process used for each tier of evidence (e.g., how to create matched samples, building a logic model), but those are all topics for another blog. 

At this point if your head is spinning and you are thinking to yourself - how could I possibly figure out if a study meets ESSA standards? I’m not a trained researcher; I don’t know whether I’ve selected an intervention that is evidence-based. Worry not, I have a little secret, I want to share with you. There is an entire website devoted to helping you figure this out!

The website to help you figure out if an intervention is evidence-based is the Institute of Education Sciences (IES) What Works Clearinghouse (WWC)… it’s those pesky acronyms again!

The WWC is great because it literally tells you what the ESSA rating is for an intervention. All you have to do is search for an intervention that you are interested in implementing and the WWC gives you the overview you need to know whether it is a quality study that will produce the outcomes you are looking for with your program.  

Contributed By Meghan Ecker-Lyster

Meghan Ecker-Lyster, Ph.D., is the director of Research, Evaluation & Dissemination for the University of Kansas Center for Educational Opportunity Programs (CEOP). She currently oversees and manages the evaluation portfolio for CEOP’s federally funded college access programs, as well as the external evaluator for other equity-focused programs, including GEAR UP, TRIO, and other educational access programs

Follow @CEOPmedia on Twitter to learn more about how our Research, Evaluation, and Dissemination team leverages data and strategic dissemination to improve program outcomes while improving the visibility of college access programs.