Using Focus Groups in Education Research and Evaluation Practices

Nadia Kardash

Using Focus Groups in Education Research and Evaluation Practices

Evaluation of educational access programs is a complex task that utilizes many research tools and techniques and involves all project partners. Throughout the project, implementation evaluators collect and analyze quantitative and qualitative data from multiple sources.

For example, long-term federally funded educational programs, such as GEAR UP or TRIO programs, use the following data sources:

Such a combination of the data sources helps to draw a clearer picture of the impact and the effects of the project activities on outcomes.

To better understand the needs, experiences, or perceptions of the target groups like students and their families, evaluators need to know what they think about a specific activity or what they feel about a certain issue.

One of the effective ways to get responses directly from the project partners (e.g., students, teachers, parents, staff, administrators) is to organize focus group discussions. The main purpose of a focus group is to get feedback and elicit information on the topic of interest. Evaluators value this practice because it allows for direct personal interactions that encourage sincere discussions with the participants and helps to better understand the situation.

In this post, you will find some hands-on tips drawn from our evaluation practices.

How to prepare

  • Clearly state the purpose of the focus group discussion.
  • Identify the individuals who will be involved in organizing and conducting focus groups and define the roles.
    • Depending on the number of participants (the recommended number is 6 – 10) appoint who is going to be a moderator and a note-taker. It is very important to have someone to sit in the back of the room and monitor the group dynamic, reactions or gestures to help interpret the responses. Such notes and a diagram of the room and seating arrangements are important to have when you later analyze the recorded responses.
  • Prepare a semi-structured protocol draft of the questions you are going to ask beginning with more general questions followed by more specific and detailed options.
    • It is important that questions are open-ended to create extended answers rather than a “yes” or “no” response. It is important to think of some follow-up or probing questions for each initial question in case the responses are too short or too general. When working in a group of several evaluators it is recommended to give it to several people for editing and suggestions.
  • Make sure that the place where the focus group will be conducted is comfortable and can accommodate all participants.
    • Carefully think of the room arrangement (especially in the current conditions and pandemic-related requirements). When selecting a room, think of the place where your participants would feel comfortable. For example, if you have a group of students, try not to have your focus group meeting in an administrative office where the students might feel intimidated. Check for any accessibility needs and if your budget allows, provide some food/drinks when it is appropriate. Prepare and check the equipment especially if you are going to record or videotape the conversation. Have nameplates for each participant.

How to conduct

  • Introduce your evaluation team and establish rapport with your focus group participants.
    • This introductory conversation sets the stage for the dialog and helps the participants feel welcome and stay engaged. Adjust your language for the age or other group characteristics.
  • If you are going to record the conversation, explain why it is necessary and ask for everyone’s consent. You can get written permission using a form or just ask if the participants agree to participate and be recorded.
  • Follow your protocol but allow enough flexibility for the participants to deviate from the question to express an opinion and give examples.
    • It is necessary to give enough time for each group member to answer the questions. Sometimes one or two individuals dominate the conversation, so the moderator should be able to time the responses and allow everyone in the group to express their views and opinions.
  • Give a short summary at the end and thank the group for their time and participation.
    • It is also important to explain why you conduct this focus group and how the results will be shared with the group or other project partners. Emphasize the anonymity of the participants when summarizing the data and quoting some responses.

How to analyze

  • From my personal experience, I recommend doing the analysis as soon as possible after finishing the focus group discussion when all details are fresh in memory. The discussion and sharing of opinions between the evaluation team members right after the conversation also help to contextualize the responses.
  • Depending on the length and complexity of the discussion evaluators decide if they need to transcribe the recording.
    • To analyze such qualitative data, it is helpful to write a summary of the responses for each question and choose the most illustrative quotes. When using qualitative software (e.g., Dedoose) the analyst has a variety of embedded tools to code the responses and organize in any way to fit their needs.
    • The final step is preparing a summary or a more formal report summarizing the results including charts, graphs, and other visual means.

Despite the many advantages of interacting with the project partners, there are certain challenges to consider. Preparation for the group discussion, accommodation, and arrangements as well as the analysis of the data is time-consuming. It is also harder to interpret, analyze and summarize the qualitative responses.

However, the benefits outweigh those challenges. Conducting focus groups as part of the project evaluation adds more in-depth meaning to the quantitative data and helps to better understand how the project partners’ attitudes or perceptions change over time and what impact the project activities have on its outcomes.

Diagonal blue lines as a separator

Contributed By Nadia Kardash

Nadzeya (Nadia) Kardash, Ph.D., is an Associate Researcher with the Research, Evaluation & Dissemination Department in the Center for Educational Opportunity Programs. She currently conducts research and evaluation of the CEOP’s federally funded college access programs including GEAR UP, TRIO, and other college access programs. 

Follow @CEOPmedia on Twitter to learn more about how our Research, Evaluation, and Dissemination team leverages data and strategic dissemination to improve program outcomes while improving the visibility of college access programs.