Creating a Data Collection System That Works… for YOU!

Whether you have inherited a system from a previous program or you are starting completely from scratch (or something in-between), creating a data collection system that works for you and your program requires some careful consideration.

Even if you have been provided with an outline or a template (perhaps by your national oversight organization, such as NCCEP or COE), how do you make it really work for you and your program? Federal grant programs are not one-size-fits-all, so their data collection systems should not be either.

Regardless of where you start, asking yourself these five questions will help you develop a data collection system that will reliably collect useful data and satisfy your funders and stakeholders but will not unnecessarily burden your team members, service providers, or clients.

Who?

Who will be responsible for collecting the data? Is it the grant program staff, an outside agency, or some combination of both? What is their capacity to collect the data?

In this instance, capacity includes time, skill level, and even personal interest in data collection and analysis. Certain team members may have more time, more technical knowledge, or even just more interest than other team members in data collection. Those team members can play a larger part in the process, such as being the point-person for gathering data from grant-wide events or helping other team members with data questions.

What?

What data are you going to collect? You might be surprised to know that as an evaluator, I have seen this question not always receive enough consideration. This is particularly true when you inherit a data collection system from a previous grant.

I strongly recommend creating your own list of required data from all possible sources. What, specifically, did you commit to in the grant? What is included in your required annual (and/or other frequency) reports?

• “Academic data” is too broad – do you need GPA? Course enrollment? Standardized test scores?
• “Service data” is similarly vague – how specifically do your services need to be defined/categorized? Do you need to measure services by hours, number of offerings, and/or something else?
• Don’t forget “participant data” as well – do you need information about specific program participants, and if so, what will you use as their unique identifiers? What participant demographic information will you need to collect?

You should develop this list of required data before you layer on any additional items. You may aspire to go above and beyond the specified data collection requirements (conduct surveys/focus groups/interviews that are not required for your grant). Make sure to answer all your requirements; you do not want to overextend the capacity of your staff if it proves to be burdensome enough just to fulfill the requirements.

Further, there is not much of a purpose in collecting extra data from a program that will not retain its funding because it did not meet its basic requirements. Focus on the basics and you can always layer in additional data collection when you have the capacity.

When?

The answer(s) to “WHEN” often logically follow from the “WHY” and the “WHAT” –e.g., if you are collecting certain data elements for your annual report, when is that report due? And when will those data elements become available (e.g., you cannot collect standardized test scores until those tests have been taken)?

You should also consider the “WHO”, specifically the capacity of the data collectors. Does it make sense for your team to submit all their required service data on a weekly basis, or perhaps bi-weekly or monthly? That interval may need to change once your data collection system is implemented, but you should establish an initial expectation and be open to adjusting it as needed.

Where?

Where (from what sources) are you going to get your data?

Data privacy and security are incredibly important in the answers to “WHERE” questions. If you are working with a partner such as a school district, you should work with them to establish an official data sharing agreement. This is also where having a highly specific list of data points is important because those often need to be spelled out in such an agreement.

Where will you store your data?

Compliance with FERPA and HIPAA are simply non-negotiable. Make sure your team members know this too. Beyond that, however, you should try to make your data storage as accessible as possible for those who need it – for example, centralizing data storage as much as possible (in a security-compliant software program or file location) is likely to work better for you and your program because it will limit the number of passwords everyone needs to remember!

Why?

Why are you collecting these data? This answer may be obvious to you if you are in a leadership role. However, other members of your team whose daily work activities don’t involve grant writing or reporting are likely to know less than you think they do.

The reason I urge you to ask and answer “WHY” when you are building your data collection system is because you should ensure that everyone who will be involved in the system knows why they will be doing it. Building buy-in and understanding amongst the “WHO” that will be collecting the data is incredibly important not just to the creation of your data collection system, but to the implementation and ultimate success of that system.

Team members who know why they are collecting certain data will provide higher-quality data because they understand that the data collection has a purpose (one that will ultimately benefit them).

The answers to the 5 W’s –WHO, WHAT, WHEN, WHERE, and WHY— will set a strong foundation for a data collection system that works for you and your team.

Contributed By Joanna Full

Joanna Full is an Associate Researcher for the Research, Evaluation & Dissemination Department at the Center for Educational Opportunity Programs. She currently manages the data collection processes and conducts the formative and summative evaluations of several of CEOP’s federally funded college access programs, including GEAR UP and Talent Search.

Follow @CEOPmedia on Twitter to learn more about how our Research, Evaluation, and Dissemination team leverages data and strategic dissemination to improve program outcomes while improving the visibility of college access programs.