There is little doubt that afterschool programs have improved student outcomes wherever those programs are available. However, showing their value in concrete terms can be challenging.
A data-centric approach to measuring the effectiveness of your program provides the definitive evidence you need to secure funding and show parents the program is helpful. Also, measuring success gives you the data to implement improvements wherever your program falls short of expectations or needs updating.
Why You Should Measure
Measuring the right metrics identifies what works and what doesn't. You may intuitively feel that some parts of your program could use some tweaking, but until you learn where the problem lies, you can't introduce effective changes.
The information you gather fuels improvements that result in positive student outcomes and positive change for the community, potential funders, and policymakers. It also helps you and other organizations realize the value of collaboration instead of worrying about competition.
An Industrial Template for Business Measurement
While you might wince at thinking of your afterschool program as a manufacturing plant, quality improvement methods from industry are also helpful in the educational sphere.
Manufacturers pioneered the use of data and evidence-based improvements to reduce costs and increase revenues. The same basic concepts and methods are adaptable to almost any business. Through feedback and reiterative improvement systems, you can improve program quality and thus the ability to obtain funding and show parents and school districts that you are a valuable part of the community.
As you build your quality improvement processes, include relationship-building and communication improvements to create a successful afterschool program.
Tools to Measure Success
There is no single method that gets you the data you need. There are three tools every afterschool administrator should master to successfully measure program outcomes.
Goal Setting and Tracking
If you don't know where you're going, you won't know when you get there. Goal setting and tracking shows you:
- What you need to accomplish
- Why you need to achieve it
- How to accomplish it
Goals must be specific and realistic. Set a realistic time frame for reaching your goals and track the results to guide the improvement and maintenance of your afterschool program.
Data Collection and Evaluation
Gather and analyze data to help you measure your goals, assess outcomes, and make adjustments to serve students, families, and staff better.
Data collection tools include:
- Focus groups
- In-depth interviews
Select the methods that best fit your budget and resources. Surveys and questionnaires are easily performed via paper, email, and text, while focus groups and interviews require more time and resources.
Once you have the feedback, review and analyze the data to determine the necessary steps to address concerns and issues.
Your afterschool program may not have access to test scores or grades. Skill-based assessments performed within the program can provide the information you need to track student success.
For example, you can measure learning languages other than English by asking students to sing a song in Spanish. You can assess fitness with measured distance running. Keep a record of student performance during these assessments to measure progress.
Data Collection and Analysis
The most helpful data for measuring program success is usable data. Start with a data quality review. Is the data complete? Is it accurate? Check portions of each batch of observational tools for clarity and completeness and to ensure it reflects the data you intended to capture.
Organize the data in a database for ease of storage and manipulation. Spreadsheets, for example, are a great way to organize and store data. You can quickly perform calculations and create graphs, charts, and reports to convey critical information.
Analyze the data for the following:
- A range of compliance numbers for each observed goal
- The average number of times the goals were reached or missed
- The percentage of observations in which you achieved each goal
You need at least three data points to identify a trend. If you see the same issue more than three times on three different occasions by three different individuals, you can call it a trend. Be conservative in identifying trends, so you don't waste time and resources on irrelevant or nonexistent issues.
Use your findings for future training and business planning.
Continuous quality improvement is a never-ending feedback cycle. Once you analyze your data and verify trends, you can prioritize programming gaps and staffing to address the issues. What areas of your program need more attention? What about more resources?
Keep adjusting and making changes to reach your goals. No afterschool program is perfect, and outside challenges like financial concerns or staffing challenges can (and probably will!) force you to keep your goals realistic.
Continuous quality improvement methods can be used in afterschool programs to track success and demonstrate it to stakeholders. Data gathered from observational tools shows where improvements or changes are needed and where you met your goals.
Though it can be daunting, intentionally collecting data to measure success can help you prove your program's effectiveness to potential investors or families interested in sending their children to your program.