Student retesting patterns have shifted over time, and we now see more students applying to college having taken the ACT multiple times.
This has an impact on higher education: Which scores should be used in the college admission decision process—the scores from the most recently taken test or the highest scores from each individual exam from across multiple tests?
And, if we use scores across multiple tests what are the implications for the validity and fairness of the standardized test scores?
Our latest research provided an opportunity to explore this issue. How predictive of success is the ACT test for students who submit multiple score results?
We know different colleges and universities use different policies for students who submit multiple sets of test scores. Some prefer the most recent test scores while others use the highest test results from across all test events.
A number of schools calculate the average across all ACT tests; others calculate a “superscore” by averaging the highest ACT subject score for each of the four sections—math, science, reading and English—from all test events to compute a new ACT composite superscore.
ACT has typically used students’ most recent scores for reporting and research purposes. This practice has been grounded in the rationale that the most recent results would be the most predictive of actual college performance since they reflect the student’s most recent level of academic achievement. Also, the notion of using superscores has raised concerns about potentially increasing measurement error, which might result in an overestimation of true achievement.
We decided to conduct research to help address these concerns and prove or disprove existing theories.
And the results surprised us!
We compared four common scoring methods (last, highest, average and superscore) against the accuracy of first-year college GPA predictions. The results: While all methods proved to be similarly predictive, the superscoring method was superior in terms of prediction accuracy.
We found that students who retest on the ACT perform better than expected in college based on their test scores for all four scoring methods; however, the prediction error was minimized when superscores were used compared to the other three scoring methods.
If the concerns about superscoring were correct, superscores would predict higher grades in college than what students actually earned. And that “overprediction” would increase the more times students retested.
The results of the study, however, showed exactly the opposite.
We dug deeper. In particular, we wanted to see if there may be unintended consequences associated with superscoring despite the strong validity evidence.
Previous research indicated that underserved students are less likely than their peers to take the ACT more than once. So, we conducted a second study exploring the impact of superscoring on students in different subgroups. The results are very promising. Subgroups are largely unaffected by superscoring. Moreover, superscores help decrease differences between different subgroups of students after taking into account the number of times tested.
What does this mean? It means that if we can encourage underserved students to take the ACT more often, superscoring may help reduce subgroup differences. If so, college opportunities and access may improve for traditionally underserved students.
ACT offers fee waivers for low-income students to take the test for free up to four times. We also provide those students free access to The Official ACT® Self-Paced Course, Powered by Kaplan® where they can find bite-sized, on-demand lessons that offer the perfect mix of structure and flexibility. If we can make more underserved students aware of these opportunities, perhaps more of them will take the test more than once and benefit when they apply to colleges.
In short, we empirically evaluated the validity and fairness of different score-use policies. Based on the findings, ACT now supports the use of superscoring in making college admissions decisions.
Follow ACT
ACT is a mission-driven, nonprofit organization dedicated to helping people achieve education and workplace success. Grounded in 60 years of research, ACT is a trusted leader in college and career readiness solutions. Each year, ACT serves millions of students, job seekers, schools, government agencies and employers in the US and around the world with learning resources, assessments, research and credentials designed to help them succeed from elementary school through career.
And, if we use scores across multiple tests what are the implications for the validity and fairness of the standardized test scores?
Our latest research provided an opportunity to explore this issue. How predictive of success is the ACT test for students who submit multiple score results?
We know different colleges and universities use different policies for students who submit multiple sets of test scores. Some prefer the most recent test scores while others use the highest test results from across all test events.
A number of schools calculate the average across all ACT tests; others calculate a “superscore” by averaging the highest ACT subject score for each of the four sections—math, science, reading and English—from all test events to compute a new ACT composite superscore.
ACT has typically used students’ most recent scores for reporting and research purposes. This practice has been grounded in the rationale that the most recent results would be the most predictive of actual college performance since they reflect the student’s most recent level of academic achievement. Also, the notion of using superscores has raised concerns about potentially increasing measurement error, which might result in an overestimation of true achievement.
We decided to conduct research to help address these concerns and prove or disprove existing theories.
And the results surprised us!
We compared four common scoring methods (last, highest, average and superscore) against the accuracy of first-year college GPA predictions. The results: While all methods proved to be similarly predictive, the superscoring method was superior in terms of prediction accuracy.
We found that students who retest on the ACT perform better than expected in college based on their test scores for all four scoring methods; however, the prediction error was minimized when superscores were used compared to the other three scoring methods.
If the concerns about superscoring were correct, superscores would predict higher grades in college than what students actually earned. And that “overprediction” would increase the more times students retested.
The results of the study, however, showed exactly the opposite.
We dug deeper. In particular, we wanted to see if there may be unintended consequences associated with superscoring despite the strong validity evidence.
Previous research indicated that underserved students are less likely than their peers to take the ACT more than once. So, we conducted a second study exploring the impact of superscoring on students in different subgroups. The results are very promising. Subgroups are largely unaffected by superscoring. Moreover, superscores help decrease differences between different subgroups of students after taking into account the number of times tested.
What does this mean? It means that if we can encourage underserved students to take the ACT more often, superscoring may help reduce subgroup differences. If so, college opportunities and access may improve for traditionally underserved students.
ACT offers fee waivers for low-income students to take the test for free up to four times. We also provide those students free access to The Official ACT® Self-Paced Course, Powered by Kaplan® where they can find bite-sized, on-demand lessons that offer the perfect mix of structure and flexibility. If we can make more underserved students aware of these opportunities, perhaps more of them will take the test more than once and benefit when they apply to colleges.
In short, we empirically evaluated the validity and fairness of different score-use policies. Based on the findings, ACT now supports the use of superscoring in making college admissions decisions.
That said, we believe that individual postsecondary institutions should decide which score-use policy is best for them, as they have unique needs and contexts within which the scores are being used. As colleges and universities go about the process of reviewing the existing score-use policy on their campuses, it is our hope that our latest research can serve as one source of evidence contributing to those conversations.
