ACT Newsroom & Blog

The following is re-posted with permission from Smart Sparrow.

Let’s face it, the word “transformation” is simply “change” rebranded, and we use that word because change - thinking about it, planning for it, doing it - is often hard, painful even. Think about the popular example that we use when describing transformation: the metamorphosis of a caterpillar into a butterfly. I mean, that transformation, or change, is super taxing on that poor caterpillar.

But change is critical in business; it means you are growing, learning, evolving. The fact that good organizations embrace change has been extensively written about; some transform to serve customers better, others transform to survive. I would argue that it is mainly because organizations evolve that internal Training & Development exists. Ultimately it’s the people behind the scenes that make such transformations possible.

So what does it look like when a whole organization is “transforming?” More importantly what does it require?

I can’t think of an organization undergoing a bigger transformation right now than ACT, moving from an Assessment organization to a Learning, Measurement and Navigation company. Not an easy task. You can tune in to our previous episode where I had the pleasure to speak with ACT’s CEO, Marten Roorda, to learn about his vision for the education ecosystem and the organization’s transformation beyond assessment, into a holistic learning company as part of that ecosystem.

In the second episode in our ACT series, I talked to someone whose job description is a little different from what most people think when we consider the field of education and learning. I had the pleasure to speak with Jennifer Yi Boyer, the Chief Talent Officer at ACT, about the need for great organizational and corporate learning and training.

How she and her team approach learning is refreshing and inspiring: no longer seeing it as an expense but as an investment in each team member. I’m so excited that she sat down with me to open the hood on this part of ACT and share her insights.

I invite you to tune into this episode of The Learning Nerds Podcast - Opening the hood of ACT’s Corporate Learning & Training: from an Expense to an Investment:

Media coverage of the recent ”Varsity Blues” investigation and prosecution involving celebrities and wealthy parents generated a lot of discussion about the role of privilege in admissions. The parents paid a “consultant” large sums of money to manipulate the admissions process and create unfair advantages for their children, who already possessed tremendous privilege.

There has been substantial attention paid to this matter and the issues it raises on both traditional and social media, including hundreds of articles, tweets and comments. However, we believe there is a national dilemma that is broader and demands greater attention: Our national credo that the doors of opportunity are fully open to all those who work hard and play by the rules cannot be reconciled with the levels of structural inequality and vast differences in educational opportunity that have grown wider and deeper in our society.

It should be no surprise that children born and raised in high-net-wealth families living in areas of high socioeconomic status realize significant and long-lasting benefits. These benefits are rarely available to similarly bright, promising and deserving children who grow up in middle class or lower-economically developed areas.

The micro-advantages enjoyed by the prosperous few accumulate and multiply over time, thus increasing advantage and privilege. It’s similar to the way that early deposits in a savings or retirement account build compounding interest and returns over time.

There is nothing wrong with parents trying to provide opportunities for learning and personal growth to their children. What is wrong is that, as a society, we have not invested enough to ensure that all children, regardless of the circumstances of their birth, have a fair chance to lead healthy and productive lives. In other words, talent and potential are widely distributed in our society, while opportunity is not.

A Holistic Mosaic

At ACT we believe that each individual is unique and complex. That’s why we strive to provide a holistic mosaic of the strengths of each student—a mosaic of skills that include academic, social and emotional learning, creative thinking, and collaborative problem-solving. Our goal is to help students, families and teachers build on those strengths and address areas where the student might need additional assistance.

We are also focused on providing many opportunities for learning, on helping individuals navigate educational and career transitions, and on guiding development of these skills and abilities in a personalized and adaptive way.

Our programs and tools are anchored on the ACT Holistic Framework, a research-based system that identifies a broad range of factors which determine success, including academic factors such as those measured by the ACT® and ACT® WorkKeys® career readiness assessments and also social and emotional learning skills and others.

It begins with measuring multiple skills that are related to college success, based not on the neighborhood in which students live, but on individual achievement, skills and experience.

Certainly, academic achievement measured by grades and the ACT test has been shown to be highly predictive of college success, but other factors in the holistic framework can also be measured for individual students, such as:

• social and emotional skills that influence engagement and learning,
• cross-cutting skills in areas like collaborative problem-solving and creative thinking, and
• skills related to exploration, planning and decision-making related to college majors and occupations.

Understanding these domains of growth and development will help students receive the support they need to pursue their goals and aspirations.

Closing the Gaps in Equity for Learning

ACT’s transformation from an assessment company to an organization providing learning, measurement, and navigation support to learners is motivated by our commitment to helping schools and communities close gaps in equity for learning. We want to help all individuals achieve their potential and be ready for college.

For example, our ACT® Academy™ provides free, personalized and adaptive learning opportunities to prepare students for courses taken in high school as well as standardized tests. We developed ACT® Tessera®, an SEL assessment, to help students identify their social and emotional skills and then improve upon them using the instructional materials and resources available through the ACT Academy platform. And ACT recently acquired NRCCUA, which through its MyOptions platform, provides students with free access to resources and recommendations for navigating the college admission process.

We believe attempts to quantify the degree of adversity encountered by students, while well-intentioned, are misguided because of the quality and types of data employed. Using census or aggregate data is common in research but unfair when applied to individuals. It is no different than using the average test score at a school to rank individual students rather than their actual test score, or to imagine that an individual student can be understood using a test score alone.

When school- or neighborhood-level data are used to compute an adversity score, the score might reflect a portion of the context for that individual’s experience, but it will rarely, if ever, reflect the fuller circumstances of that individual’s life and, as such, it runs the risk of or introducing or reinforcing potential bias or stereotypes.

A person’s environment does matter. But even among students living in the same neighborhood, there can be significant differences in the family lives, resources, opportunities and challenges experienced by individual students. When adversity or opportunity is used to make decisions about individuals (such as in admissions, employment, scholarships, etc.), the measures must be based on individual factors, not an average across a zip code.

Every person deserves a fair chance. At ACT, we focus on creating more equitable environments and opportunities that will impact an individual’s future success at college or work, rather than another score that attempts to account for societal differences without making any improvement in the lives or educational experiences of students.

---

Follow ACT

          

Well, let’s clarify that statement. Developing a reliable, valid, and fair test is no walk in the park.

You may not be aware of the amount of time and resources that we dedicate to research, develop, and maintain the ACT® test to ensure it is of the highest quality. We will describe some of those key activities for those interested in getting a peek behind the curtain.

How do we determine what to measure on the ACT?

First, ACT begins test construction by developing a blueprint, which is not that dissimilar from a building blueprint. We specify which skills will be included for each subject and how they will be measured (e.g., functions, algebra, historical passage, and literary passage).

The blueprint ensures coherence and consistency in what is measured and how it is measured across all test forms. Nearly every three years, ACT surveys educators at all levels to understand what students need to know and be able to do to succeed from elementary school to college and into the workforce. These results are used to inform the test design and to ensure the test measures what is taught in the classroom.

How do we determine that ACT items are reliable, valid, and fair?

Item quality starts with quality item writers.

ACT hires experienced item writers from diverse backgrounds and requires them to successfully complete training and meet quality standards. In terms of diversity, ACT is careful to include item writers who represent the diversity of the US population with respect to race, ethnicity, gender and geographic location to ensure that the content of the items are fair, unbiased, accessible, and non-offensive to all students.

In addition to having an external review to confirm the accuracy and clarity of items, every single test item is reviewed for bias and fairness multiple times by fairness experts. ACT conducts formal external fairness reviews of test material and of each item before any are used to calculate ACT scores.

Finally, ACT pretests every item before it appears on an operational form to verify that items are functioning properly.


When items survive pretesting and multiple reviews, they then move to operational administration to go through another stage of review. When an entire test form is assembled, items and the form are reviewed again to ensure the items conform to the blueprint, meet statistical and subject matter criteria, and are balanced in terms of focus, subject, and skill.

How do we determine that ACT scores are reliable, valid, and fair?

To ensure score comparability across ACT administrations, each test form is equated so that a 22 on one test administration means the same thing as a 22 on another administration. Given that the ACT is predominately used for college admissions, the relationship between ACT scores and college success (e.g., grades, retention, graduation) is continually evaluated. And we don’t just make sure that scores are predictive of success for the total group but also for subgroups of students (e.g., males, Hispanic students, low-income students). We also examine whether (and if so, why) prediction accuracy varies by student subgroup. In particular, we want to know whether subgroups of students do better or worse in college than what their test scores would suggest.

The collection of this evidence can take months and often years. But it is necessary.

The ACT is used to make high-stakes decisions. We owe it to all of our stakeholders – students, parents, educators, and policymakers – to demonstrate that the ACT is a reliable, fair, and valid measure of college readiness.

We have only scratched the surface in terms of describing the work that is done to ensure the quality of the ACT. To learn more, you can follow this link to the ACT technical manual.

Last fall, ACTNext, the innovation unit of ACT, unveiled its new tool for learning assessment systems (LASs), the Recommendations and Diagnostics (RAD) application programming interface (API).

Before I get into how the tool is helping students and its potential for benefiting EdTech companies, let’s cover a few definitions to clarify the alphabet soup:

A learning assessment system (LAS) contains information on specific learning goals or standards aligned to curriculum, learning and assessment. This provides students and educators with information to navigate the next steps of the learning process to achieve learning goals or standards.

Next, an application programming interface (API) allows different software systems to talk to each other. APIs act as connectors between customers and companies.

So, back to the learning assessment system tool ACTNext built: the RAD API.

RAD delivers on ACT’s promise to transform learning through the development of holistic education technology, utilizing 60 years of ACT research and item response data. It’s a clever approach that integrates assessment with personalized and adaptive learning.

And it’s a reality today because of ACTNext’s pioneering work to merge artificial intelligence, machine learning and algorithmic development with computational psychometrics, the science of measuring mental capacities.

How does RAD help learners?

RAD tracks test results and learner evidence over time to build a diagnostic model of the skills students have mastered and ones still requiring review. Building those diagnostic models then drives a “recommendations engine” that provides specific sets of personalized learning resources to students.

Once the diagnostic model identifies a subject area, like math or science, where learners can benefit from review, then they’re connected to resources. This gives specific practice and prep work, such as providing learning resources for linear equations within algebra, for example.

When learners navigate to the resources tab, they can choose an area of study by going to a subject category. RAD builds personalized lists of recommendations based on identified areas of need. For example, complex number operations have floated to the top for this learner's number and quantity area in math.

How does it work?

RAD gathers learner data in real time and applies results immediately.

Here‘s how it works: A student takes a quiz. Then, the learning assessment system relays the results of the quiz (in what’s known as a standard data format using the Caliper standard),
to the API. RAD then uses that knowledge to update the learner’s diagnostic model.

The RAD engine not only supports individual learners, it delivers insights gleaned from student populations in aggregate. RAD fine-tunes learning recommendations according to the difficulty of a skill (or skill area) and can detect misconceptions in a subject by evaluating patterns of response selections across student populations. In the future, RAD will be able to adaptively change content through instructional materials.

RAD generates snapshots of learner knowledge that are sent to the recommendations engine.

Our team had a “light bulb moment” after creating ACTNext’s prototype learning mobile app, called the Education Companion, when we realized RAD could serve other learning assessment systems. That was a big moment for the development team!

We found the RAD engine can be extended and configured for any learning assessment system that uses a standard compliance for learning technologies—the IMS Global Caliper standard. In fact, any taxonomy that meets the Caliper standard can connect with RAD.

Adjustments to skill predictions at lower levels are propagated up the hierarchy so that RAD can also yield probabilistic predictions at higher levels.

What is the Dashboard component of RAD?

Our goal is to provide educators, administrators and customer support reps with insight into the learning assessment system, so we developed a dashboard that includes a web-facing interface. You now have a detailed view into what’s happening inside the software service so you can see what RAD is recommending and the diagnostics.

Using the dashboard, educators will be able to check learner progress in any subject. Over time, with longitudinal tracking, a teacher can see the progress of one student, their entire classroom, a whole school, or district.

While the RAD enables students to work at their own pace and in a self-directed way, receiving personalized feedback and help, the dashboard allows teachers and administrators access to the “big picture.”

What else makes RAD different than other learning software?
RAD is a modular, separable component, not hard wired into any particular platform. It can be plugged into different learning assessment systems, providing widespread application possibilities, for various customers.

Download the fact-sheet for more information about RAD capabilities and technical specifications.

After the College Board announced it will provide an adversity score to every SAT score report it sends to colleges, many people, including representatives from the media, asked me if ACT is preparing something similar.

Let me start by saying that I appreciate the College Board’s effort to level the playing field for students. Like the College Board, ACT is a mission-driven nonprofit organization—we will do anything we can to improve equity by making our products and services affordable—and free for those who are from low-income families.

I acknowledge that a measurement of academic skills is only one of many factors that define readiness for college and future success, which is why at ACT we developed the Holistic Framework, a science-based framework that underlies our philosophy, our tests and our new learning products. We are focused on measuring and helping students improve their 21st-century and social emotional learning skills in addition to their academic skills. We work hard to establish the validity and reliability of those products.

But ACT will not follow the College Board’s example, even though I’m convinced of David Coleman’s good intentions.

I think the SAT “adversity score” is not a great idea. Let me explain.

Specific Concerns

Scores that affect students’ futures require transparency, validity and fairness. The algorithm and research behind this adversity score have not been published. It is basically a black box. Any composite score and any measurement in general requires transparency; students, teachers and admissions officers have the right to know. Now we can’t review the validity and the fairness of the score. And even if that changes, there is also an issue with the reliability of the measure, since many of the 15 variables come from an unchecked source—for example, when they are self-reported by the student.

The plan to report the adversity score only to the college is another example of not being transparent. If I were a student, I would become concerned or angry if the testing company would provide an adversity score to colleges without me knowing it, without me approving it, and without any of the end users understanding how this score is calculated.

I understand when David Coleman says, “We can’t ignore the disparities of wealth reflected in the SAT.” But the test is not biased in itself, and the score is what it is, even though it is sometimes a messenger of what’s unequal in society, not in the student. The test should be the equalizer, but by providing an adversity score you invite its users to adjust scores with it, starting an equating practice that undermines the equalizing effect of the score scale and invites bad behavior.

If parents, teachers and counselors know test scores will be re-equated for adversity, some will attempt to manipulate and game the system. That is easy: You can use an address of someone you know who is living in a poor neighborhood or report lower family income.

Mutual Responsibilities

I acknowledge that testing companies have a responsibility in leveling the playing field for students, which is why we offer fee waivers for low-income students and ACT Academy, a personalized learning and practice platform, free to all students and teachers.

I acknowledge that underserved students face barriers that their more fortunate peers don’t, requiring them to work harder and show greater resilience to reach their goals. But I think admissions officers are already very capable of assessing students’ hurdles without an adversity score, and that assessing an individual student’s resilience or grit is a much better measure than neighborhood adversity.

While I don’t think the College Board’s solution is the right one, I do believe that virtually everyone involved in the educational process wants to increase opportunities for students, particularly those who could most benefit from our assistance. I think we all can agree on that.

IOWA CITY, Iowa—ACT announced today that NewSchools Venture Fund, a national nonprofit that invests in educators and innovators who want to reimagine learning, has awarded the nonprofit learning, measurement and navigation organization a grant to advance its work on social and emotional skills.

The grant, which is typically awarded to early-stage innovators and educators, rather than established organizations such as ACT, will support further development of ACT Tessera, a social and emotional skills measurement program, including the ability to conduct critical research among underserved learners. Funding will also provide a real world application to demonstrate the efficacy of the ACT Tessera Teacher Playbook for improving social and emotional skills.

“We are very proud and excited to have won this grant, which will assist our efforts to help students, particularly those who are underserved, succeed both in and out of school,” said Jim Larimore, chief officer for the ACT’s Center for Equity in Learning. “We are committed to leveling the playing field for all students, regardless of needs, backgrounds or resources, in their efforts to learn essential skills and navigate their pathway to college and career readiness.”

The grant is part of the NewSchools Ignite Expanded Definition of Student Success Challenge, a program open to innovators developing technology-enabled learning experiences, instructional content, learning diagnostics, administrative tools and other products that promote an expanded definition of student success by enhancing academic and social-emotional learning and supporting the development of nurturing school and classroom environments.

“We believe ed tech tools hold great promise in this area,” said Tonika Cheek Clayton, Managing Partner at NewSchools Venture Fund. “ACT Tessera has the potential to increase equity and improve academic outcomes by supporting the development of social-emotional competencies as well as improvements in school culture and climate. That’s why NewSchools Ignite is proud to invest in ACT.”

“There is now a large amount of research showing that social and emotional skills are essential to both education and workplace success,” said Jeremy Burrus, ACT’s senior director for social, emotional, and academic learning. “Research supported by this grant will improve ACT Tessera’s ability to assess and teach those skills and help students succeed.”

ACT’s study will involve approximately 14,000 students at 25 middle schools in south Texas who participate in GEAR UP (Gaining Early Awareness and Readiness for Undergraduate Programs), a program designed to help underserved learners prepare for college. The research will be conducted over a full calendar year.

---

Follow ACT



About ACT
ACT is a mission-driven, nonprofit organization dedicated to helping people achieve education and workplace success. Headquartered in Iowa City, Iowa, ACT is trusted as a leader in college and career readiness, providing high-quality assessments grounded in nearly 60 years of research. ACT offers a uniquely integrated set of solutions designed to provide personalized insights that help individuals succeed from elementary school through career.

About ACT’s Center for Equity in Learning
ACT’s Center for Equity in Learning focuses on closing gaps in equity, opportunity and achievement for underserved populations and working learners. Through purposeful investments, employee engagement, and thoughtful advocacy efforts, the Center supports innovative partnerships, actionable research, initiatives, campaigns, and programs to further ACT’s mission of helping people achieve education and workplace success. http://equityinlearning.act.org

About NewSchools Venture Fund
NewSchools Venture Fund is a national nonprofit that supports and invests in promising innovators and teams of educators who want to reimagine learning. We help them accomplish their missions to achieve outstanding results for the schools, students, and educators they serve. We are committed to helping students graduate high school prepared and inspired to achieve their most ambitious dreams and plans. Through our investments, management assistance, network building, and thought leadership, NewSchools helps to reimagine PreK-12 education. To learn more, visit newschools.org, or follow us on Twitter, Facebook, and Instagram.

This month, ACT launched its first podcast, Ready for Work. The podcast focuses on workforce, career education, and economic development. Listeners will learn about how to make our communities more competitive and hear from practitioners working at the grassroots level to strengthen the workforce ecosystem.

Ready for Work is hosted by Jasen Jones, a member of the ACT® Work Ready Communities team. ACT Work Ready Communities empowers states, regions, and countries with data, processes, and tools that drive economic growth. Participants leverage the ACT® WorkKeys® National Career Readiness Certificate® (NCRC®) to measure and close the skills gap. To-date, over 4.6 million NCRCs have been issued, with nearly 25,000 employers supporting the program.

New episodes will be released every other week. Listen to the first episode below, and never miss an episode by subscribing wherever you listen to podcasts.

Episode 1: Role of Economic Development in the Workforce Ecosystem

Episode 1 includes a discussion of the ACT Work Ready Communities model to empower economic developers to assist stakeholders in closing the skills gap and maximizing the value of the workforce.

Jasen talks with Will Coppage, Deputy Director of the Washington County Economic Alliance, about regionalism and alignment of local career education and targeted fast-track training to strengthen the region’s workforce.