[Assessment Literacy Video Series] Making Sense of Computer Adaptive Tests

Our assessment literacy video series aims to demystify, unpack, and connect assessment concepts and principles to help you make more sense out of your assessment data. Maybe you’re just learning the ropes of some of the more complicated metrics reported in educational assessments, or perhaps you’re hoping to see how an assessment concept applies to Edmentum’s suite of assessment programs. Either way, let our top-notch research team of former educators and subject-matter experts be your guide.

In this assessment literacy video, we’re talking about CATs! But, not your friendly housecat—we’re digging into CAT, as in computer-adaptive test. Do your students ever feel discouraged or frustrated when taking a test with questions that are really hard for them? Do they ever feel bored answering really easy questions? This is where adaptive tests help.

Computer Adaptive Test vs. Fixed-Form Test

When the questions of a test are predetermined and in a set order, we call the assessment fixed form. Paper-and-pencil exams are one example of a fixed-form test. A CAT is different. In a computer adaptive test, students receive targeted questions that adapt in real time based on a student’s responses and individual ability level. We’re excited about computer-adaptive tests because the Exact Path diagnostic is just that!

Understanding How a Computer-Adaptive Test Works

So, how does the CAT know which questions to administer? Each CAT has four main components:

1. a bank of questions that can be administered,
2. a rule for which question a student will get first,
3. an algorithm to determine which question to give next (harder or easier),
4. and a rule for when to end the test so that once we know as much as possible about the student’s ability, the test will stop.

So, how does the CAT know where to start? In the Exact Path diagnostic, the first question is chosen based on what we know about a student, which could be:

• how the student performed on their last test
• or what grade level they are in.

If the student gets the question right, the student gets a harder question; answer it wrong—get an easier question. After each question, we learn more about the student’s ability level and can update the estimate of their ability. Not only does the CAT algorithm look for harder or easier questions, it works to make sure there is enough information about a student’s ability in different domains. In fact, the CAT works to balance the number of questions given from each domain and even to vary the types of questions, like technology-enhanced items!

As the student answers more questions, the test learns more about what the student does or doesn’t know. Every answered question provides greater confidence and precision in the student’s scale score. Once the scale score is precise enough, the test is over. That means some students may receive fewer questions than others; it just depends on how many questions it takes to hit that targeted level of precision. In some cases, the test may stop after a set maximum number of questions have been administered.

The Exact Path diagnostic works just in this way so that each student receives an efficient and adaptive assessment experience that determines his or her strengths and needs and then uses that information to identify exactly where the student is ready to start learning. Because questions are regularly adjusting between harder and easier based on real-time responses, it’s expected that students only get about 50 percent of the questions correct.

Viewing the Adaptive Experience in Exact Path

You can follow the student’s adaptive diagnostic experience by viewing the Student Summary Report. See how the estimate of the student’s ability, those circles that are plotted across the chart, changes with each correct and incorrect answer? The bars around the estimate show the precision of the ability estimate, so notice how the bars tighten as the test continues and narrows in on the student’s ability. By using an adaptive diagnostic test, Exact Path is able to place students into content at the level where they are ready to learn and practice new skills.

Interested in more assessment literacy topics? Check out our Edmentum Assessment Literacy video series, and continue to follow along on the blog as we dig deeper, making you assessment experts along the way! Want to learn more about Exact Path? Get more information about our award-winning program on our website.