Validity and Reliability: How CommonLit's Assessment Series Measures Up

In our last post, we talked about what sets large-scale, standardized assessments apart from classroom assessments. If you're going to trust a standardized assessment enough to make decisions based on it, there are two questions worth asking first: Is it measuring the right thing? And would it give me a similar answer if a student took it again tomorrow? Those two questions get at validity and reliability, the backbone of any trustworthy assessment. Here's what those terms mean, and how CommonLit's Assessment Series is built to hold up on both counts.

First, What Do Validity and Reliability Actually Mean?

Validity is really a question of purpose: does this assessment measure what it claims to measure, and can you trust the conclusions you draw from it? A reading assessment is valid if a high score reflects strong reading comprehension, not something unrelated. Validity isn't a single yes-or-no property—it's built and supported through evidence, like showing that assessment scores align with other established measures of the same skill.

Reliability is a question of consistency: if a student took a similar version of the assessment again under similar conditions, would they get a similar score? Reliability is what allows educators to trust that a score reflects a student's actual skill level rather than test-taking luck.

Together, validity and reliability are what separate a well-built standardized assessment from something like an exit ticket. They're the evidence that a score means what it says it means, every time.

Building Validity Through Rigorous Assessment Development

Every question on the Assessment Series goes through a rigorous development and review process before it ever reaches a student. Passages are thoroughly vetted and selected, and standards-aligned questions are carefully drafted. Then they’re reviewed by content experts to confirm they're measuring reading comprehension rather than testing unrelated background knowledge or vocabulary quirks. This careful development process is a core piece of validity evidence: it shows that each question is designed, from the start, to measure students’ reading comprehension.

Building Reliability Through Field Testing and IRT Calibration

Once questions are developed, they go through multiple rounds of field testing with real students before being used operationally. Field testing allows CommonLit's assessment team to see how questions actually perform: whether a question is too easy, too hard, or confusing. Questions that don't perform as expected are revised or removed.

From there, CommonLit uses Item Response Theory (IRT) to calibrate each question. In plain terms, IRT analyzes how real students of varying skill levels responded to each question and uses that data to place every question on a shared difficulty scale. This calibration is what allows the Assessment Series to produce scaled scores—scores that account for the difficulty of the test a student took. A scaled score makes it possible to compare a student's performance across the Pre-Assessment, Mid-Year assessment, and Post-Assessment, since each is calibrated to the same underlying scale. That consistency is exactly what reliability requires.

What the Research Shows

This process isn't just theoretical—independent research backs it up. A study by LearnPlatform by Instructure, an independent edtech evaluation group, examined CommonLit Assessment Series scores against New York State end-of-year test results at two New York City middle schools and found a positive correlation between both the beginning-of-year and end-of-year CommonLit assessments and the New York State end-of-year tests. 

A second study conducted by CommonLit researchers in Alachua County, Florida looked at Assessment Series scores alongside the Florida Assessment of Student Thinking (FAST) ELA test and found an overall correlation of .71 between the two, a result considered substantial, meaning students who scored high on the Assessment Series also tended to score high on the FAST exam, and vice versa. 

Findings like these are what validity evidence looks like in practice: they show that Assessment Series scores meaningfully relate to other established measures of reading comprehension.

Validity and reliability aren't just academic terms—they're what makes it possible for educators to interpret assessment data with confidence. Because of CommonLit's rigorous assessment development process, multiple rounds of field testing, and IRT calibration, the Assessment Series produces scores that are both meaningful and consistent, backed by independent research showing they connect to real outcomes on state tests. Understanding this process is part of what it means to be assessment literate, and it's what allows schools to use the Assessment Series as a trusted piece of their broader assessment system.

Next Steps

You shouldn’t have to take the quality of your data on faith. The Assessment Series gives your school one consistent measure of reading growth, from the Pre-Assessment through the Post-Assessment, backed by research linking its scores to state test results. If you’re deciding which assessments belong in your school’s system, contact our team to see how the Assessment Series works and what it would look like for your students.