Relative Progress Measures and Standard Errors

Keep in mind that all relative progress measures reported in EVAAS are estimates. They are reliable estimates generated from a large amount of data using robust, research-based statistical modeling. But they are estimates, nonetheless.

Because relative progress measures are estimates, EVAAS reports display the standard error associated with each relative progress measure. Error is expected with any measurement. The standard error is a mathematical expression of certainty in an estimated value. The standard error on the EVAAS reports can be used to establish a confidence band around the relative progress measure. This confidence band helps us determine whether the increase or decrease in student achievement is statistically significant. In other words, it indicates how strong the evidence is that the group's achievement level increased, decreased, or remained about the same.

More Information about Standard Error

The standard error is specific to each relative progress measure because it expresses the certainty around that one estimate. The size of the standard error will vary depending on the quantity and quality of the data that was used to generate the relative progress measure. A smaller standard error indicates more certainty, or confidence, in the relative progress measure. A number of factors affect the size of the standard error, including:

• The number of students included in the analyses
• The number of assessment scores each student has, across grades and subjects
• Which specific scores are missing from the students' testing histories

To understand why these factors affect the size of the standard error, let's consider a few examples.

Imagine two groups of students. Each group could represent the students in a particular teacher's class, all the students in a grade and subject at a particular school, or even all of the students in a district. Both groups have the same relative progress measure in math. The relative progress measure for the first group is based on 11 students, while the relative progress measure for the second group is based on 60 students. Because this second group's measure is based on many more students, and therefore, much more data, we would be more confident in that estimate of Relative Progress. As a result, the standard error would be smaller for the second group.

Now let's consider two other groups of students with the same relative progress measure in math. This time both groups are equal in size, each having 60 students. However, in the first group all students have complete testing histories for the past five years. In contrast, quite a few students in the second group are missing prior test scores. The missing test scores create more uncertainty, so the relative progress measure for that group would have a larger standard error.

In our final example, there are again two groups of students, and again, both groups have the same relative progress measure in math. The groups are equal in size, with each group having 60 students. Both groups of students have five years of test scores, and in both groups there are five students who are missing one prior score. However, in the first group, the five students with missing scores are missing the previous year's math score. In the second group, the five students with missing scores are missing the math score from three years ago. Because we are generating a relative progress measure for math, the missing math scores from the prior year create more uncertainty than the missing math scores from several years ago. As a result, the group with the missing math scores would have a larger standard error.

The standard error is a critical part of the reporting because it helps to ensure that districts and schools are not disadvantaged in EVAAS because they serve a small number of students or because they serve students with incomplete testing histories.