Error Reduction Over Time
The impact of risk-based data monitoring and remediation was to decrease the number of inconsistent scores generated by raters, thus the overall quality of scores is higher than without the implementation of this system. There is a statistically significant difference in error as indicated by the metric of scoring inconsistency reduction over visits (T-test comparing scoresv(at visit 1 and 8) T=2.29, df=6 and p ≤ 0.05).
Quantification of Rater Drift
In one industry sponsored study risk-based data monitoring was implemented midway through a schizophrenia trial to determine if there was any significant benefit. We compared serious inconsistencies generated before and after data monitoring began and quantified rater drift as Chi-square (Pearson) χ²(1, N = 2415) = 30.977, p < 0.001. This clearly demonstrated the difference in before and after risk-based data monitoring procedures.