Monitoring the quality of clinical trial efficacy outcome data has received increased attention in the past decade, with regulatory guidance encouraging it to be conducted proactively, and remotely. However, the methods utilized to develop and implement risk-based data monitoring (RBDM) programs vary, and there is a dearth of published material to guide these processes in the context of central nervous system (CNS) trials. We reviewed regulatory guidance published within the past 6 years, generic white papers, and studies applying RBDM to data from CNS clinical trials. Methodologic considerations and system requirements necessary to establish an effective, real-time risk-based monitoring platform in CNS trials are presented. Key RBDM terms are defined in the context of CNS trial data, such as “critical data,” “risk indicators,” “noninformative data,” and “mitigation of risk.” Additionally, potential benefits of, and challenges associated with implementation of data quality monitoring are highlighted. Application of methodological and system requirement considerations to real-time monitoring of clinical ratings in CNS trials has the potential to minimize risk and enhance the quality of clinical trial data.

Rater training and the maintenance of the consistency of ratings are critical to ensuring reliability of study measures and sensitivity to changes in the course of a clinical trial. The Positive and Negative Syndrome Scale (PANSS) has been widely used in clinical trials of schizophrenia and other disorders and is considered the “gold standard” for assessment of antipsychotic treatment efficacy. The various features associated with training and calibration of this scale are complex, reflecting the intricacy and heterogeneity of the disorders that the PANSS is used to evaluate. In this article, the authors review the methods for ensuring reliability of the PANSS as well as a proposed trajectory for its use in the future. An overview of the current principles, implementation, technologies, and strategies for the best use of the PANSS; tips for how to achieve consistency among raters; and optimal training practices of this instrument are presented.

The Depression Inventory Development project is an initiative of the International Society for CNS Drug Development whose goal is to develop a comprehensive and psychometrically sound measurement tool to be utilized as a primary endpoint in clinical trials for major depressive disorder. Using an iterative process between field testing and psychometric analysis and drawing upon expertise of international researchers in depression, the Depression Inventory Development team has established an empirically driven and collaborative protocol for the creation of items to assess symptoms in major depressive disorder. Depression-relevant symptom clusters were identified based on expert clinical and patient input. In addition, as an aid for symptom identification and item construction, the psychometric properties of existing clinical scales (assessing depression and related indications) were evaluated using blinded datasets from pharmaceutical antidepressant drug trials. A series of field tests in patients with major depressive disorder provided the team with data to inform the iterative process of scale development. We report here an overview of the Depression Inventory Development initiative, including results of the third iteration of items assessing symptoms related to anhedonia, cognition, fatigue, general malaise, motivation, anxiety, negative thinking, pain and appetite. The strategies adopted from the Depression Inventory Development program, as an empirically driven and collaborative process for scale development, have provided the foundation to develop and validate measurement tools in other therapeutic areas as well.

This paper summarizes the results of the CNS Summit Data Quality Monitoring Workgroup analysis of current data quality monitoring techniques used in central nervous system (CNS) clinical trials. Based on audience polls conducted at the CNS Summit 2014, the panel determined that current techniques used to monitor data and quality in clinical trials are broad, uncontrolled, and lack independent verification. The majority of those polled endorse the value of monitoring data. Case examples of current data quality methodology are presented and discussed. Perspectives of pharmaceutical companies and trial sites regarding data quality monitoring are presented. Potential future developments in CNS data quality monitoring are described. Increased utilization of biomarkers as objective outcomes and for patient selection is considered to be the most impactful development in data quality monitoring over the next 10 years. Additional future outcome measures and patient selection approaches are discussed.

The Cognitive Drug Research (CDR) System is a set of nine computerized tests of attention, information processing, working memory, executive control and episodic memory which was designed for repeated assessments in research projects. The CDR System has been used extensively in clinical trials involving healthy volunteers for over 30 years, and a database of 7751 individuals aged 18–87 years has been accumulated for pre-treatment data from these studies. This database has been analysed, and the relationships between the various scores with factors, including age, gender and years of full-time education, have been identified. These analyses are reported in this paper, along with tables of norms for the various key measures from the core tasks stratified by age and gender. These norms can be used for a variety of purposes, including the determination of eligibility for participation in clinical trials and the everyday relevance of research findings from the system. In addition, these norms provide valuable information on gender differences and the effects of normal ageing on major aspects of human cognitive function.

Background: Electronic administration of clinician-reported outcomes (eClinROs) has advantages over paper-based methods, but the mode of administration change has the potential to affect the validity of the scale. The literature on migration of patient reported outcomes (PROs) suggests that there are different levels of modification, which necessitate different approaches to demonstrating mode equivalence. However, little has been written on the migration of ClinROs to electronic administration. Methods: We propose a method of comparing paper and electronic versions of scales that includes a comparison based on content and a comparison based on format. The determination of whether the eClinRO has undergone minor, moderate, or substantial modification will drive the necessary studies required for validation. The unique characteristics of ClinROs suggest 2 additional types of modifications, including functionality adaptation and adaptation of instructions. Conclusions: In many respects, the migration of a ClinRO to electronic administration is similar to that of a PRO. This article has explored the ways in which there might be special considerations for ClinROs that have not been elaborated for PROs

Background: Considering the scarcity of longitudinal assessments of reliability, there is need for a more precise understanding of cognitive decline in Alzheimer’s Disease (AD). The primary goal was to assess longitudinal changes in inter-rater reliability, test retest reliability and internal consistency of scores of the ADAS-Cog. Methods: 2,618 AD subjects were enrolled in seven randomized, double-blind, placebo-controlled, multicenter-trials from 1986 to 2009. Reliability, internal-consistency and cross-sectional analysis of ADAS-Cog and MMSE across seven visits were examined. Results: Intra-class correlation (ICC) for ADAS-Cog was moderate to high supporting their reliability. Absolute Agreement ICCs 0.392 (Visit-7) to 0.806 (Visit-2) showed a progressive decrease in correlations across time. Item analysis revealed a decrease in item correlations, with the lowest correlations for Visit 7 for Commands (ICC=0.148), Comprehension (ICC=0.092), Spoken Language (ICC=0.044). Discussion: Suitable assessment of AD treatments is maintained through accurate measurement of clinically significant outcomes. Targeted rater education of ADAS-Cog items over-time can improve ability to administer and score the scale.

Background: Debate persists with regard to how best to categorize the syndromal dimension of negative symptoms in schizophrenia. The aim was to first review published Principle Components Analysis (PCA) of the PANSS, and extract items most frequently included in the negative domain, and secondly, to examine the quality of items using Item Response Theory (IRT) to select items that best represent a measurable dimension (or dimensions) of negative symptoms. Methods: First, 22 factor analyses and PCA met were included. Second, using a large dataset (n = 7187) of participants in clinical trials with chronic schizophrenia, we extracted items loading on one or more PCA. Third, items not loading with a value of ≥0.5, or loading on more than one component with values of ≥0.5 were discarded. Fourth, resulting items were included in a non-parametric IRT and retained based on Option Characteristic Curves (OCCs) and Item Characteristic Curves (ICCs). Results: 15 items loaded on a negative domain in at least one study, with Emotional Withdrawal loading on all studies. Non-parametric IRT retained nine items as an Integrated Negative Factor: Emotional Withdrawal, Blunted Affect, Passive/Apathetic Social Withdrawal, Poor Rapport, Lack of Spontaneity/Conversation Flow, Active Social Avoidance, Disturbance of Volition, Stereotyped Thinking and Difficulty in Abstract Thinking. Conclusions: This is the first study to use a psychometric IRT process to arrive at a set of negative symptom items. Future steps will include further examination of these nine items in terms of their stability, sensitivity to change, and correlations with functional and cognitive outcomes.

Background: Considering the increasing attention to the study of failed clinical trials, the goal of this study was to identify the sources of unreliability in a failed clinical trial by assessing scores on the Positive and Negative Syndrome Scale (PANSS). Methods: This study is a substudy from a failed phase 2 double-blind, placebo-controlled trial of schizophrenia. Using the generalizability theory, this substudy assesses reliability on 3 conditions: raters, time points (PANSS evaluations, 1 week apart), subjects for 3 groups (placebo responders, placebo nonresponders, and treatment group). Results: The placebo response rate was 40.07% (32/71). For all PANSS positive symptom items, the most variability was for raters (range, 33%Y72%) for the placebo responders, 31% to 68% for the placebo nonresponders, and 29% to 60% for the treatment group. The variability of the interaction of rater and time point was the second source of unreliability, with an average of 12.28% compared to 12.00% for the placebo nonresponders and 10.00% for the treatment group. All items of the negative symptom subscale showed the most percent variability for raters, for all groups. For general psychopathology items (except preoccupation), raters accounted for the most variability in the scores for placebo responders with an average of 51.00% across items. A similar pattern was observed for the placebo nonresponders and for the treatment group; for the treatment group, the interaction between rater and time point accounted for the most variability for somatic concern and anxiety. Conclusions: Results confirm the efficacy of applying the generalizability theory to the estimation of reliability to identify a source of unreliability and provide evidence for the relationship between low reliability and failed trials. Findings can be used to guide data monitoring, rater training, and identification of PANSS items, which may require supplementary training.