Item 18a: Plans for assessment and collection of outcome, baseline, and other trial data, including any related processes to promote data quality (e.g., duplicate measurements, training of assessors) and a description of study instruments (e.g., questionnaires, laboratory tests) along with their reliability and validity, if known. Reference to where data collection forms can be found, if not in the protocol.
Example 1
“Primary Outcome
Delirium Recognition: In accordance with national guidelines [Reference X], the study will identify delirium by using the RASS [Richmond Agitation-Sedation Scale] and the CAM-ICU [Confusion Assessment Method for the ICU] on all patients who are admitted directly from the emergency room or transferred from other services to the ICU [intensive care unit]. Such assessment will be performed after 24 hours of ICU admission and twice daily until discharge from the hospital . . . RASS has excellent inter-rater reliability among adult medical and surgical ICU patients and has excellent validity when compared to a visual analogue scale and other selected sedation scales [Reference X] . . . The CAM-ICU was chosen because of its practical use in the ICU wards, its acceptable psychometric properties, and based on the recommendation of national guidelines [Reference X] . . . The CAM-ICU diagnosis of delirium was validated against the DSM-III-R [Diagnostic and Statistical Manual of Mental Disorders, Third Edition – Revised] delirium criteria determined by a psychiatrist and found to have a sensitivity of 97% and a specificity of 92% [Reference X]. The CAM-ICU has been developed, validated and applied into ICU settings and multiple investigators have used the same method to identify patients with delirium [Reference X].
Delirium Severity: Since the CAM-ICU does not evaluate delirium severity, we selected the Delirium Rating Scale revised-1998 (DRS-R-98) [Reference X] developed by Dr. P.T. and colleagues. The DRS-R-98 was designed to evaluate the breadth of delirium symptoms for phenomenological studies in addition to measuring symptom severity with high sensitivity and specificity . . . The DRS-R-98 is a 16-item clinician-rated scale with anchored items descriptions . . . The DRS-R-98 has excellent inter-rater reliability (intra-class correlation 0.97) and internal consistency (Cronbach’s alpha 0.94) [Reference X].
Secondary Outcomes
The study will collect demographic and baseline functional information from the patient’s legally authorized representative and/or caregivers. Cognitive function status will be obtained by interviewing the patient’s legally authorized representative using the Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE). IQCODE is a questionnaire that can be completed by a relative or other caregiver to determine whether that person has declined in cognitive functioning. The IQCODE lists 26 everyday situations . . . Each situation is rated by the informant for amount of change over the previous 10 years, using a Likert scale ranging from 1-much improved to 5-much worse. The IQCODE has a sensitivity between 69% to 100% and specificity of 80% to 96% for dementia [Reference X].
Utilizing the electronic medical record system (RMRS), we will collect several data points of interest at baseline and throughout the study period . . . We have previously defined hospital-related consequences to include: the number of patients with documented falls, use of physical restraints . . . These will be assessed using the RMRS, direct daily observation, and retrospective review of the electronic medical record. This definition of delirium-related hospital complications has been previously used and published [Reference X].”266
Example 2
“Training and Certification Plans
. . . Each center’s personnel will be trained centrally in the study requirements, standardized measurement of height, weight, and blood pressure, requirements for laboratory specimen collection including morning urine samples, counseling for adherence and the eliciting of information from study participants in a uniform reproducible manner.
. . . The data to be collected and the procedures to be conducted at each visit will be reviewed in detail. Each of the data collection forms and the nature of the required information will be discussed in detail on an item by item basis. Coordinators will learn how to code medications using the WHODrug software and how to code symptoms using the MedDRA software. Entering data forms, responding to data discrepancy queries and general information about obtaining research quality data will also be covered during the training session.
. . .
13.7. Quality Control of the Core Lab
Data from the Core Lab will be securely transmitted in batches and quality controlled in the same manner as Core Coordinating Center data; i.e. data will be entered and verified in the database on the Cleveland Clinic Foundation SUN with a subset later selected for additional quality control. Appropriate edit checks will be in place at the key entry (database) level.
The Core Lab is to have an internal quality control system established prior to analyzing any FSGS [Focal segmental glomerulosclerosis] samples. This system will be outlined in the Manual of Operations for the Core Lab(s) which is prepared and submitted by the Core Lab to the DCC [Data Coordinating Center] prior to initiating of the study.
At a minimum this system must include:
1) The inclusion of at least two known quality control samples; the reported measurements of the quality control samples must fall within specified ranges in order to be certified as acceptable.
2) Calibration at FDA [Food and Drug Administration] approved manufacturers’ recommended schedules.
13.8. Quality Control of the Biopsy Committee
The chair of the pathology committee will circulate to all of the study pathologist . . . samples [sic] biopsy specimens for evaluation after criteria to establish diagnosis of FSGS has been agreed. This internal review process will serve to ensure common criteria and assessment of biopsy specimens for confirmation of diagnosis of FSGS.”267
Explanation
The validity and reliability of trial data depend on the quality of the data collection methods. The processes of acquiring and recording data often benefit from attention to training of study personnel and use of standardised, pilot tested methods. These should be identical for all study groups, unless precluded by the nature of the intervention.
The choice of methods for outcome assessment can affect study conduct and results.268-273 Substantially different responses can be obtained for certain outcomes (e.g., harms) depending on who answers the questions (e.g., the participant or investigator) and how the questions are presented (e.g., discrete options or open ended).269;274-276 Also, when compared to paper-based data collection, the use of electronic handheld devices and Internet websites has the potential to improve protocol adherence, data accuracy, user acceptability, and timeliness of receiving data.268;270;271;277
The quality of data also depends on the reliability, validity, and responsiveness of data collection instruments such as questionnaires278 or laboratory instruments. Instruments with low inter-rater reliability will reduce statistical power,272 while those with low validity will not accurately measure the intended outcome variable. One study found that only 35% (47/133) of randomised trials in acute stroke used a measure with established reliability or validity.279 Modified versions of validated measurement tools may no longer be considered validated, and use of unpublished measurement scales can introduce bias and inflate treatment effect sizes.280
Standard processes should be implemented by local study personnel to enhance data quality and reduce bias by detecting and reducing the amount of missing or incomplete data, inaccuracies, and excessive variability in measurements.281-285 Examples include standardised training and testing of outcome assessors to promote consistency; tests of the validity or reliability of study instruments; and duplicate data measurements.
A clear protocol description of the data collection process – including the personnel, methods, instruments, and measures to promote data quality – can facilitate implementation and helps protocol reviewers to assess their appropriateness. Inclusion of data collection forms in the protocol (i.e., as appendices) is highly recommended, as the way in which data are obtained can substantially affect the results. If not included in the protocol, then a reference to where the forms can be found should be provided. If performed, pilot testing and assessment of reliability and validity of the forms should also be described.
| 17b: Emergency unblinding | 18b: Retention |
