Validity Assessment in Patients Seen for Clinical Care: an Important but Challenging Task.
Image by @davidmatos
A 59-year-old patient was referred by her GP to a memory clinic of a general hospital because of progressive cognitive complaints. Based upon a previous neuropsychological assessment, she was diagnosed with Mild Cognitive Impairment (MCI) – a preclinical stage of dementia. During the new assessment, however, symptom reporting was not in line with her fully intact functioning in daily life. For example, the patient reported not to be able to make simple calculations, such as adding 1 + 1. Despite such apparent cognitive difficulties, she successfully pursued a career as an artist. And although the new neuropsychological assessment confirmed bottom scores on nearly all cognitive tests, there also were clear psychometric indications that these were the result of performance below best of her capabilities. Combined, the inconsistent symptom reporting and noncredible performance precluded a valid determination of cognitive impairment, compromising the MCI diagnosis.
This clinical case is a clear illustration of assessing the credibility of symptom reporting and cognitive test performance through psychometric indicators as the preferred method, instead of relying solely on clinical judgment. More specifically, symptom validity tests (SVTs) are questionnaires designed to measure implausible symptom endorsement of items that are rare, atypical, or improbable. Performance validity tests (PVTs) are designed to measure implausible low performance on cognitive tests.
In recent decades, there is a growing recognition that noncredible symptom presentation extends beyond forensic contexts. Nowadays, regardless of the diagnostic focus and assessment context, every psychological assessment should incorporate SVTs and/or PVTs for comprehensive evaluation. With this shift, some important questions were left unaddressed that may aid validity assessment in routine clinical care. Two of the most important being: (1) How often clinical patients perform below their actual cognitive abilities?, and (2) how should one proceed when a patient is not performing at his/her best capabilities?
To answer the first question, we performed a systematic review and meta-analysis and found a significant minority of adult clinical patients to fail a stand-alone PVT (i.e., 16%, 95% [CI 14-19]). This finding underlines the importance of assessing performance validity in every assessment before interpreting neuropsychological test performance. Put otherwise, neglecting a standard check on performance validity potentially leads to misdiagnosis in a substantial number of clinical cases.
Regarding the second question on how to proceed in case of noncredible responding, a key difference exists between forensic and clinical psychological assessment. In forensic settings, the clinician typically answers legal questions on how noncredible behaviors are related to the examinees’ health claims (e.g., how noncredible memory test performance is related to an alleged crime). In clinical contexts, however, the clinician forms a doctor-patient relationship and acts as an advocate for the patient. The focus is still on providing useful clinical services, despite noncredible symptom presentation. Arguably, management of noncredible responding in the clinical setting poses more challenges to the clinician as compared to the forensic context. We therefore examined whether a brief corrective feedback intervention would impact subsequent performance in patients who performed below best of their capacities, and found mixed results.
Want to know more?
On January 19th 2024, Jeroen Roor defended his PhD thesis entitled “Performance Validity in Clinical Neuropsychological Assessment: Base Rates, Impact of Feedback, and Relevance to Outcomes”.
A digital version of his thesis can be found here: https://cris.maastrichtuniversity.nl/en/publications/performance-validity-in-clinical-neuropsychological-assessment-ba
To receive a hardcopy or for any questions regarding his PhD thesis, contact Jeroen through jeroen.roor@maastrichtuniversity.nl