Abstract
Background
This study aimed at: (1) assessing, in an Italian cohort of non-demented Parkinson’s disease (PD) patients, the construct validity of the Montreal Cognitive Assessment (MoCA) against both first- and second-level cognitive measures; (2) delivering an exhaustive and updated evaluation of its diagnostic properties.
Methods
A retrospective cohort of N = 237 non-demented PD patients having been administered the MoCA was addressed, of whom N = 169 further underwent the Mini-Mental State Examination (MMSE) and N = 68 the Parkinson’s Disease Cognitive Rating Scale (PD-CRS). A subsample (N = 60) also underwent a second-level cognitive battery encompassing measures of attention/executive functioning, language, memory, praxis and visuo-spatial abilities. Construct validity was assessed against both the PD-CRS and the second-level cognitive battery. Diagnostics were tested via receiver-operating characteristics analyses against a below-cut-off MMSE score.
Results
The MoCA was associated with both PD-CRS scores (p < .001) and the vast majority of second-level cognitive measures (ps < .003). Both raw and adjusted MoCA scores proved to be highly accurate to the aim of identifying patients with MMSE-confirmed cognitive dysfunctions. A MoCA score adjusted for age and education according to the most recent normative dataset and < 19.015 is herewith suggested as indexing cognitive impairment in this population (AUC = .92; sensitivity = .92; specificity = .80).
Discussion
The Italian MoCA is a valid and diagnostically sound screener for global cognitive inefficiency in non-demented PD patients. Further studies are nevertheless needed that confirm its diagnostic values against a measure other than the MMSE.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Background
Up to 40% of non-demented patients with Parkinson’s disease (PD) present with cognitive impairment [1] within both non-instrumental functions—i.e. attention and executive functioning—and instrumental domains—i.e. memory, visuo-spatial skills and language [2]. Since such dysfunctions detrimentally impact on patients’ functional outcomes [3] and prognosis [4], to screen for them via clinimetrically sound and feasible performance-based tests is clinically pivotal [1, 2] and thus highly advisable [5].
To this aim, according to the 2018 Movement Disorders Society (MDS) guidelines [5], the Montreal Cognitive Assessment (MoCA) [6] is—amongst those tests that are disease-nonspecific—strongly recommended. Such a screener has indeed received major support for use in this population by the International literature as far as its psychometrics, diagnostics as well as both cross-sectional and longitudinal feasibility are concerned [7], being also recommended within clinical trials as an outcome measure [8]. In fact, the MoCA samples from all of the abovementioned cognitive functions and domains are typically involved in PD [5,6,7].
However, with specific regard to the Italian scenario, the only two studies that focussed on the clinimetrics MoCA in non-demented PD patients—the first by Biundo et al. [9] and the second by Federico et al. [10]—are both lacking in relevant information and outdated.
Indeed, first, neither of these reports [9, 10] delivers evidence on the construct validity of the MoCA in this population—as both merely focussing on its diagnostic properties.
Moreover, although both Biundo et al. [9] and Federico et al. [10] overall provided data supporting the diagnostic value of the MoCA in non-demented PD patients, such studies either preceded the availability of demographically adjusted norms for the Italian MoCA (which were first delivered in 2015 [10,11])—as is the case for Biundo et al.’s [9] report—or addressed a limited sample size [6]—as is the case for Federico et al.’s [10] investigation, which included N = 43 patients. Moreover, Federico et al.’s [9, 10] study, which dates back to 2015, of course could not address the most recent normative dataset for the MoCA—which has been updated in 2021 [12]. Taken together, such stances imply that, in Italy, no up-to-date, generalizable PD-specific cut-off is available for the MoCA [13].
Given the relevance of delivering comprehensive and up-to-date clinimetric information for a given cognitive screener, to the aim of increasing their level of recommendation for clinical and research use [14], the present study aimed at: (1) assessing, in an Italian cohort of non-demented PD patients, the construct validity of the MoCA against both first- and second-level cognitive measures; (2) delivering an exhaustive and updated evaluation of its diagnostic properties.
Methods
Participants
Data on N = 237 PD patients—diagnosed according to the UK Brain Bank criteria [15]—consecutively referred to IDC-Hermitage Capodimonte, Napoli, Italy between 2013 and 2019 and having undergone the MoCA were retrospectively collected. Patients did not present with dementia according to the DSM-5 criteria for a Major Neurocognitive Disorder due to PD [16]. Moreover, patients were free from (1) further neurological/psychiatric diseases, (2) severe and/or unstable general-medical conditions that could have possibly affected cognition (i.e. system/organ failures and uncompensated metabolic/internal conditions) 3) uncorrected hearing/vision deficits.
Materials
All patients were administered the MoCA [13]. Patients assessed up to 2016 (N = 169) further underwent the Mini-Mental State Examination (MMSE) [17], whilst those assessed since 2017 (N = 68) were screened via the Parkinson’s Disease Cognitive Rating Scale (PD-CRS) [18]—the latter being is a 9-test, performance-based cognitive screening battery assessing attention/executive functioning, memory, language and visuo-spatial abilities/constructional praxis (range = 0–134). Out of patients assessed up to 2016 (N = 169), a subsample (N = 60) also underwent a second-level cognitive battery (Table 1) encompassing measures of attention and executive functioning (Trail-Making Test; Stroop Colour-Word Test; Phonemic Verbal Fluency; Backward Digit Span), memory (Rey Auditory Verbal Learning Test; Babcock Memory Test), language (Noun- and Verb-naming tasks from the Esame Neuropsicologico per l’Afasia; Semantic Verbal Fluency), praxis (Design Copy) and visuo-spatial abilities (Benton Judgment of Line Orientation) [19,20,21,22,23,24,25]. Figure 1 displays the abovementioned patient subsamples, defined according to which test they were administered.
Statistics
Skewness and kurtosis statistics were addressed to check for Normality on raw variables (with values >|1| and |3| being judged as abnormal, respectively) [26]; thus, based on such an assumption being met or not, either parametric or non-parametric techniques were employed to test associations/predictions of interest. Skewness and kurtosis values for each cognitive measure are shown in Supplementary Table 1.
The construct validity of the MoCA against the PD-CRS was tested via a Pearson’s coefficient by addressing age- and education-adjusted scores on both tests [13, 18], whilst that against each measure of the second-level cognitive battery via a set of Bonferroni-corrected Spearman’s correlations that addressed test raw scores and covaried for age, education and sex.
The diagnostics of both raw and adjusted [13] MoCA scores were examined via receiver-operating characteristics (ROC) analyses by addressing a below-cut-off, age- and education-adjusted MMSE score [17] as the positive state. Such an operationalization was indeed supported both by a previous Italian report demonstrating the suitability of the MMSE for cognitive screening aims in this population [6], and by the fact that, within the present cohort, the effect size for the association between both raw (rs(169) = 0.64; p < 0.001) and adjusted scores (rs(169) = 0.53; p < 0.001) on the two tests was large—according to Cohen’s [27] benchmark (i.e., ≥ 0.5). Sensitivity (Se), specificity (Sp), positive and negative predictive values (PPV; NPV) and likelihood ratios (LR + ; LR-) were computed at the optimal cut-off identified via Youden’s J statistic. The number needed for screening utility (NNSU) was also computed—with a value 1.02 ≤ being deemed as indexing an optimal screening performance [28].
Analyses were run with R 4.1 (R Core Team, 2012), jamovi 2.3 (the jamovi project, 2022) and IBM SPSS 27 (IBM Corp., 2020). The Partial Correlation macro of jamovi 2.3 was employed to compute partial Spearman’s coefficients.
Results
Table 2 summarises patients’ background, clinical and cognitive measures.
The MoCA was significantly associated with both PD-CRS scores (r(68) = 0.68; p < 0.001) and to the vast majority of second-level cognitive measures (Table 3).
Twenty-five out of 169 patients performed defectively on the MMSE (14.8%). Both raw and adjusted MoCA scores proved to be highly accurate to the aim of identifying patients with MMSE-confirmed cognitive dysfunctions (Fig. 2), also coming with sound diagnostics at the optimal cut-offs of ≤ 18 (J = 0.66; Se = 0.88; Sp = 0.78; PPV = 0.41; NPV = 0.97; LR + = 3.96; LR- = 0.15; NNSU = 0.89) and < 19.015 (J = 0.72; Se = 0.92; Sp = 0.80; PPV = 0.44; NPV = 0.98; LR + = 4.57; LR- = 0.10; NNSU = 0.84), respectively.
When applied to the whole sample, the two cut-offs yielded two moderately consistent classifications—with their agreement rate being of 88% (Cohen’s k = 0.74; z = 11.38; p < 0.001). Discrepancies were accounted for by 14 patients being classified as impaired by the raw cut-off but not by the adjusted one and by other 14 patients classified as impaired by the adjusted cut-off but not by the raw one.
Notably, when exploratively testing the diagnostic properties of the optimal cut-off values previously derived on raw scores by Biundo et al. [9]—i.e., ≤ 25—and by Federico et al. [10]—i.e., ≤ 24— for detecting mild cognitive impairment in PD, accuracy was low (Biundo et al. [9]: 29%; Federico et al. [10]: 36%) and nominal metrics were excessively biassed towards Se as well as overall unacceptable (Biundo et al. [9]: Se = 0.96; Sp = 0.17; PPV = 0.17; NPV = 0.96; LR + = 1.16; LR- = 0.23; NNSU = 3.06; Federico et al. [10]: Se = 0.96; Sp = 0.26; PPV = 0.18; NPV = 0.97; LR + = 1.29; LR- = 0.16; NNSU = 2.35). Consistently, an unlikely extreme proportion of patients was classified as impaired according to such cut-offs (Biundo et al. [9]: 88%; Federico et al. [10]: 81%).
Discussion
The present study provides Italian practitioners and clinical researchers with updated, and mostly unprecedented, evidence on the validity and diagnostic value of the MoCA in non-demented PD patients.
Indeed, the MoCA herewith proved to be an accurate estimate of cognitive efficiency in this population—as (1) being associated with a disease-specific measure of global cognition (i.e. the PD-CRS) and (2) converging with several second-level measures of both instrumental and non-instrumental cognitive domains/functions. Notably, such findings align with the international literature [5, 7].
As to its diagnostics, sound evidence has been herewith delivered on the fact that the MoCA is able to detect overall cognitive inefficiency (operationalized as a below-cut-off MMSE score) in non-demented PD patients. In this respect, it is worth noting that the cut-off derived on demographically adjusted MoCA scores slightly—but systematically—outperformed the “raw” cut-off. Such a finding is hardly surprising, given the undoubtable relevance of controlling for demographic confounders when interpreting test scores [14]. Hence, a MoCA score adjusted for age and education according to the most recent normative dataset [13] and falling below 19.015 should be addressed in the view of detecting cognitive impairment in this population.
In this respect, it is also worth noting that the raw cut-off herewith derived also clearly outperformed those previously suggested by Biundo et al. [9] and Federico et al. [10]—this strengthening, a posteriori¸ the rationale underlying the present study (i.e. the need of delivering up-to-date and generalizable thresholds).
The present study is of course not free from limitations.
The first, and the most relevant, issue lies in the operationalization herewith adopted for the positive state—i.e. a below-cut-off MMSE score. Indeed, albeit evidence on its feasibility as a cognitive screener in PD patients has been delivered in Italy [10], it has to be borne in mind that the MMSE should not represent the preferred choice to the aim of screening for cognitive impairment in this population—as recently highlighted by the MDS [5]. Indeed, the MMSE is mostly unable to capture executive deficits—which, however, typically characterise the cognitive profile of PD [31, 32]. Hence, it is mandatory that future studies aim at replicating the present findings by employing a different operationalization of the positive state—e.g. by means of a second-level cognitive battery or via a disease-specific cognitive screener (such as the PD-CRS). With that being said, the strong association detected between MMSE and MoCA scores within the current cohort, as well as the sound diagnostics herewith yielded, suggests that a sufficient degree of generalizability might be granted to the present cut-offs.
As to the second, it should be noted that PPVs proved to be relatively poor—even more if addressed within the context of the remaining metrics. However, such a finding is not surprising and does not in any case undermine the overall optimal diagnostic performance of the MoCA herewith detected. Indeed, at variance with the LR + , PPVs are heavily influenced by the prevalence of the positive state in the study sample: the lower the prevalence (such is the case for this investigation), the lower the PPV.
A final limitation lies in fact that the normative dataset herewith addressed, by Aiello et al. [13], has been derived from individuals in Northern Italy—whilst the present patient cohort was recruited in Southern Italy. In this respect, Aiello et al. [13] themselves noted that, in healthy subjects, the three normative datasets that are currently available for the MoCA in Italy, which comes from individuals residing in different Italian regions, might classify individuals’ cognitive status differently [13]. However, it should be noted that Aiello et al.’s [13] normative dataset includes a larger number of individuals and covers a wider range of age and education when compared to the previous ones published by Conti et al. [11] and Santangelo et al. [12] between 2014 and 2015—this having led the present Authors to opt for Aiello et al.’s [13]. Whilst it would be far beyond the aim of this study to do so, future investigations might aim at exploring these three normative datasets classify PD patients differently as to their normative status—similarly to what Salvadori et al. [33] recently did in stroke patients. Finally, it is worth mentioning the present PD cohort was not particularly heterogeneous as far as their age and level of educational attainment were concerned; hence, the cut-offs herewith provided might not be representative of all ranges of age and education, and future investigations should be aimed at filling such a gap.
In conclusion, the Italian MoCA is a valid and diagnostically sound screener for global cognitive inefficiency in non-demented PD patients, whose adoption is recommended within both clinical and research settings. Further studies are nevertheless needed that confirm its diagnostic values against a measure that is more appropriate than the MMSE.
Ethics
Participants provided informed consent. This study was approved by the Ethics Committee the University of Campania “Luigi Vanvitelli”.
Data availability
Datasets associated with the present study are available upon reasonable request of interested researchers.
References
Baiano C, Barone P, Trojano L et al (2020) Prevalence and clinical aspects of mild cognitive impairment in Parkinson’s disease: a meta-analysis. Mov Disord 35:45–54
Papagno C, Trojano L (2018) Cognitive and behavioral disorders in Parkinson’s disease: an update. I: cognitive impairments. Neurol Sci 39:215–223
Marras C, Rochon P, Lang AE (2002) Predicting motor decline and disability in Parkinson disease: a systematic review. Arch Neurol 59:1724–1728
Pedersen KF, Larsen JP, Tysnes OB et al (2013) Prognosis of mild cognitive impairment in early Parkinson disease: the Norwegian ParkWest study. JAMA Neurol 70:580–586
Skorvanek M, Goldman JG, Jahanshahi M et al (2018) Global scales for cognitive screening in Parkinson’s disease: Critique and recommendations. Mov Disord 33:208–218
Nasreddine ZS, Phillips NA, Bédirian V et al (2005) The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc 53:695–699
Julayanont P, Phillips N, Chertkow H, Nasreddine ZS (2017) The Montreal Cognitive Assessment (MoCA): Concept and Clinical Review. A.J. Larner (Ed.) Cognitive Screening Instruments: A Practical Approach (pp. 111–152). Springer
Chou KL, Amick MM, Brandt J et al (2010) A recommended scale for cognitive screening in clinical trials of Parkinson’s disease. Mov Disord 25:2501–2507
Biundo R, Weis L, Facchini S et al (2014) Cognitive profiling of Parkinson disease patients with mild cognitive impairment and dementia. Parkinsonism Relat Disord 20:394–399
Federico A, Maier A, Vianello G et al (2015) Screening for mild cognitive impairment in Parkinson’s disease: comparison of the Italian versions of three neuropsychological tests. Parkinson’s Dis 2015:681976
Conti S, Bonazzi S, Laiacona M et al (2015) Montreal Cognitive Assessment (MoCA)-Italian version: regression based norms and equivalent scores. Neurol Sci 36:209–214
Santangelo G, Siciliano M, Pedone R et al (2015) Normative data for the Montreal Cognitive Assessment in an Italian population sample. Neurol Sci 36:585–591
Aiello EN, Gramegna C, Esposito A et al (2022) The Montreal Cognitive Assessment (MoCA): updated norms and psychometric insights into adaptive testing from healthy individuals in Northern Italy. Aging Clin Exp Res 34:375–382
Aiello EN, Rimoldi S, Bolognini N et al (2022) Psychometrics and diagnostics of Italian cognitive screening tests: a systematic review. Neurol Sci 43:821–845
Hughes AJ, Daniel SE, Kilford L et al (1992) Accuracy of clinical diagnosis of idiopathic Parkinson’s disease: a clinico-pathological study of 100 cases. J Neurol Neurosurg Psychiatry 55:181–184
American Psychiatric Association (2013) Diagnostic and statistical manual of mental disorders(5th ed.). American Psychiatric Association
Carpinelli Mazzi M, Iavarone A, Russo G et al (2020) Mini-Mental State Examination: new normative values on subjects in Southern Italy. Aging Clin Exp Res 32:699–702
Santangelo G, Lagravinese G, Battini V et al (2017) The Parkinson’s disease-cognitive rating scale (PD-CRS): normative values from 268 healthy Italian individuals. Neurol Sci 38:845–853
Monaco M, Costa A, Caltagirone C et al (2013) Forward and backward span for verbal and visuo-spatial data: standardization and normative data from an Italian adult population. Neurol Sci 34:749–754
Barbarotto R, Laiacona M, Frosio R et al (1998) A normative study on visual reaction times and two Stroop colour-word tests. The Ital J Neurol Sci 19:161–170
Giovagnoli AR, Del Pesce M, Mascheroni S et al (1996) Trail making test: normative values from 287 normal adult controls. Neurol Sci 17:305–309
Carlesimo GA, Caltagirone C, Gainotti G and the MDB Group (1996) The mental deterioration battery: normative data, diagnostic reliability and qualitative analyses of cognitive impairment. Eur Neurol 36:378–384
Capasso R, Miceli G (2001) Esame Neuropsicologico per l'Afasia: ENPA. Springer Science & Business Media
Spinnler H, Tognoni G (1987) Standardizzazione e taratura italiana di una batteria di test neuropsicologici. Neurol Sci 6:S1–S120
Benton AL, Varney NR, Hamsher KD (1978) Visuospatial judgment: A clinical test. Arch Neurol 35:364–367
Kim HY (2013) Statistical notes for clinical researchers: assessing normal distribution (2) using skewness and kurtosis. Restor Dent Endod 38:52–54
Cohen J (1988) Statistical power analysis for the behavioral sciences, 2nd edn. Erlbaum, Hillsdale, NJ
Larner AJ (2019) New unitary metrics for dementia test accuracy studies. Prog Neurol Psychiatry 23:21–25
Fahn S, Elton RL (1987) UPDRS program members. Unified Parkinson’s disease rating scale. In: Fahn S, Marsden CD, Goldstein M, Calne DB (eds) Recent developments in Parkinson’s disease, pp 153–163
Jankovic J, McDermott M, Carter J et al (1990) Variable expression of Parkinson’s disease: a base-line analysis of the DAT ATOP cohort. Neurology 40:1529–1529
Hoops S, Nazem S, Siderowf AD et al (2009) Validity of the MoCA and MMSE in the detection of MCI and dementia in Parkinson disease. Neurology 73:1738–1745
Chou KL, Lenhart A, Koeppe RA et al (2014) Abnormal MoCA and normal range MMSE scores in Parkinson disease without dementia: cognitive and neurochemical correlates. Parkinsonism Relat Disord 20:1076–1080
Salvadori E, Cova I, Mele F et al (2022) Prediction of post-stroke cognitive impairment by Montreal Cognitive Assessment (MoCA) performances in acute stroke: comparison of three normative datasets. Aging Clin Exp Res 34:1855–1863
Acknowledgements
The authors are thankful to patients and their caregivers.
Funding
Open access funding provided by Università degli Studi della Campania Luigi Vanvitelli within the CRUI-CARE Agreement.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
AD, ENA, MA, CV, FV, AC and GS declare that they have no conflict of interests. VS received compensation for consulting services and/or speaking activities from AveXis, Cytokinetics, Italfarmaco, Liquidweb S.r.l., and Novartis Pharma AG, receives or has received research supports from the Italian Ministry of Health, AriSLA, and E-Rare Joint Transnational Call. He is in the Editorial Board of Amyotrophic Lateral Sclerosis and Frontotemporal Degeneration, European Neurology, American Journal of Neurodegenerative Diseases, Frontiers in Neurology and Exploration of Neuroprotective Therapy. BP received compensation for consulting services and/or speaking activities from Liquidweb S.r.l BP is Associated Editor for Frontiers in Neuroscience. NT received compensation for consulting services from Amylyx Pharmaceuticals and Zambon Biotech SA. He is Associate Editor for Frontiers in Aging Neuroscience.
Statement of human and animal rights
There are no human and animal right issues to declare.
Informed consent
All participants provided written informed consent.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
D’Iorio, A., Aiello, E.N., Amboni, M. et al. Validity and diagnostics of the Italian version of the Montreal Cognitive Assessment (MoCA) in non-demented Parkinson’s disease patients. Aging Clin Exp Res 35, 2157–2163 (2023). https://doi.org/10.1007/s40520-023-02493-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40520-023-02493-w