Reliability and validity of Arabic version of the brief international cognitive assessment for multiple sclerosis: Egyptian dialect

Abstract

Background

Given the diversity of multiple sclerosis (MS) symptoms including cognitive impairment in certain domains, the need to develop a rapid and at the same time thorough tool for cognitive assessment is mandatory and represents an unmet need in the clinical and research fields of MS. The Brief International Cognitive Assessment for MS (BICAMS) is a good and practical tool to achieve this mission but is not present in the Arabic language for Arabic speaking countries yet.

Objectives

To assess the reliability and validity of Arabic version of the BICAMS (Egyptian dialect).

Methods

Ninety Egyptian MS patients and 85 matched healthy controls underwent neuropsychological testing using the BICAMS Arabic version (Egyptian dialect) battery including the Symbol Digit Modality Test (SDMT), California Verbal Learning Test 2nd edition (CVLT-II), and revised Brief Visuospatial Retention Test- (BVRT-R). Test–retest data were obtained from MS patients 2 weeks after the initial assessment. Mean differences between both groups were assessed controlling for age, gender, and educational level.

Results

The MS patients scored significantly lower on the SDMT, CVLT-II, and BVMT-R tests compared to healthy controls (p<0.001). For MS patients’ group, intra-observer (test–retest) reliability was satisfactory for SDMT, CVLT-II total, and BVRT-R total with r values of 0.85, 0.61, and 0.68, respectively.

Conclusion

BICAMS Arabic version is a reliable and valid tool for cognitive assessment of Arabic speaking MS patients in different clinical and research settings.

Introduction

Multiple sclerosis (MS) is a common neurological disease which causes physical, psychological, and cognitive disabilities. MS is associated with high rates of unemployment in Egypt [1], the prevalence of MS in Egypt has been shown to be 13.7/100,000 and 25/100,000 in two studies, respectively [2, 3], and so early assessment to manage factors that increase disability such as cognitive impairment is crucial. The burden of MS on the patients’ cognitive function [4] is well established; MS patients show greater impairments on tests of non-verbal intellectual ability, processing speed, and selective/focused attention, verbal-recall, and verbal fluency skills [5, 6]. Assessment of cognition in MS started late in the 1980s when Rao and colleagues started addressing information processing speed slowing in patients with MS. [7] Consequently, several cognitive batteries for MS patients were developed as “Brief Repeatable Battery (BRB) of Neuropsychological Tests,” [8] “MS Functional Composite” (MSFC), [4] “Minimal Assessment of Cognitive Function in MS” (MACFIMS), [9] “MS-Cog,” [10] and the National Institute of Neurological Disorders and Stroke (NINDS) has a series of recommended assessments across multiple clinical domains for several neurological diseases, called “Common Data Elements” (CDE) and MS standards [11]. Most recently, the “Brief International Cognitive Assessment for MS” (BICAMS) has been recommended [12, 13]. BICAMS is a validated test for detection of cognitive impairment in MS patients that can be completed by a neurologist. Symbol Digit Modalities Test (SDMT), [14] the five initial learning trials of the second edition of the California Verbal Learning Test (CVLT-II), [15] and the revised Brief Visuospatial retention Test (BVRT-R) [16] are well established psychometric tests and are of documented good face validity and consistent stimulus presentation.

The Egyptian Arabic dialect is the most recognized and widely understood dialect by Arabic speakers around the world. Although there are several neuropsychological tests in classical Arabic language, lack of Arabic tests for MS matching Egyptian culture and dialect is an obstacle in neuropsychological assessment of patients in Egypt. In this study, our objective was to assess the reliability and validity of the Arabic translation of the BICAMS test.

Methods

Patients were recruited from Kasr Al-Ainy  MS clinic in Cairo University consecutively between March and July 2017. Around 817 patients with MS diagnosis based on the revised McDonald’s criteria [17] visited the MS clinic during this period either for regular follow-up visits or in an attack, only patients fulfilling inclusion criteria were recruited. The inclusion criteria were patients 18–55 years of age, with no evidence of relapse or steroids intake during the previous 4 weeks before enrollment. Patients were excluded if they were illiterate, or had a history of any neuropsychiatric disorder other than MS, a history of drug abuse or cognitive enhancing medication, or having systemic diseases or metabolic disorders that may impair cognition, any visual, or hearing problem that could interfere with performance of the test, or EDSS ≥7, as well as those residing in governates other than Great Cairo (as they may not be able to come back in the retesting step). All patients participated voluntarily. Age, gender, years of education, MS subtype, age of onset, Expanded Disability Status Scale (EDSS), and disease duration were recorded. Figure 1 shows the flowchart of patients’ recruitment.

Fig. 1

Flowchart showing the recruitment process of patients with multiple sclerosis for the study

Ninety patients were recruited and compared with 85 healthy participants in the control group. The healthy control (HC) group was recruited by an advertisement placed in the hospital calling for volunteers to participate in a study directed to cognitive assessment. Participants included paramedical staff, caregivers, and patients’ relatives (first-degree relatives were excluded). Volunteers with history of drug abuse or history of psychiatric disease or medication that interferes with cognition were excluded.

The test was performed by the clinical staff in  Kasr Al-Ainy  MS clinic; all clinical staff participating in this study attended a training session to ensure unified administration procedures and data recording. The scoring of the test for all participants was performed by the same researcher. All participants provided informed written consent to all procedures. The study was approved by the neurology ethical scientific committee, neurology department, Cairo University, February 2017. This committee does not provide a reference number.

BICAMS was performed for all patients in the morning in a quiet room during patient’s scheduled regular visits; it included three tests: Symbol Digit Modalities Test (SDMT), the California Verbal Learning Test (CVLT-II), and the revised Brief Visuospatial retention Test (BVRT-R). The tests were administered in the same sequence: SDMT, CVLT-II, first 5 trials, and BVRT-R, first 3 recall trials. We followed the suggested international validation standards of BICAMS [13].

Step (1): Standardization and Translation of Test Stimuli

BICAMS was translated and culturally adapted into the Arabic language. The SDMT (oral) [18] presents a series of nine symbols, and each paired with a single digit in a key at the top of a standard sheet of paper. An adapted version of the test is used and numbers used for instructions and response were in the Arabic language. Participants were asked to say the digit associated with each symbol as rapidly as possible for 90 s. The number of correct responses in 90 s was recorded. The CVLT-II [15]: the CVLT-II list of words was translated and retranslated from English to Arabic by a professional translator. Four semantic categories were used: cooking utensils, vegetables, clothes, and tools. Cooking utensils were chosen because this matches Egyptian culture familiarity more than different types of sports. Words of average familiarity and frequency in Egypt culture were used (Table 1). The total number of recalled items over the five learning trials was calculated (CVLT TL). Visual/spatial memory is assessed in BICAMS using the BVRT-R [16], and for visual stimuli, six abstract designs that have no semantic associations to stimuli in the culture or language in Egypt were used. The total score of the three trials was calculated (BVRT TL).

Table 1 Standardization of California Verbal Learning Test using familiar and frequent words in Egyptian culture

Step (2): Standardization and Translation of Test Instructions

All information from the test manual necessary for administration and interpretation was translated, back translated, and checked for errors in Arabic language (Egyptian Dialect). All examiners were trained to use standardized instructions in Arabic language.

Step (3): Test–retest reliability

Forty-eight MS patients and 44 healthy volunteers were assessed on two occasions separated by 2 weeks to test reliability. All tests and procedures were identical to the first visit with the same examiner, and all retests were scored with the same rater who scored the first visit test. Only intra-rater assessment was carried out without inter-rater comparisons as this was the discipline adopted by the international group of BICAMS.

Step (4): Criterion-related validity

The MS sample was compared to a healthy control group to determine if BICAMS is sensitive to MS disease state. Impairment on individual tests was defined as −1.5 standard deviation (SD) below reference group means.

Data was coded and entered using the IBM statistical package SPSS (Statistical Package for the Social Sciences) for windows, version 24, released 2016, Armonk, New York. Data was summarized using mean, standard deviation, median, minimum, and maximum in quantitative data and using frequency (count) and relative frequency (percentage) for categorical data. Comparisons between quantitative variables were done using the non-parametric Kruskal-Wallis and Mann-Whitney tests. For comparison of serial measurements within each patient, the non-parametric Wilcoxon signed rank test was used for comparing categorical data, chi squared (χ²) test was performed. Exact test was used instead when the expected frequency is less than 5. Correlations between quantitative variables were done using Spearman correlation coefficient. Correlation of BICAMS test–retest scores was evaluated using Pearson correlation coefficient r. P values less than 0.05 were considered as statistically significant.

Results

Patients with MS were matched with the HC group with regard to age, sex, and years of education. There were no significant differences between the MS and HC group on age (MS mean 30.8 years, SD 6.7, range 19–52; HC mean 30.5 years, SD 7.9, range 19–52; P=0.41); gender (MS 70 females, 20 males; HC 60 females, 25 males; P=0.22); or education (MS mean 14.5 years, SD 2.6, range 8–10; HC mean 14.3 years, SD 3.3, range 11–24; P=0.21. Patients included were 76 (87.3%) diagnosed with relapsing remitting (RR) MS, 12 (13.3%) with secondary progressive (SP) MS, and 2 (2.2%) with primary progressive (PP) MS. The mean age of onset in patients was 25.24 (SD 7.1), while the mean disease duration was 6.2 (SD 5.8), EDSS ranged from 1 to 6.5 with mean 2.8 (SD 1.8).

Test validity

The test time in HC ranged from 6 to 25 min with a median of 12 min (mean was 13.27 ± 4.22), and no statistically significant difference was found in time consumed in test performance between control and MS patients in which duration ranged from 7 to 50 min with median 14 min (mean was 14.95 ± 5.75).

All BICAMS subtests discriminated between MS patients and HC (Table 2). MS patients scored significantly lower on the SDMT, CVLT total (TL), and BVRT-R total (TL) tests compared to healthy controls with P values of <0.001, <0.001, and 0.001, respectively, which supports the validity of the Arabic BICAMS. BICAMS subtests were categorized as “impaired” and “not impaired”; with impairment defined as −1.5 SD below (cutoff scores used were 34 in SDMT, 46 in CVLT TL, and 12 in BVRT-R TL). A significant number of MS patients showed impairment in all BICAMS subtests in comparison to controls; this was more pronounced in SDMT where 31% of patients were affected (Table 3).

Table 2 Comparison between MS patients versus healthy controls on BICAMS

Table 3 Comparison between frequency of impaired scores for MS patients and healthy controls on BICAMS subtests (using chi-squared test)

Patients’ variables including age, education, disease duration, age of onset of disease, and disability measured by EDSS were tested in relation to performance of BICAMS subtests (Table 4). Formal education level had a significant influence on all BICAMS subtests; SDMT (P=0.001), CVLT (P=0.014), and BVRT-R (P=0.02). Both age of the patient and duration of illness affected patients’ performance in SDMT and BVRT-R, but not in CVLT. On the other hand, the age of onset of disease had no significant impact on the performance of any test in our sample. In addition, degree of disability correlated negatively with patients’ performance on the SDMT (P=0.001) and CVLT (P=0.007), but not BVRT-R (P=0.11).

Table 4 Spearman’s correlations between clinical characteristics of the patients and BICAMS subtests

Test reliability

Test–retest reliability was assessed in all subjects who were recruited for retest (Table 5). The test–retest reliability coefficients for each test were as follows: SDMT: r = 0.85; CVLT-II: r = 0.65; and BVRT-R: r = 0.75 (p < 0.0001).

Table 5 Pearson’s correlation coefficients between the tests and the retests

Discussion

BICAMS is a reliable and valid test that fills a gap in neuropsychological assessment of patients with MS in Arabic speaking population, as proven by our results. It offers a rapid assessment tool, usually around 15 min, that is easily completed in an outpatient clinic by the clinical staff. Translation and validation of the BICAMS is established in several countries [19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35] which highlights its importance as a unified brief international assessment tool in patients with MS worldwide (Table 6).

Table 6 Previous studies validating BICAMS

We found a significant difference between MS patients and HC group in all BICAMS Arabic version subsets (P value < 0.001). In this study, the level of education affected the performance in all BICAMS Arabic version subsets; also, the age of the patient, duration of illness, and level of disability (but not age of onset) contributed negatively to the performance on different tests.

All validation studies adopted the −1.5 SD of mean in determining cut-off point apart from one study by Caneda and colleagues who used −1 SD for cut-off [23]. In our study, the cutoff scores according to 1.5 SD below the mean were 34 in SDMT, 46 in CVLT TL, and 12 in BVMT-R TL; which is different from cut off points suggested recently by Beier and colleagues for 1.5 SD below the mean which is 44 for SDMT; 39 for CVLT TL score and 17 for BVRT TL score [36]. Difference in cutoff points used was subsequently associated with difference in percent of patients impaired in tests observed in different validation studies which ranged from 28 to 67% for SDMT; 20–73% CVLT total score and 10–58% for BVRT total score [20, 24, 26, 28, 31]. In this study, 31% of patients showed impairment in SDMT, 20% CVLT total score, and 21% for BVRT total score. A possible explanation is the younger age, shorter disease duration, and less disability in our patients. A recent study by Hamdy and colleagues reported that the mean age of onset in Egypt is slightly lower than the reported estimates in the middle east and north Africa region (1); the mean of age of patients included in this study is 30.8 (SD 6.7), which is younger than the mean of most previous validation studies that ranged from 34 to 66.8 years (Table 6); also, the duration of illness was less than most other validation studies where mean duration of illness ranged from 8 to 13.1 years while in our study mean disease duration was 6.2 (SD 5.8); and mean EDSS in our study was 2.8 (SD 1.8) while in other studies mean of EDSS ranged from 2.5 to 4.2 (Table 6).

Variability in clinical characteristics of participants in different validation studies in addition to different languages, ethnicity, nationality, and culture contributed to the differences in results. Some of the differences between levels of impairment reported in different validation studies can be explained by differences in the validation sample, for example, our patient sample is younger. However, despite the proven validity and reliability of BICAMS in all previous studies, individual national validations of BICAMS according to language and culture are required [37].

Limitations

Many subjects dropped out in the retesting step which limited the test–retest comparisons.

Conclusion

BICAMS Arabic version is a reliable and valid tool for cognitive assessment of Arabic speaking MS patients in different clinical and research settings.