"Never doubt that a small group of thoughtful, committed citizens can change the world. Indeed, it is the only thing that ever has."

Margaret Mead
Original article

Diagnostic Reliability of Leptomeningeal Disease Using Magnetic Resonance Imaging


Brain metastases are seen in 20%-50% of patients with metastatic solid tumors. On the other hand, leptomeningeal disease (LMD) occurs more rarely. The gold standard for the diagnosis of LMD is serial cerebrospinal fluid (CSF) analyses, although in daily practice, the diagnosis of LMD is often made by neuroimaging. Leptomeningeal metastases (LM) have been a relative contra-indication to radiosurgery. It can be noted that focal LMD can be difficult to distinguish from a superficially located/cortical-based brain metastasis which is not a contra-indication for radiosurgery. Hence, justifying the need of a reliable diagnosis method.

The goal of this study was to determine the inter-observer reliability of contrast-enhanced magnetic resonance imaging (gdMRI) in the differentiation of focal cortical-based metastases from leptomeningeal spread.

This is a retrospective review of a prospectively collected database of patients with brain metastases. A total of 42 cases with superficial lesions were selected for review. Additionally, eight control cases demonstrating deep and/or white-matter based lesions were included in the study.

Three neuroradiologists and three radiation oncologists were asked to review each study and score the presence of LM. Inter-observer agreement was calculated using group-derived agreement coefficients (Gwet’s AC1 and Gwet's AC2). Pair-wise inter-observer agreement coefficients never reached substantial values for trichotomized outcomes (LMD, non-LMD or indeterminate) but did reach a substantial value in a minority of cases for dichotomised outcomes (LMD or non-LMD). The control subgroup analysis revealed substantial agreement between most pairs for both trichotomized and dichotomised outcomes.

We observed low inter-observer agreement amongst specialists for the diagnosis of focal LMD by gdMRI. Neuroimaging should not be relied upon to make treatment decisions, notably to deny patients radiosurgery.


Brain metastases occur in 20%-50% of patients with advanced cancer. Leptomeningeal disease (LMD) is diagnosed in 5%-10% of patients with otherwise metastatic cancer, although rates approaching 20% have been reported in autopsy series [1-2]. The gold standard for the diagnosis of LMD is serial cerebrospinal fluid (CSF) analyses, which has a sensitivity of approximately 55% on the first CSF examination and reaches up to 85% sensitivity when done thrice - while maintaining a theoretical specificity of 100% [3-4]. The presence of malignant cells in a sample of CSF, cranial nerve palsies or widespread meningeal enhancement (particularly in the cerebellar folia) is convincing evidence of a process that is not amenable to focal treatment. On the other hand, nodular lesions described as being of leptomeningeal origin can cause more therapeutic ambiguity. In current practice, diagnosis of LMD is also often made without CSF analysis, using gadolinium-enhanced magnetic resonance imaging (gdMRI) of the brain and/or spinal cord, which is reported to have a sensitivity and specificity of approximately 75% [5-6]. LMD thus diagnosed can represent a relative contra-indication to stereotactic radiosurgery.

The goal of this study is to characterize the inter-observer reliability of gdMRI to differentiate focal cortical-based metastases from focal LMD in the context of known metastatic disease.

Materials & Methods

This is a retrospective study of cerebral magnetic resonance imaging (MRI) studies of patients suspected to have focal leptomeningeal metastases (LM). All patients included in this study underwent radiosurgery between 2009 and 2013. The study protocol was approved by our institutional ethics review board.

We completed a retrospective review of a prospectively collected database of 438 patients with known primary disease and neurological symptoms referred for evaluation to our tertiary care center radiation-oncology clinic between May 2009 and June 2013. Of these patients, 174 had a reviewable planning MRI prior to intervention in radiation-oncology. These pre-intervention studies were screened randomly by a single author (FG, neuroradiologist with 15 years of experience) until 42 cases were selected for review, while blinded to the original imaging report. The sole inclusion criteria was the presence of one or more enhancing cortical-based lesions. Cases demonstrating widespread leptomeningeal or cranial nerve enhancement, as well as cases with previous radiosurgery or surgery were excluded from the study, as were cases with suboptimal study quality. Additionally, eight control cases were included in the study, consisting of eight patients demonstrating only deep and/or white-matter based lesions.

Readers included three neuroradiologists (two senior staff members (5-18 years of experience) and a neuro-radiology fellowship trainee) and three radiation oncologists with active radiosurgery practices (5-15 years of experience).

All MRIs studies were independently reviewed by each of our six examiners using our institution’s Picture Archiving and Communication System (PACS), and were assigned to one of three diagnostic categories (LMD, non-LMD, indeterminate). To establish a diagnosis per specialty, cases who received the same diagnostic label by two readers were assigned to that category, while cases who received three different diagnoses were re-evaluated by the three specialists to establish a consensus diagnosis.

In addition to analysing the data using a ternary scale (LMD, non-LMD, indeterminate), statistical analyses were also performed using a binary scale, in which cases assigned to the “indeterminate” category were clustered with cases assigned to the “non-LMD” category, based on the assumption that patients without a firm diagnosis of LMD would be treated in a similar fashion in regards to radiotherapy.

Inter-rater coefficients were calculated between each pair of readers using Gwet’s AC1 and AC2, for dichotomized and trichotomized ratings respectively, due to their more robust nature, allowing inter-observer analyses on data demonstrating a low prevalence index and presence of constants, notably in the controls. Gwet’s AC1 and AC2 was also used to derive group inter-rater coefficients amongst the entire cohort, as well as subgroups (amongst radiologists, radio-oncologists and between the two specialties, using the consensus agreements described above).

Agreement coefficients are interpreted using the highest benchmark level that is associated with the smallest cumulative membership probability (CMP) that exceeds 95% [7]. The specific benchmark was the one described by Landis and Koch [8] (< 0, poor agreement; 0-0.20, slight agreement; 0.21-0.40, fair agreement; 0.41-0.60, moderate agreement; 0.61-0.80, substantial agreement; and 0.81-1.00, almost perfect agreement).


While some cases achieved an agreement by all readers, some other cases presented more ambiguous lesions thus causing division amongst readers (Figures 1-2).

In trichotomized ratings, only one out of the 42 cases and only three out of the eight controls were assigned to the same diagnostic category by all six raters. All other studies were assigned to two different diagnostic categories (out of three) by at least two raters, with variable agreement amongst the rest of the raters. In dichotomized ratings, these numbers rose to nine out of 42 cases and four out of eight controls. In both trichotomized and dichotomized ratings, only the non-leptomeningeal category was assigned by all six raters. No consensus was obtained within the leptomeningeal category.

Ternary inter-rater agreement coefficients amongst pairs of raters ranged from poor (AC1 = -0.16; 95% CI: -0.50, 0.19) to fair (AC1 = 0.56; 95% CI: 0.34, 0.79), while they ranged from poor (AC1 = 0.18; 95% CI: 0, 0.49) to substantial (AC1 = 0.80; 95% CI: 0.66, 0.95) in binary analyses. Using cumulative membership probabilities, only one inter-rater coefficient reached a substantial value for the dichotomized ratings, while none did so for the trichotomized ratings.

In the control subgroup analysis, inter-rater agreement coefficients amongst pairs of raters ranged from poor (AC2 = 0.25; 95% CI: -0.55, 1.06) to almost perfect (AC2 = 0.94; 95% CI: 0.82, 1.06) for trichotomized ratings, while they ranged from poor (AC1 = 0.34; 95% CI: -0.37, 1.058) to perfect agreement (AC1 = 1.00) for dichotomized ratings. Out of 15 pairs of raters, seven demonstrated an inter-rater agreement coefficient > 0.61 for trichotomized ratings; 11 for dichotomized ratings. 

Interrater agreement coefficient amongst the entire cohort was slight (AC2 = 0.19; 95% CI: 0.01, 0.38) for trichotomized ratings and fair (AC1 = 0.43; 95% CI: 0.28, 0.59) for dichotomized ratings. By specialty, inter-rater agreement coefficients amongst radiologist was slight (AC2 = 0.35; 95% CI: 0.14, 0.57) for trichotomized ratings and moderate (AC1 = 0.66; 95% CI: 0.50, 0.82) for dichotomized ratings, while radio-oncologists demonstrated poor (AC2 = 0.17; 95% CI: -0.05, 0.40) and slight agreement (AC1 = 0.34; 95% CI: 0.13, 0.54) for trichotomized and dichotomized ratings, respectively (Table 1). Agreement between the two specialties was poor for both trichotomized (AC2 = -0.13; 95% CI: -0.34, 0.084) and dichotomized (AC1 = 0.02; 95% CI: -0.150, 0.193) ratings (Table 2). For controls, group-derived inter-rater agreement coefficients reached a substantial value for the entire cohort, for both trichotomized (AC2 = 0.60; 95% CI: 0.18, 1.02) and dichotomized (AC1 = 0.68; 95% CI: 0.32, 1.04) ratings.

  Coefficient Inference/Subjects
StdErr 95% C.I. Interpretation using CMPs
Radiologists (n=3) Ternary Gwet's AC2 0,35452 0,10929 0.135 to 0.574 Slight
Binary Gwet's AC1 0,66208 0,07901 0.503 to 0.821 Moderate
Radio-oncologists (n=3) Ternary Gwet's AC2 0,17477 0,10993 -0.046 to 0.396 Poor
Binary Gwet's AC1 0,33685 0,10196 0.132 to 0.542 Slight
Cohort (n=6) Ternary Gwet's AC2 0,19222 0,09073 0.011 to 0.375 Slight
Binary Gwet's AC1 0,43436 0,07789 0.278 to 0.591 Fair
  Coefficient Inference/Subjects
StdErr 95% C.I. Interpretation using CMPs
Radiologists (n=3) vs Radio-oncologists (n=3) Ternary Gwet's AC2 -0,12700 0,10546 -0.338 to 0.084 Poor
Binary Gwet's AC1 0,02135 0,08579 -0.150 to 0.193 Poor


Cortical-based metastases are difficult to differentiate radiologically from focal leptomeningeal carcinomatosis. Our study demonstrates significant variability in the interpretation of studies demonstrating cortical-based enhancing disease, both amongst clinicians and radiologists.

However, inter-rater agreement was nonetheless higher amongst neuro-radiologists (moderate) than radio-oncologists (slight), suggesting increased reliability of certain radiological signs, best evaluated with radiological expertise.

LMD remains a clinical diagnosis, based on clinical symptoms, imaging, and cerebrospinal fluid analysis. Presence of CSF cytology remains the gold standard, with 60% sensitivity on initial puncture, 80% total sensitivity on second puncture and 2%-5% sensitivity increase per following repeated collection [9]. Most diagnostic algorithms recommend initial evaluation with a CSF profile, CSF cytology and gdMRI of the neuroaxis, with at least one additional lumbar puncture if initial CSF cytology is negative [9-12]. Meanwhile, the frequency of LM-related MRI abnormalities for solid tumors on neuroaxis imaging varies from 56% to 65%, depending on the primary [13]. Although in some cases, the constellation of clinical symptoms of LMD combined with bulky LMD on neuroaxis imaging may obviate the need for lumbar puncture, our study shows that, in more mitigated cases, focal cortical-based metastases are difficult to differentiate from focal LMD [3-4,6]. In such cases, we advocate that imaging findings should be correlated by CSF analysis, serial of need be, before LMD can be diagnosed, in large part because of the implication for treatment, most notably radiotherapy.

Although there is no universally accepted standard treatment for LMD, whole-brain radiotherapy (WBRT) is often considered in this patient population. While some cases can benefit from WBRT, this treatment does expose healthy brain tissue to high doses of radiations [14]. Adverse effects of cerebral radiation therapy may include neurocognitive disorders, radionecrosis and, rarely, radiation-associated meningiomas (RAM) [15]. Alternatively, stereotactic surgery (SRS) achieve high rate of local control while diminishing radiation exposure to the rest of the brain [2].

Ross et al. outlined the importance of a clearly established diagnostic before exposing patients to SRS or WBRT, as the implications of misdiagnosis might either be radiation overexposure or missed tumors [16]. Flickinger et al. argue that imaging alone represents an accepted diagnostic tool although they report a 2.3% chance of misdiagnosis, when imaging is not confirmed by CSF analysis [17]. Alternatively, Wolf et al. suggest that the use of SRS might successfully treat focal LMD and delay WBRT for some patients [2]. Consequently, radiosurgical approach has been used as a treatment for focal LMD [2,18].

Therefore, because LMD is often difficult to differentiate from cortical based metastases and because initial focal radiotherapy may delay whole-brain radiotherapy, we argue that patients with a known solid primary neoplasm and equivocal imaging findings for LMD should still be considered for stereotactic surgery initially.


The diagnosis of nodular LMD as opposed to cortical-based brain metastases on gadolinium-enhanced brain MRI demonstrates low inter-observer agreement amongst specialists, both radiologist and radio-oncologists. If imaging findings may alter the treatment approach, they should at least be confirmed by serial CSF analysis.


  1. Bruna J, Gonzalez L, Miro J, Velasco R, Gil M, Tortosa A: Leptomeningeal carcinomatosis: prognostic implications of clinical and cerebrospinal fluid features. Cancer. 2009, 115:381-389. 10.1002/cncr.24041
  2. Wolf A, Donahue B, Silverman JS, Chachoua A, LeeJK, Kondziolka D: Stereotactic radiosurgery for focal leptomeningeal disease in patients with brain metastases. J Neurooncol. 2017, 134:139-143. 10.1007/s11060-017-2497-6
  3. Chamberlain MC: Leptomeningeal metastasis. Curr Opin Oncol. 2010, 22:627-635. 10.1097/cco.0b013e32833de986
  4. Groves MD: Leptomeningeal disease. Neurosurg Clin N Am. 2011, 22:67-78. 10.1016/j.nec.2010.08.006
  5. Costa R, Kumthekar P: Management of central nervous system metastases in breast cancer. The Breast (Fifth Edition). Bland KI, Copeland EM, Klimberg VS, Gradishar WJ (ed): Elsevier, Amsterdam; 2018. 942-960.
  6. An YJ, Cho HR, Kim TM, et al.: An NMR metabolomics approach for the diagnosis of leptomeningeal carcinomatosis in lung adenocarcinoma cancer patients. Int J Cancer. 2015, 136:162-171. 10.1002/ijc.28949
  7. Gwet KL: Handbook of Inter-Rater Reliability, 4th Edition: The Definitive Guide to Measuring The Extent of Agreement Among Raters. Advanced Analytics, LLC, Gaithersburg; 2014.
  8. Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1977, 33:159-174. 10.2307/2529310
  9. Nayar G, Ejikeme T, Chongsathidkiet P, et al.: Leptomeningeal disease: current diagnostic and therapeutic strategies. Oncotarget. 2017, 8:73312-73328. 10.18632/oncotarget.20272
  10. Chamberlain M, Junck L, Brandsma D, et al.: Leptomeningeal metastases: a RANO proposal for response criteria. Neuro Oncol. 2017, 19:484-492. 10.1093/neuonc/now183
  11. Gleissner B, Chamberlain MC: Neoplastic meningitis. Lancet Neurol. 2006, 5:443-452. 10.1016/S1474-4422(06)70443-4
  12. Kesari S, Batchelor TT: Leptomeningeal metastases. Neurol Clin. 2003, 21:25-66. 10.1016/S0733-8619(02)00032-4
  13. Chamberlain MC: Comprehensive neuraxis imaging in leptomeningeal metastases: a retrospective case series. CNS Oncol. 2013, 2:121-128. 10.2217/cns.12.45
  14. Zindler JD, Bruynseel AM, Eekers DB, Hurkmans CW, Swinnen A, Lambin P: Whole brain radiotherapy versus stereotactic radiosurgery for 4-10 brain metastases: a phase III randomised multicentre trial. BMC Cancer. 2017, 17:500. 10.1186/s12885-017-3494-z
  15. Aboukais R, Zairi F, Le Rhun E, Lejeune JP, Devos P, Reyns N: Radiation-associated grade 2 meningeiomas: A nine patient-series and review of the literature. Clin Neurol Neurosurg. 2015, 136:10-14. 10.1016/j.clineuro.2015.05.022
  16. Roos DE, Brophy BP, Taylor J: Lessons from a 17-year radiosurgery experience at the Royal Adelaide Hospital. Int J Radiat Oncol Biol Phys. 2012, 82:102-106. 10.1016/j.ijrobp.2010.08.054
  17. Flickinger JC, Kondziolka D, Maitz AH, Lunsford LD: Gamma knife radiosurgery of imaging-diagnosed intracranial meningioma. Int J Radiat Oncol Biol Phys. 2003, 56:801-806. 10.1016/S0360-3016(03)00126-3
  18. Ganau M, Foroni RI, Gerosa M, Zivelonghi E, Longhi M, Nicolato A: Radiosurgical options in neuro-oncology: a review on current tenets and future opportunities. Part I: therapeutic strategies. Tumori Journal. 2014, 100:459-465.
Original article

Diagnostic Reliability of Leptomeningeal Disease Using Magnetic Resonance Imaging

Author Information

Philippe Harris Corresponding Author

Miscellaneous, Laval University, Montréal, CAN

Ange Diouf

Radiology, University of Montreal Health Centre, Montréal, CAN

François Guilbert

Radiology, University of Montreal Health Centre, Montréal, CAN

Fatima Ameur

Radiology, University Hospital of Lyon, Lyon, FRA

Laurent Letourneau-Guillon

Radiology, University of Montreal Health Centre, Montréal, CAN

Cynthia Ménard

Radiation Oncology, University of Montreal Health Centre, Montréal, CAN

Laura Masucci

Radiation Oncology, University of Montreal Health Centre, Montréal, CAN

Manon Bélair

Radiology, University of Montreal Health Centre, Montréal, CAN

David Roberge

Radiation Oncology, University of Montreal Health Centre, Montréal, CAN

Ethics Statement and Conflict of Interest Disclosures

Human subjects: Consent was obtained by all participants in this study. Comité d'éthique de la recherche en arts et en sciences du Centre Hospitalier de l'Université de Montréal (CÉRAS) issued approval 15.111. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.

Original article

Diagnostic Reliability of Leptomeningeal Disease Using Magnetic Resonance Imaging

Figures etc.


Scholary Impact Quotient™ (SIQ™) is our unique post-publication peer review rating process. Learn more here.