Articles

Diagnostic accuracy of the Salzburg EEG criteria for

non-convulsive status epilepticus: a retrospective study
Markus Leitinger, Eugen Trinka, Elena Gardella, Alexandra Rohracher, Gudrun Kalss, Erisela Qerama, Julia Höfler, Alexander Hess,
Georg Zimmermann, Giorgi Kuchukhidze, Judith Dobesberger, Patrick B Langthaler, Sándor Beniczky

Summary
Lancet Neurol 2016; 15: 1054–62 Background Several EEG criteria have been proposed for diagnosis of non-convulsive status epilepticus (NCSE), but
See Comment page 1001 none have been clinically validated. We aimed to assess the diagnostic accuracy of the EEG criteria proposed by a
Department of Neurology, panel of experts at the fourth London–Innsbruck Colloquium on Status Epilepticus in Salzburg, 2013 (henceforth
Christian Doppler Klinik, called the Salzburg criteria).
Paracelsus Medical University,
Salzburg, Austria
(M Leitinger MD, Methods We did a retrospective, diagnostic accuracy study using EEG recordings from patients admitted for
Prof E Trinka MD, neurological symptoms or signs to three centres in two countries (Danish Epilepsy Centre, Dianalund, Denmark;
A Rohracher MD, G Kalss MD, Aarhus University Hospital, Aarhus, Denmark; and Paracelsus Medical University, Salzburg, Austria). Participants
J Höfler MD,
were included from the Danish centres if they were aged 4 months or older, and from the Austrian centre if aged
G Zimmermann MSc,
G Kuchukhidze MD, 18 years or older. Participants were sorted into two groups: consecutive patients under clinical suspicion of having
J Dobesberger MD, NCSE (the clinical validation group) or consecutive patients with abnormal EEG findings but no clinical suspicion of
P B Langthaler BSc); Centre for NCSE (the control group). Two raters blinded to all other patient data retrospectively analysed the EEG recordings
Cognitive Neuroscience
Salzburg, Austria (M Leitinger,
and, using the Salzburg criteria, categorised patients as in NCSE or not in NCSE. By comparing with a reference
Prof E Trinka, A Rohracher, standard inferred from all clinical and para-clinical data, therapeutic response, and the final outcome, we calculated
G Kalss, J Höfler, G Zimmermann, sensitivity, specificity, overall diagnostic accuracy, positive and negative predictive values, and inter-rater agreement
G Kuchukhidze, J Dobesberger, for the Salzburg criteria. The reference standard was inferred by two raters who were blinded to the scorings of the
P B Langthaler); Department of
Neurology, Medical University
Salzburg criteria.
of Innsbruck, Innsbruck,
Austria (G Kuchukhidze); Findings We retrospectively reviewed EEG data from 220 patients. EEGs in the clinical validation group were recorded
Department of Clinical in 120 patients between Jan 1, and Feb 28, 2014 (Austria), and Aug 1, 2014, and Jan 31, 2015 (Denmark). EEGs in the
Neurophysiology, Danish
Epilepsy Centre, Dianalund,
control group were recorded in 100 patients between Jan 13 and Jan 22, 2014 (Austria) and Jan 12 and Jan 26, 2015
Denmark (Denmark). According to the reference standard, 43 (36%) of the 120 patients in the validation group had NCSE. In
(E Gardella MD, S Beniczky MD); the validation cohort sensitivity was 97·7% (95% CI 87·9–99·6) and specificity was 89·6% (80·8–94·6); overall
Institute of Regional Health accuracy was 92·5% (88·3–97·5). Positive predictive value was 84·0% (95% CI 74·1–91·5) and negative predictive
Research, University of
Southern Denmark, Odense,
value was 98·6% (94·4–100). Three people in the control group (n=100) fulfilled the Salzburg criteria and were
Denmark (E Gardella); and therefore false positives (specificity 97·0%, 95% CI 91·5–99·0; sensitivity not calculable). Inter-rater agreement was
Department of Clinical high for both the Salzburg criteria (k=0·87) and for the reference standard (k=0·95). Therapeutic changes occurred
Neurophysiology, Aarhus significantly more often in the group of patients fulfilling Salzburg criteria (42 [84%] of 50 patients) than in those
University Hospital, Aarhus,
Denmark (E Qerama MD,
who did not (11 [16%] of 70; p<0·0001).
A Hess MD, S Beniczky)
Correspondence to: Interpretation The Salzburg criteria for diagnosis of NCSE have high diagnostic accuracy and excellent inter-rater
Dr Sándor Beniczky, Department agreement, making them suitable for implementation in clinical practice.
of Clinical Neurophysiology,
Aarhus University Hospital,
Aarhus 8000, Denmark
Funding None.
[email protected]
Introduction periorbital low-amplitude jerking).3 In the most extreme
A new classification of status epilepticus was established forms (ie, when the patient is in a coma), only EEG can
in 2015, aiming to resolve the ambiguities of the previous reveal the epileptiform or rhythmic discharges that lead
classifications and integrate knowledge about status to diagnosis.2 However, no consensus exists as to which
epilepticus.1 The framework for the new classification of EEG patterns represent status epilepticus in deep coma.4
status epilepticus was built on four axes, which show Several EEG criteria to diagnose NCSE have been
EEG gaining a greater context: semiology, aetiology, EEG published in the last 20 years,2,5–13 but their diagnostic
correlates, and age. Although clinical presentation is accuracy has not yet been systematically investigated. Thus,
often clear in the convulsive form, in non-convulsive it is unknown how reliable and clinically useful these EEG
status epilepticus (NCSE) a correct diagnosis usually criteria are for diagnosis. At the fourth London–Innsbruck
needs an EEG,2 since the clinical signs are often subtle Colloquium on Status Epilepticus in Salzburg, April 4–6,
and non-specific.3 Clinical suspicion of NCSE arises in 2013, a panel of experts developed a set of consensus
patients who have disturbance of consciousness with or criteria for NCSE12 based on the expertise of the consensus
without minor motor phenomena (eg, perioral or group invited by the organisers of the Colloquium and on

1054 www.thelancet.com/neurology Vol 15 September 2016

 Articles

Research in context
Evidence before this study status epilepticus (NCSE). We analysed EEG data by applying
We searched PubMed for relevant articles published between the Salzburg criteria for NCSE to recordings from 220 patients,
1950 and Dec 23, 2015. We used the search terms “EEG” and from three centres. Inter-rater agreement for Salzburg criteria
“non-convulsive status epilepticus”, and did not use any was almost perfect and sensitivity and specificity were high,
language restrictions. We found 279 PubMed entries. Seven demonstrating clinical validity of these criteria for
papers, and an additional three publications identified by a non-convulsive status epilepticus. Therapeutic changes were
hand search of their references, reported a range of diagnostic seen significantly more often in the group of patients fulfilling
criteria, including EEG criteria alone, criteria with EEG and the Salzburg criteria as compared with patients who did not,
clinical response to antiepileptic drugs applied during suggesting that the criteria correlate well with the therapeutic
recording, and criteria defining both EEG and clinical decisions.
conditions. The EEG criteria in these publications focus mainly
Implications of all the available evidence
on epileptiform discharges and rhythmic activity. Frequency
Previous publications suggested that EEG plays an essential role
criteria and occurrence of certain patterns (eg, spatiotemporal
in the diagnosis of NCSE, and criteria have been proposed for
evolution or subtle clinical phenomena) are shared by many
the diagnosis of non-convulsive status epilepticus. This clinical
sets of criteria. Although eleven studies provide
validation study provides evidence for the diagnostic accuracy
epidemiological data or present case series with more than ten
of the Salzburg criteria for non-convulsive status epilepticus
patients, diagnostic accuracy has not yet been determined in
and therefore allows application in patient management. Our
any one of the above-mentioned studies.
results will help further research by providing specific, clinically
Added value of this study validated diagnostic criteria to improve the quality of
To our knowledge, this is the first study to clinically validate therapeutic studies.
the diagnostic accuracy of expert criteria for non-convulsive

previous proposals. The Salzburg criteria applied EEG board-certified experts (SB, EG, AH, and EQ in
terminology from the SCORE consensus statement14 and Denmark; JD, JH, GKa, GKu, and ML in Austria)
integrated the revised terminology for Critical Care EEG reviewed EEGs at admission. We excluded participants
(2012 version) of the American Clinical Neurophysiology with technically insufficient EEG recordings (where
Society (ACNS).9 In this study we aimed to assess the interpretation was not possible due to artifacts) and
diagnostic accuracy of the Salzburg criteria and compare it EEG recordings lasting less than 20 min.
with a reference standard in patients with and without The regional ethics committees from each country
clinical suspicion of NCSE. reviewed this project, in which only de-identified
patient data were collected. According to Austrian and
Methods Danish regulations, this study did not need written
Study design and participants informed consent from the patients since it involves a
We did this retrospective study at three centres: a retrospective analysis of anonymised data, and is non-
tertiary referral centre for patients with epilepsy interventional.
(Danish Epilepsy Centre, Dianalund, Denmark) and
two departments providing care for general neurology Procedures
patients and with emergency rooms (Paracelsus The authors (SB, EG, AH, and EQ in Denmark; JH,
Medical University, Salzburg, Austria; and Department GKa, GKu, and ML in Austria) retrospectively reviewed
of Clinical Neurophysiology, Aarhus University, EEG recordings while blinded to the patients’ clinical data
Denmark). Participants were included from the Danish and final diagnoses. The Salzburg criteria were strictly
centres if they were aged 4 months or older, and from applied to the recordings, and the results were logged in
the Austrian centre if aged 18 years or older. the study database, including information about which
Two groups were analysed. The validation group criteria were fulfilled. Using the unified terminology
consisted of consecutive patients with clinical suspicion proposed previously,12–14 we scored the recordings (NCSE,
of NCSE. Patients were required to have decreased possible NCSE, or not NCSE; figure 1). The definition of
levels of cognitive performance or consciousness for at status epilepticus used in this study1 implied that we
least 10 min for inclusion.13 The control group was considered patients without prominent myoclonic jerks
formed of consecutive patients without any clinical as having NCSE (ie, category B.1 NCSE with coma,
suspicion of NCSE, but abnormal EEG findings. including so-called subtle status epilepticus), but we did
Patients were referred to EEG recording as part of their not consider as NCSE category A.2 myoclonic status
diagnostic work-up when admitted to hospital for epilepticus (prominent epileptic myoclonic jerks).1 For
neurological symptoms or signs. Nine experienced, calculation of inter-rater agreement, each recording was

www.thelancet.com/neurology Vol 15 September 2016 1055

 Articles

reviewed independently by two of the authors and the for NCSE.12 Therefore, in this study we used three
score of each author was logged. When the raters’ scores different time-epochs for evaluation: 10–30 s, 31–60 s,
did not agree, recordings were reviewed in joint reading and more than 60 s; we compared their performance
sessions by the two raters, and the final conclusion was using a receiver operating characteristics (ROC) curve.
based on consensus. When consensus could not be Scoring was based on several features. In patients
reached, a third author’s scoring was added. without epileptic encephalopathy, there are criteria for
To determine diagnostic accuracy, we also categorised NCSE and others for possible NCSE (figure 1). The
patients as NCSE or non-NCSE using a reference criterion of frequency of epileptiform discharges (sharp
standard. Although there is currently no established waves and spikes) of higher than 2·5 per s qualifies for
and validated reference standard to diagnose NCSE, a NCSE. Frequency is measured as epileptiform
consensus decision inferred from multimodal data can discharges per 10 s epoch.13 If epileptiform discharges
provide the reference standard when no single or fixed do not exceed 2·5 per s, or if rhythmical activity is
reference method is available.15,16 In this study, the present, a secondary criterion has to be present: subtle
reference standard was inferred from all clinical and clinical phenomena (1), clinical and EEG improvement
para-clinical data, including EEG readings (but not to intravenous antiepileptic drugs (2), or typical
the results of Salzburg criteria), laboratory data, spatiotemporal evolution of the defining grapho-
neuroimaging data, therapeutic response, follow-up, elements (3). A score of possible NCSE is given if there
and final outcome. For all patients and recordings, two is fluctuation of defining graphoelements without
authors evaluated these data independently, while evolution, or if only EEG improvement with intravenous
blinded to the Salzburg criteria scorings. The antiepileptic drugs is present without clinical
conclusion of each rater was logged for inter-rater improvement. In patients with pre-existing epileptic
agreement analysis. When consensus was not achieved encephalopathy (defined as “the epileptic activity itself
between the scores of the two raters, a third author may contribute to severe cognitive and behavioural
evaluated the data. impairments above and beyond what might be expected
The minimal duration of the EEG epoch that had to from the underlying pathology alone, and that these
fulfil the Salzburg criteria was not specified in the can worsen over time” by International League Against
See Online for appendix original publication outlining terminology and criteria Epilepsy [ILAE] criteria17), one additional element is
needed for diagnosis of NCSE: either the improvement
of EEG and clinical state with intravenous antiepileptic
Epileptiform discharges Clinical suspicion of NCSE No epileptiform discharges drugs administered during recording or the increase in
frequency or prominence of defining graphoelements
with concomitant clinical worsening as compared with
Continuous (quasi-) rhythmic the baseline.12
Frequency >2·5 cycles per s* Frequency ≤2·5 cycles per s delta-theta activity with
frequency >0·5 cycles per s*
Statistical analysis
Patients whose EEG data were scored as possible NCSE
were considered to have a positive diagnosis of NCSE in
Typical Subtle clinical ictal IV AED Fluctuation analyses. We calculated sensitivity, specificity, overall
spatiotemporal phenomenon during the EEG without
evolution* EEG patterns mentioned above* definitive accuracy, and positive and negative predictive values,
evolution* according to standardised formulas. For significance
testing, we used Egon Pearson’s version of the χ²-squared
EEG and clinical EEG without clinical No test; for the tables with small cell counts we used Irwin’s
improvement improvement version of Fisher’s exact test.18–22 We used the Benjamini-
after IV AED* after IV AED* Hochberg method with α=0·05 to adjust the p-values
presented for subgroup analyses.23
No other secondary criteria
Based on the sensitivities and specificities for each of
are fulfilled the three epoch durations, we calculated an estimated
ROC curve.25 We calculated 95% CIs for sensitivity and
specificity using Wilson’s method.28–30 We expressed
NCSE Possible NCSE No NCSE positive predictive values and negative predictive values
as functions of the prevalence and the estimated
Figure 1: Salzburg EEG criteria for the diagnosis of NCSE sensitivity and specificity using Bayes‘ theorem,24 and we
To qualify for a diagnosis of NCSE, the whole EEG recording should be abnormal, and EEG criteria have to be plotted them to model the differences in predictive values
continuously present for at least 10 s. If criteria are not fulfilled at any stage, EEG recording will not qualify for a among subgroups, depending on the prevalence in the
diagnosis of NCSE or possible NCSE. NCSE=non-convulsive status epilepticus. IV AED=intravenous antiepileptic
studied population (appendix).24 For positive predictive
drug. *Patients with known epileptic encephalopathy should fulfil one of the additional secondary criteria: increase
in prominence or frequency of the features above when compared to baseline, and observable change in clinical value, negative predictive value, and Gwet’s AC1 we
state; or improvement of clinical and EEG features with IV AEDs (panel). provided 95% bootstrap BCa CIs.31,32

1056 www.thelancet.com/neurology Vol 15 September 2016

 Articles

For assessment of inter-rater agreement, we calculated medical records, we assessed the proportion of patients
Gwet’s AC1 for the whole patient group as well as for the fulfilling the Salzburg criteria in whom a change in
two populations (validation and control groups). We used therapy was instituted after the EEG recording.
this measure of agreement because it yields more
reasonable values than Cohen’s k in case of low-trait
prevalences.26,27 Inter-rater agreement was interpreted Validation Control group
group (n=120) (n=100)
according to the conventional groups: poor (k<0), slight
(k 0·01–0·2), fair (k 0·21–0·4), moderate (k 0·41–0·6), Median age (years) 65·0 53·0
(0·75–94) (0·8–93)
substantial (k 0·61–0·8), and almost perfect agreement
Sex
(k>0·8).33 We did analyses with R version 3.1.1 For Gwet’s For more about R see http://
Female 56 (47%) 57 (57%) www.R-project.org/
AC1 calculations, we used a modified version of the script
Male 64 (53%) 43 (43%)
provided by Gwet. For Gwet’s script see http://
As an additional analysis we also calculated diagnostic Vigilance status during EEG* www.agreestat.com/r_functions.
Awake 40 (33%) 87 (87%) html
accuracy measures for the alternative scenario of the
score possible NCSE being considered as a negative Somnolence 37 (31%) 10 (10%)
NCSE diagnosis. Finally, by retrieving information from Stupor 11 (9%) 1 (1%)
Coma 32 (27%) 2 (2%)
Pre-existing epilepsy 45 (38%) 52 (52%)
Panel: Specifications for the Salzburg criteria Not classified 0 5 (5%)
Symptomatic
Frequency of the epileptiform discharges Focal 28 (23%) 23 (23%)
Frequency higher than 2·5 cycles per s is considered when Generalised 1 (1%) 2 (2·0%)
more than 25 epileptiform discharges are seen per 10 s Cryptogenic (unknown) 13 (11%) 10 (10%)
epoch.13
Genetic or idiopathic
Continuous (quasi-)rhythmic delta-theta activity Focal 0 1 (1%)
Repetition of waveforms with relatively uniform morphology Generalised 3 (3%) 11 (11%)
and duration, and without an interval between consecutive Patients with hypoxia 15 (13%) 0
waveforms. The duration of one cycle (ie, the period) of the Pre-existing epileptic 8 (7%) 3 (3%)
rhythmic pattern should vary by less than 50% from the encephalopathy
duration of the subsequent cycle for most (>50%) cycle pairs Mean EEGs per patient 2 (1–15) 1 (1–1)
to qualify as rhythmic.9 Mean continuous EEG duration (h) 74·8 †
(5·0–142·0)
Typical spatiotemporal evolution Aetiology of NCSE
Sequential change in voltage and frequency, or evolution in Acute symptomatic 13/43 (30%) ··
frequency and change in location: Remote unprovoked 27/43 (63%) ··
• Change in voltage (increase or decrease) with a minimum Symptomatic seizure or progressive 2/43 (5%) ··
factor of two of the voltages measured between the first disease
and last graphoelement. Unprovoked unknown aetiology 1/43 (2%) ··
• Change in frequency more than 1 Hz: frequency of the Dynamics in evolution of semiology of NCSE
second with highest rate of graphoelements and the CSE→NCSE with minor motor 1/43 (2%) ··
second with lowest rate of graphoelements differed by phenomena
more than 1 Hz. CSE→NCSE without motor 7/43 (16%) ··
• Evolution in frequency is defined as at least two phenomena
consecutive changes in the same direction by at least NCSE with minor motor 1/43 (2%) ··
phenomena→without motor
0·5 per s.9 phenomena
• Change in location sequential spreading into or out of at NCSE with minor motor phenomena 13/43 (30%) ··
least two different standard 10–20 electrode locations.9
NCSE without motor phenomena 21/43 (49%) ··
• To qualify as present, a single frequency or location must
STESS score 3 or higher (high risk) 18/43 (42%) ··
persist at least three cycles. The criteria for evolution must
EMSE 64 or higher (high risk; Austria) 11/15 (73%) ··
be reached without the pattern remaining unchanged in
frequency, morphology, or location for 5 min or more.9 Data are median or mean (range), n (%), or n/N (%). Additional data on final
outcome are presented in the appendix. CSE=convulsive status epilepticus.
Fluctuation without definite evolution NCSE=non-convulsive status epilepticus. STESS=status epilepticus severity scale.
Three or more changes, not more than 1 min apart, in EMSE=epidemiology-based mortality score in status epilepticus. *Somnolence was
defined as decreased level of consciousness that was arousable by loud voice and
frequency (by at least 0·5 per s) or three or more changes in touch, whereas in stupor only very strong stimuli caused arousal. †In the control
location (by at least one standard interelectrode distance), group we included standard recordings of 30 min duration.
but not qualifying as evolving.9
Table 1: Demographic and clinical characteristics

www.thelancet.com/neurology Vol 15 September 2016 1057

 Articles

Validation group Control group

120 eligible patients 100 eligible patients

Salzburg criteria applied to 120 patients Salzburg criteria applied to 100 patients

50 fulfilled Salzburg criteria 70 did not fulfil Salzburg criteria 3 fulfilled Salzburg criteria 97 did not fulfil Salzburg criteria

50 had reference standard available 70 had reference standard available 3 had reference standard available 97 had reference standard available

42 had NCSE confirmed by 1 had NCSE confirmed by 0 had NCSE confirmed by 0 had NCSE confirmed by
reference standard (TP) reference standard (FN) reference standard (TP) reference standard (FN)

8 had NCSE not confirmed by 69 had NCSE not confirmed by 3 had NCSE not confirmed by 97 had NCSE not confirmed by
reference standard (FP) reference standard (TN) reference standard (FP) reference standard (TN)

Figure 2: Study profile

FN=false negative. FP=false positive. TN=true negative. TP=true positive.

epoch is the best classifier: with a slightly lower

All patients Validation group Control group
specificity, it gives a substantial gain in sensitivity
Reference 0·95 (0·91–0·98) 0·91 (0·82–0·97) 0·99 (0·94–1) compared to the other epochs.
standard
In the validation group, using the optimal 10 s epoch
Salzburg 0·87 (0·81–0·92) 0·81 (0·71–0·89) 0·94 (0·87–0·98)
duration, sensitivity of the Salzburg criteria for NCSE
criteria
was 97·7% (95% CI 87·9–99·6) and specificity was
Table 2: Inter-rater agreement (κ [95% CI]) 89·6% (80·8–94·6); the overall diagnostic accuracy
was 92·5% (88·3–97·5); the positive predictive value was
84·0% (74·1–91·5) and the negative predictive value
Role of the funding source was 98·6% (94·4–100). There were no significant
There was no funding source for this study. All authors differences in sensitivity and specificity between the
had full access to all of the study data, and SB had the various diagnostic subgroups and clinical settings
final responsibility for the decision to submit the report (table 3; appendix). When the lower limit of the time
for publication. epoch was prolonged to 30 s and to 60 s, sensitivity
decreased by 9·3 and 11·7 percentage points respectively,
Results with an increase in specificity of 1·3 and 2·6 percentage
We reviewed EEGs from 220 patients. EEGs in the points (table 3). Positive predictive value increased by
clinical validation group (n=120) were recorded between 0·4 and 2·0 percentage points, whereas negative
Jan 1 and Feb 28, 2014 (Austria), and Aug 1, 2014, and predictive value dropped by 5·3 and 6·3 percentage
Jan 31, 2015 (Denmark). EEGs in the control group points, respectively. Performance of the different epochs
(n=100) were recorded between Jan 13 and Jan 22, 2014 is shown in the ROC curve (appendix).
(Austria), and Jan 12 and Jan 26, 2015 (Denmark). Table 1 29 (69%) of the 42 true-positive patients fulfilling the
summarises demographic and baseline clinical data of Salzburg criteria had more EEG features than the
the included patients. 15 patients in the validation group minimum necessary for qualifying as positive for a
were post-hypoxic—ie, after cardiac arrest. NCSE diagnosis (table 4; appendix).
According to the reference standard, 43 (36%) in the Patients fulfilling the criteria for possible NCSE were
validation group had NCSE (figure 2). Inter-rater considered positives; 13 (31%) of the 42 true-positive
agreement was higher than 0·8 in both patient groups patients and six (75%) out of the eight false-
(table 2). Of the 42 true positives in the validation group, positive patients were classified as possible NCSE.
most of the patients (n=37) fulfilled the EEG criteria for Considering these 19 patients with possible NCSE as
continuous epochs of more than 60 s. In four patients, negatives increased specificity by 7·8 percentage points,
this epoch was between 10 s and 30 s, and in one patient and positive predictive value by 10·4 percentage points;
it was between 31 s and 60 s. The area under the ROC however, it decreased sensitivity by 18·6 percentage
curve estimate was 0·94, indicating the high points, and negative predictive value by 9·3 percentage
performance of the Salzburg criteria, and that 10 s points (table 3).

1058 www.thelancet.com/neurology Vol 15 September 2016

 Articles

Patients (n) Time-epoch Sensitivity (%) Specificity (%) PPV (%) NPV (%) Accuracy (%)
Epoch length
All patients 120 10 s 97·7 89·6 84·0 98·6 92·5
All patients 120 30 s 88·4 90·9 84·4 93·3 90·0
All patients 120 60 s 86·0 92·2 86·0 92·2 90·0
Coma
Non-coma 88 10 s 96·7 87·9 80·6 98·1 90·9
Coma 32 10 s 100 94·7 92·9 100 96·9
Hypoxic
Non-hypoxic 105 10 s 97·2 88·4 81·4 98·4 91·4
Post-hypoxic 15 10 s 100 100 100 100 100
Epilepsy
Pre-existing epilepsy 45 10 s 95·7 81·8 84·6 94·7 88·9
Without pre-existing epilepsy 75 10 s 100 92·7 83·3 100 94·7
Epileptic encephalopathy
Epileptic encephalopathy 6 10 s 75·0 100 100 66·7 83·3
Without epileptic encephalopathy 114 10 s 100 89·3 83·0 100 93·0
Age
Age <10 years 10 10 s 100 100 100 100 100
Age ≥10 years 110 10 s 97·1 89·3 81·0 98·5 91·8
Possible NCSE considered negative 120 10 s 79·1 97·4 94·4 89·3 90·8

Data are n (%), unless otherwise stated. No significant differences between subgroups. PPV=positive predictive value. NPV=negative predictive value.

Table 3: Diagnostic accuracy for the various subgroups and disorders in the validation cohort

The only false-negative case was a patient with Lennox- criterion for fluctuating epileptiform discharges. They
Gastaut syndrome, who had tonic seizures in clusters also showed a significantly higher chance of EEG and
and did not recover between the tonic seizures. Whereas clinical improvement to intravenous antiepileptic drugs
the tonic seizures themselves belong to the category of and they had a significantly higher chance of survival,
status with major motor activity, the periods between the compared with patients without pre-existing epilepsy
clustered seizures were classified as non-convulsive.1 The (appendix).
ictal EEG correlate was the cessation of the interictal In the control group of 100 patients, only three fulfilled
epileptiform pattern, which consisted of abundant 2 Hz the Salzburg criteria (specificity 97·0%; 95% CI
sharp-and-slow wave complexes. 91·5–99·0), but none were confirmed by the reference
Eight cases in the validation group were false standard (false positives). In all three patients, criteria
positives. In two cases, the patients had clusters of were fulfilled during a seizure with postictal recovery but
seizures of short duration, and they recovered without clinical suspicion of NCSE.
completely between them. According to the recently
published ILAE definition, these episodes were not Discussion
considered NCSE.1 However, the EEG recorded in this EEG patterns in NCSE are diverse; therefore, diagnostic
period fully satisfied the Salzburg criteria. Six of these criteria have to cover a broad spectrum of EEG features.
eight false-positive patients scored as possible NCSE In this study, we scored and evaluated in detail the EEG
due to the pattern of fluctuation without evolution; features recorded in patients with NCSE. Most of the
these patients had, respectively, septicaemia, traumatic patients fulfilled several combinations of criteria,
brain injury, acute ischaemic infarction, intracerebral indicating that the Salzburg criteria are over-secured to
haemorrhage in brainstem, renal failure, and hepatic make sure the relevant patterns are not missed.
failure (the latter also had systemic infection). However, this was not detrimental to specificity, which
Therapeutic changes were instituted in 42 (84%) of was higher than 80%. Reliability and generalisability
50 patients fulfilling the Salzburg criteria in the validation were achieved by assessing inter-rater agreement and
group. Among the 70 patients who did not fulfil the by including a large number of consecutive patients
Salzburg criteria in the validation group, therapeutic admitted to three hospitals in two countries, including
change was instituted in 11 patients (15·7%, χ² p<0·0001). a tertiary referral centre for patients with epilepsy and
Patients with pre-existing epilepsy significantly more general neurology hospital departments. Our results
often fulfilled the criterion for more than 2·5 epileptiform indicate that the Salzburg criteria have high diagnostic
discharges per s, and significantly less often fulfilled the accuracy and excellent inter-rater agreement. Sensitivity

www.thelancet.com/neurology Vol 15 September 2016 1059

 Articles

True positives and false True positives (n=42)

positives (n=50)
Patients who fulfilled Salzburg criteria criterion I (without pre-existing epileptic encephalopathy) 47 (94%) 39 (93%)
(1) More than 25 epileptiform discharges per 10 s epoch (2·5 per s; NCSE-ED >2·5)34 18 (36%) 17 (41%)*
(2) Patients with epileptiform discharges 2·5 per s or less or rhythmic delta-theta activity exceeding 29 (58%) 22 (52%)
0·5 per s and at least one of:
(2a) EEG and clinical improvement with IV AEDs (NCSE-ECI) 6 (12%) 5 (12%)
Epileptiform discharges (NCSE-ED-ECI) 1 (2%) 1 (2%)
Rhythmic delta-theta activity (NCSE-RDT-ECI) 1 (2%) 0
Epileptiform discharges and rhythmic delta-theta activity (NCSE-EDRD-ECI) 4 (8%) 4 (10%)†
(2b) Subtle clinical phenomena (NCSE-SCP) 9 (18%) 9 (21%)
Epileptiform discharges (NCSE-ED-SCP) 3 (6%) 3 (7%)‡
Rhythmic delta-theta activity (NCSE-RDT-SCP) 1 (2%) 1 (2%)
Epileptiform discharges and rhythmic delta-theta activity (NCSE-EDRD-SCP) 5 (10%) 5 (12%)§
(2c) Typical spatiotemporal evolution (NCSE-STE) 6 (12%) 5 (12%)
Epileptiform discharges (NCSE-ED-STE) 1 (2%) 1 (3%)
Rhythmic delta-theta activity (NCSE-RDT-STE) 3 (6%) 2 (5%)
Epileptiform discharges and rhythmic delta-theta activity (NCSE-EDRD-STE) 2 (4%) 2 (5%)¶
Possible NCSE 19 (38%) 13 (31%)
(2d) Only EEG improvement to IV AEDs (NCSE-OEI) 3 (6%) 3 (7%)
Epileptiform discharges (NCSE-ED-OEI) 3 (6%) 3 (7%)||
(2e) Fluctuation, no evolution (NCSE-FLUCT) 16 (32%) 10 (24%)
Epileptiform discharges (NCSE-ED-FLUCT) 15 (30%) 9 (21%)**
Epileptiform discharges and rhythmic delta-theta activity (NCSE-EDRD-FLUCT) 1 (2%) 1 (2%)††
Patients who fulfilled Salzburg criteria criterion II (with epileptic encephalopathy; NCSE-EE)‡‡ 3 (6%) 3 (7%)
(1) More than 25 epileptiform discharges per 10 s epoch (2·5 per s; NCSE-EE-ED >2·5) 2 (4%) 2 (5%)§§
(2b) Subtle clinical phenomena (NCSE-EE-SCP) 1 (2%) 1 (2%)

Codes in parentheses are diagnostic classifications. EEG improvement defined as significant reduction (more than 50%) in the abnormal EEG features. IV AED=intravenous
antiepileptic drug. *Five patients also had typical spatiotemporal evolution, five patients also had subtle clinical phenomena, nine patients had EEG and clinical
improvement after IV AEDs, four patients had only EEG without clinical improvement, and one patient had fluctuation without evolution (another one had fluctuation but
also evolution). †All four patients had also subtle clinical phenomena. ‡One patient also had EEG without clinical improvement; another had fluctuation without evolution
(both received narcotics, ie, status epilepticus stage 3). §All five patients had EEG and clinical improvement after IV AEDs. ¶One patient also had fluctuation and received
narcotics (status epilepticus stage 3). ||Two patients had fluctuation of epileptiform discharges without evolution, one had spatiotemporal evolution (status epilepticus
stage 3), and one also had subtle clinical phenomena (status epilepticus stage 3). **Two patients also had EEG without clinical improvement, another had spatiotemporal
evolution (status epilepticus stage 3). ††Patient also had spatiotemporal evolution and status epilepticus stage 3. ‡‡Increase in prominence and frequency with observable
change in clinical state. §§One patient had EEG and clinical response after IV AEDs, another one had only EEG without clinical improvement.

Table 4: EEG features in patients fulfilling the Salzburg criteria in the validation group

and specificity did not significantly differ among the departments of general neurology. The fluctuating
various diagnostic subgroups and clinical settings patterns share many features with patterns caused by
(appendix). This broad spectrum of subgroups metabolic encephalopathies, and are frequent, since
and clinical settings supports the feasibility and people with disturbed consciousness (eg, due to
generalisability of the criteria for widespread use. The septicaemia, renal or hepatic failure) are first evaluated
high specificity was confirmed in a control group of in this setting. Despite the heterogeneous causes and
patients with abnormal EEG recordings, but without clinical presentations, we found that the Salzburg
clinical suspicion of NCSE. criteria performed well in all clinical settings.
The EEG features fulfilling the Salzburg criteria We investigated the minimum duration of EEG epoch
differed between the tertiary referral centre for epilepsy that has to fulfil the criteria (>10 s, 30–60 s, and >60 s).
and the general neurology departments with emergency Although specificity increased slightly with the longer
rooms. Whereas most patients in the tertiary centre had durations, sensitivity dropped by more than 10%, and
pre-existing epilepsy, most of the cases in the general the ROC curve showed that 10 s was the best classifier.
neurology departments were acute onset, first-time Therefore, we suggest that 10 s minimum EEG epoch is
seizures. Patients with more than 2·5 epileptiform used in clinical practice. We have to emphasise that in
discharges per s were mainly seen in the tertiary centre, all recordings fulfilling the Salzburg criteria, the whole
whereas those with fluctuating epileptiform discharges EEG recording was abnormal, but the prevalence of
without evolution (as only criterion) were seen only in abnormalities fluctuated, and the minimum duration

1060 www.thelancet.com/neurology Vol 15 September 2016

 Articles

where the EEG criteria were fulfilled represented the Declaration of interests
peak prevalence of the abnormalities. The minimum ML reports grants from Medtronic and UCB Pharma and personal fees
from Everpharma. ET reports personal fees from Eisai, Everpharma,
duration of peak abnormalities in EEG (ie, 10 s) is not Medtronics, Bial, Newbridge, GL Pharm, GlaxoSmithKline, Boehringer
the same as the minimum duration of the clinical Viropharma, and Actavis; grants and personal fees from Biogen Idec,
episode, which was 10 min. UCB Pharma and Eisai; and grants from Red Bull, Merck, European
Most of the features in the Salzburg criteria focus on Union, FWF Österreichischer Fond zur Wissenschaftsförderung,
Bundesministerium für Wissenschaft und Forschung, and the
detection of ictal activity, which makes them an efficient Jubiläumsfond der Österreichischen Nationalbank. GKa reports travel
set of criteria for identifying various ictal patterns. grants from UCB Pharma and Eisai. JH reports travel grants and
However, since NCSE is never diagnosed solely based on personal fees from UCB Pharma and Eisai. AR reports travel support
EEG, clinical data—such as signs of impaired from Eisai. JD reports personal fees from Gerot-Lanach, and support
from Eisai, GlaxoSmithKline, and Neurodata GmbH/Micromed Austria.
consciousness for more than 10 min1,13—are necessary SB reports non-financial support from Elekta and personal fees from
for accurate diagnosis, and identification of the best UCB Pharma. All other authors declare no competing interests.
reference standard is challenging. We emphasise that Acknowledgments
this study assesses the accuracy of the Salzburg criteria We thank all medical and non-medical staff of involved centres.
and not the accuracy of EEG per se. Currently, there is References
no established and validated single test that can diagnose 1 Trinka E, Cock H, Hesdorffer D, et al. A definition and classification
NCSE. Therefore, as previously suggested,15,16 we inferred of status epilepticus - Report of the ILAE Task Force on
Classification of Status Epilepticus. Epilepsia 2015; 56: 1515–23.
the reference standard from multimodal data, including 2 Young GB, Jordan KG, Doig GS. An assessment of nonconvulsive
all clinical and para-clinical data, therapeutic response, seizures in the intensive care unit using continuous EEG monitoring:
and final outcome. Two raters evaluated these data an investigation of variables associated with mortality. Neurology 1996;
47: 83–89.
independently, and inter-rater agreement was almost 3 Shorvon S. The outcome of tonic-clonic status epilepticus.
perfect (k=0·95). Curr Opin Neurol 1994; 7: 93–95.
The clinical decision of instituting a therapeutic 4 Bauer G, Trinka E. Nonconvulsive status epilepticus and coma.
Epilepsia 2010; 51: 177–90.
change in the validation group was based on all available
5 Litt B, Wityk RJ, Hertz SH, et al. Nonconvulsive status epilepticus
data, and not on application of the Salzburg criteria. The in the critically ill elderly. Epilepsia 1998; 39: 1194–202.
fact that change of treatment was significantly more 6 Trinka E, Leitinger M. Which EEG patterns in coma are
common among the patients fulfilling the Salzburg nonconvulsive status epilepticus? Epilepsy Behav 2015; 49: 203–22.
criteria emphasises the potential clinical impact of the 7 Chong DJ, Hirsch LJ. Which EEG patterns warrant treatment in the
critically ill? Reviewing the evidence for treatment of periodic
criteria. epileptiform discharges and related patterns. J Clin Neurophysiol
The prevalence of NCSE in the validation group 2005; 22: 79–91.
would affect the positive and negative predictive values, 8 Hirsch LJ, Brenner RP, Drislane FW, et al. The ACNS subcommittee
on research terminology for continuous EEG monitoring: proposed
yet there are no available data on the prevalence of standardized terminology for rhythmic and periodic EEG patterns
NCSE in these clinical settings. This limitation should encountered in critically ill patients. J Clin Neurophysiol 2005;
22: 128–35.
be taken into account in the interpretation of the
9 Hirsch LJ, LaRoche SM, Gaspard N, et al. American Clinical
positive predictive value and negative predictive value. Neurophysiology Society’s Standardized Critical Care EEG
Another limitation of the study is the heterogeneity of Terminology: 2012 version. J Clin Neurophysiol 2013; 30: 1–27.
the recording durations due to the local guidelines of 10 Kaplan PW. EEG criteria for nonconvulsive status epilepticus.
Epilepsia 2007; 48 (suppl 8): 39–41.
the participating centres. Continuous EEG and long- 11 Sutter R, Kaplan PW. Electroencephalographic criteria for
term recordings were available in two out of the three nonconvulsive status epilepticus: synopsis and comprehensive
participating centres (Aarhus and Dianalund). In the survey. Epilepsia 2012; 53 (suppl 3): 1–51.
third centre (Salzburg), when needed, repeated 12 Beniczky S, Hirsch LJ, Kaplan PW, et al. Unified EEG terminology
and criteria for nonconvulsive status epilepticus. Epilepsia 2013;
recordings of up to 45 min were done (up to 54 (suppl 6): 28–29.
15 recordings per patient). It is possible that recording 13 Leitinger M, Beniczky S, Rohracher A, et al. Salzburg Consensus
continuous EEG for all patients would have resulted in Criteria for Non-Convulsive Status Epilepticus-approach to clinical
application. Epilepsy Behav 2015; 49: 158–63.
an even higher sensitivity. 14 Beniczky S, Aurlien H, Brøgger JC, et al. Standardized
In conclusion, the Salzburg criteria have high computer-based organized reporting of EEG: SCORE. Epilepsia
diagnostic accuracy for patients with clinical suspicion 2013; 54: 1112–24.
15 Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and
of NCSE. Besides the impact on clinical practice in accurate reporting of studies of diagnostic accuracy: the STARD
accurate diagnosis of NCSE, the criteria will promote initiative. BMJ 2003; 326: 41–44.
further research in this field since they provide precise 16 Weller SC, Mann NC. Assessing rater performance without a “gold
standard” using consensus theory. Med Decis Making 1997; 17: 71–79.
diagnostic standards for inclusion into therapeutic
17 Berg AT, Berkovic SF, Brodie MJ, et al. Revised terminology and
studies in patients with NCSE. concepts for organization of seizures and epilepsies: report of the
Contributors ILAE Commission on Classification and Terminology, 2005–2009.
ML, ET, and SB contributed to the conception and design of the study. Epilepsia 2010; 51: 676–85.
All authors acquired and analysed data. ML, ET, and SB drafted the 18 Pearson ES. The choice of statistical tests illustrated on the
manuscript. ML, ET, PBL, GZ, and SB drafted the figures. All authors interpretation of data classed in a 2 × 2 table. Biometrika 1947;
34: 139–67.
contributed to editing the final manuscript.

www.thelancet.com/neurology Vol 15 September 2016 1061

 Articles

19 Irwin JO. Tests of significance for differences between percentages 27 Gaspard N, Hirsch LJ, LaRoche SM, Hahn CD, Westover MB;
based on small numbers. Metron 1935; 12: 83–94. Critical Care EEG Monitoring Research Consortium. Interrater
20 Campbell I. Chi-squared an Fisher-Irwin tests of two-by-two tables agreement for Critical Care EEG Terminology. Epilepsia 2014;
with small sample recommendations. Stat Med 2007; 26: 3661–75. 55: 1366–73.
21 Pearson, K. On the criterion that a given system of deviations from 28 Wilson EB. Probable inference, the law of succession, and statistical
the probable in the case of a correlated system of variables is such inference. J Am Stat Assoc 1927; 22: 209–12.
that it can be reasonably supposed to have arisen from random 29 Vollset SE. Confidence intervals for a binomial proportion. Stat Med
sampling. Philos Mag Ser 5 1900; 50: 157–72. 1993; 12: 809–24.
22 Fisher, RA. On the interpretation of χ² from contingency tables, and 30 Agresti A, Coull BA. Approximate is better than “exact” for interval
the calculation of P. J R Stat Soc 1922; 85: 87–94. estimation of binomial proportions. Am Stat 1998; 52: 119–26.
23 Benjamini Y, Hochberg, Y. Controlling the false discovery rate: a 31 Efron B, Tibshirani RJ. An introduction to the bootstrap. 3rd edn.
practical and powerful approach to multiple testing. J R Stat Soc B New York: Chapman and Hall, 1993.
1995; 57: 289–300. 32 Carpenter J, Bithell J. Bootstrap confidence intervals: when, which,
24 Lange K, Brunner, E. Analysis of predictive values based on what? A practical guide for medical statisticians. Stat Med 2000;
individual risk factors in multi-modality trials. Diagnostics 19: 1141–64.
2013; 3: 192–209. 33 Landis JR, Koch GG. The measurement of observer agreement for
25 Cantor SB, Kattan MW. Determining the Area under the ROC categorical data. Biometrics 1977; 33: 159–74.
Curve for a Binary Diagnostic Test. Med Decis Making 2000; 34 Leitinger M, Kalss G, Rohracher A, et al. Predicting outcome of
20: 468–70. status epilepticus. Epilepsy Behav 2015; 49: 126–30.
26 Gwet KL. Computing inter-rater reliability and its variance in the
presence of high agreement. Br J Math Stat Psychol 2008; 61: 29–48.

1062 www.thelancet.com/neurology Vol 15 September 2016