Online data supplement

Exhaled nitric oxide as a marker of asthma control in smoking patients

______

Alain Michils1, MD; Renaud Louis2, MD; Rudi Peché3, MD; Sandra Baldassarre1, MD; Alain Van Muylem1, PhD

1 Chest Department - CUB Erasme, Brussels, Belgium

2 Chest Department - CHU Sart-Tilmant, Liège, Belgium

3 Chest Department- CHU André Vésale, Montigny- le-Tilleul, Belgium

METHODS

Statistical methods

1. Intrinsic characteristics of an index

1.1. Sensitivity and specificity

If N events are rated as positive or negative (e.g. positive = well-controlled asthma; negative = not-well controlled asthma) by a reference method (Gold Standard: GS), the ability of an index to discriminate between these complementary states, for a given cut-off value, may be assessed by using a contingency table. The index is considered as positive or negative depending on its position relative to the cut-off value. The contingency table appears like

GS positive
n+ / GS negative
n-
Index positive / TP / FP
Index negative / FN / TN

where n+ an n- are the amounts of positive and negative events (N = n+ + n-) according to the Gold standard, respectively. TP and TN are the amounts of true positive and true negative events, respectively; i.e. the amounts of events correctly rated by the index. FP and FN are the amounts of false positive and false negative events, respectively, i.e. the amounts of events incorrectly rated by the index.

From the contingency table, sensitivity (Se) and specificity (Sp) may be derived by

Se = TP/n+ (1)

Sp = TN/n- (2)

Se and Sp are intrinsic characteristics of the index (for this given cut-off value). Se and Sp are independent from the prevalence, the latter being defined as the positive events rate in a considered context (the "prior" probability of a positive event). From the contingency table, the prevalence is equal n+/N.

1.2. ROC curve

A contingency table may be constructed, and sensitivity and specificity computed, for a range of cut-off values. The Receiving Operator Characteristics (ROC) curve is the true positive rate (sensitivity) as a function of the false positive rate (1-specificity) for the considered range of cut-off values. This curve allows visually assessing the relative "quality" of several indexes. It also allows "choosing" an optimal cut-off value for a given index according to investigator's criteria.

The area under the ROC curve (AUC) may be computed and its difference relative to 0.5 statistically assessed. If AUC not different from 0.5 (which is the area under the first bisector), it means that the considered index is as useful as tossing a coin.

1.3. The Youden's index

The Youden's index (J), is the difference between the true positive rate and the false positive rate. Maximizing this index allows to find, from the ROC curve, an optimal cut-off point independently from the prevalence. According to its definition and as illustrated on Fig.1, J is the vertical distance between the ROC curve and the first bisector (or chance line). If F(x) is the function describing the ROC curve, with x = 1-specificity, we may write

J(x) = F(x)-x (3)

When J is maximal, J'(x) = 0, where J' is the derivative of J.

From Eq. 1: J'(x)= F'(x) -1, (4)

where F' is the derivative of F.

Hence, when J is maximal, F'(x) = 1, meaning that the tangent to the ROC curve is parallel to the first bisector (slope = 1). It implies that, around this point, a gain (or a loss) in specificity results in a loss (or a gain) of the same amplitude in sensitivity.

2. Predictive values

Positive (PPV) and negative (NPV) predictive values are the probability to rate correctly an event using the index; i.e. the probability that the event is actually positive (negative) if the index is positive (negative). They may be derived from the contingency table by

PPV = TP/(TP+FP) (5)

NPV = TN/(TN+FN) (6)

Contrary to Se and Sp, PPV and NPV are dependent on the context through the prevalence. It implies that PPV and NPV, derived from the contingency table, may be used only if n+/N is close to the actual prevalence of the considered event in the considered "real-life" context.

The accuracy (Acc) defined as (TP+TN)/N is also context dependent.

It is to be noted that PPV and NPV may be computed from Se and Sp (intrinsic characteristics) and any given prevalence P (linked to a specific context) by the Baye's formula:

(7)

(8).

RESULTS

Operating characteristics of FeNO at various cut-off values

Table 1: Cross-sectional assessment of asthma control

Non-smokers (N=411, n+=197)
p<0.001
Cut-off
(ppb) / 30 / 40 / 50* / 60 / 70

TP

/ 112 / 134 / 142 / 171 / 179

TN

/ 150 / 131 / 120 / 86 / 58

FP

/ 64 / 83 / 94 / 128 / 156

FN

/ 85 / 63 / 55 / 26 / 18

Se (%)

/ 57 / 68 / 72 / 87 / 91
Sp (%) / 70 / 61 / 56 / 40 / 27
PPV (%) / 64 / 62 / 61 / 57 / 53
NPV (%) / 64 / 68 / 68 / 77 / 76
Acc (%) / 64 / 64 / 64 / 63 / 58
Smokers (N=59, n+=15)
p=0.39

Cut-off

(ppb) / 15 / 20 / 25* / 35 / 45

TP

/ 7 / 10 / 10 / 11 / 12

TN

/ 30 / 26 / 21 / 14 / 11

FP

/ 14 / 18 / 23 / 30 / 33

FN

/ 8 / 5 / 5 / 4 / 3

Se (%)

/ 48 / 64 / 66 / 76 / 80
Sp (%) / 68 / 59 / 48 / 32 / 24
PPV (%) / 33 / 36 / 30 / 27 / 27
NPV (%) / 79 / 84 / 81 / 78 / 79
Acc (%) / 63 / 61 / 53 / 42 / 39

*: cut-off value corresponding to maximal Youden's index. N and n+ are the total amount of cases and the amount of positive cases, respectively. A positive case is a (partially or totally) controlled asthma. TP, TN, FP, FN are the amount of true positive, true negative, false positive, and false negatives cases, respectively. Se, Sp, PPV, NPV, and Acc are the sensitivity, specificity, positive predictive value, negative predictive value, and accuracy, respectively; p is the statistical significance of rejecting the hypothesis AUC=0.5. A true positive event is defined as FeNO<cut-off associated with a (partially or totally) controlled asthma.

Table 2: Assessment of a change from an uncontrolled (ACQ score ³ 1.5) to a controlled (ACQ < 1.5) asthma

Non-smokers (N=283, n+=133)
p<0.001
Cut-off
(%change) / -20 / -25 / -30* / -35 / -40

TP

/ 101 / 96 / 90 / 88 / 82

TN

/ 93 / 98 / 107 / 113 / 120

FP

/ 57 / 52 / 43 / 37 / 30

FN

/ 32 / 37 / 43 / 45 / 51

Se (%)

/ 76 / 72 / 68 / 66 / 62
Sp (%) / 62 / 65 / 71 / 75 / 80
PPV (%) / 64 / 65 / 68 / 70 / 73
NPV (%) / 74 / 73 / 72 / 72 / 70
Acc (%) / 69 / 69 / 70 / 71 / 71
Smokers (N=52, n+=17)
p=0.016

Cut-off

(%change) / 15 / 20 / 25* / 35 / 45

TP

/ 12 / 12 / 12 / 11 / 9

TN

/ 22 / 23 / 23 / 23 / 24

FP

/ 13 / 12 / 12 / 12 / 11

FN

/ 5 / 5 / 5 / 6 / 8

Se (%)

/ 71 / 71 / 71 / 65 / 53
Sp (%) / 64 / 65 / 66 / 66 / 69
PPV (%) / 48 / 50 / 50 / 48 / 45
NPV (%) / 81 / 82 / 82 / 79 / 75
Acc (%) / 65 / 67 / 67 / 65 / 63

*: cut-off value corresponding to maximal Youden's index. N and n+ are the total amount of cases and the amount of positive cases, respectively. A positive case is an uncontrolled asthma becoming controlled. TP, TN, FP, FN are the amount of true positive, true negative, false positive, and false negatives cases, respectively. Se, Sp, PPV, NPV, and Acc are the sensitivity, specificity, positive predictive value, negative predictive value, and accuracy, respectively; p is the statistical significance of rejecting the hypothesis AUC=0.5. A true positive event is defined as FeNO change <cut-off (e.g. -40% vs -30%) associated with a positive case.

Table 3: Assessment of a change from a controlled (ACQ score < 1.5) to an uncontrolled (ACQ ³ 1.5) asthma

Non-smokers (N=360, n+=65)
p=0.001
Cut-off
(%change) / 40 / 45 / 50* / 55 / 60

TP

/ 29 / 29 / 27 / 27 / 26

TN

/ 212 / 215 / 221 / 224 / 227

FP

/ 83 / 80 / 74 / 71 / 68

FN

/ 36 / 36 / 38 / 38 / 39

Se (%)

/ 44 / 44 / 42 / 41 / 40
Sp (%) / 72 / 73 / 75 / 76 / 77
PPV (%) / 26 / 27 / 26 / 28 / 28
NPV (%) / 85 / 86 / 86 / 85 / 85
Acc (%) / 67 / 68 / 69 / 70 / 70
Smokers (N=40, n+=10)
p=0.017

Cut-off

(%change) / 40 / 45 / 50* / 55 / 60

TP

/ 7 / 7 / 7 / 6 / 6

TN

/ 24 / 25 / 26 / 26 / 26

FP

/ 6 / 5 / 4 / 4 / 4

FN

/ 3 / 3 / 3 / 4 / 4

Se (%)

/ 70 / 70 / 68 / 62 / 58
Sp (%) / 80 / 84 / 87 / 87 / 88
PPV (%) / 54 / 58 / 63 / 60 / 60
NPV (%) / 89 / 89 / 89 / 87 / 87
Acc (%) / 78 / 80 / 83 / 80 / 80

*: cut-off value corresponding to maximal Youden's index. N and n+ are the total amount of cases and the amount of positive cases, respectively. A positive case is a controlled asthma becoming uncontrolled. TP, TN, FP, FN are the amount of true positive, true negative, false positive, and false negatives cases, respectively. Se, Sp, PPV, NPV, and Acc are the sensitivity, specificity, positive predictive value, negative predictive value, and accuracy, respectively; p is the statistical significance of rejecting the hypothesis AUC=0.5. A true positive event is defined as FeNO change >cut-off (e.g. 60% vs 50%) associated with a positive case.

Table 4: Improvement (DACQ < -0.5) assessment of asthma control

Non-smokers
Total
(N=643, n+=257) / ICS dose £500$
(N=306, n+=116 / ICS dose 500$
(N=337, n+=97)
p<0.001 / p<0.001 / p<0.001
Cut-off
(%change) / -10 / -15 / -20* / -25 / -30 / -10 / -15 / -20* / -25 / -30 / -10 / -15 / -20* / -25 / -30

TP

/ 146 / 177 / 164 / 159 / 149 / 85 / 82 / 86 / 73 / 68 / 100 / 96 / 78 / 82 / 79

TN

/ 274 / 255 / 274 / 282 / 297 / 127 / 135 / 127 / 143 / 150 / 108 / 122 / 145 / 133 / 141

FP

/ 112 / 131 / 112 / 104 / 89 / 63 / 55 / 63 / 47 / 40 / 88 / 74 / 51 / 63 / 55

FN

/ 111 / 80 / 93 / 98 / 108 / 31 / 34 / 30 / 43 / 48 / 41 / 45 / 63 / 59 / 62

Se (%)

/ 57 / 69 / 64 / 62 / 58 / 73 / 71 / 74 / 63 / 59 / 71 / 68 / 55 / 58 / 56
Sp (%) / 71 / 66 / 71 / 73 / 77 / 67 / 71 / 67 / 75 / 79 / 55 / 62 / 74 / 68 / 72
PPV (%) / 57 / 57 / 61 / 62 / 63 / 57 / 60 / 58 / 61 / 63 / 53 / 56 / 60 / 57 / 59
NPV (%) / 71 / 76 / 74 / 74 / 73 / 80 / 80 / 80 / 77 / 76 / 72 / 73 / 70 / 69 / 69
Acc (%) / 65 / 67 / 68 / 69 / 69 / 69 / 71 / 70 / 71 / 71 / 62 / 65 / 66 / 64 / 65
Smokers
Total
(N=92, n+=40) / ICS dose £500$
(N=35, n+=14) / ICS dose >500$
(N=57, n+=26)
p<0.001 / p<0.001 / p=0.070

Cut-off

(%change) / -10 / -15 / -20* / -25 / -30 / -10 / -15 / -20* / -25 / -30 / -10 / -15 / -20* / -25 / -30

TP

/ 25 / 24 / 23 / 20 / 20 / 12 / 12 / 9 / 8 / 8 / 14 / 13 / 13 / 12 / 11

TN

/ 37 / 37 / 38 / 40 / 42 / 16 / 16 / 18 / 18 / 18 / 21 / 22 / 22 / 22 / 23

FP

/ 15 / 15 / 14 / 12 / 10 / 5 / 5 / 3 / 3 / 3 / 10 / 9 / 9 / 9 / 8

FN

/ 15 / 16 / 17 / 20 / 20 / 2 / 2 / 5 / 6 / 6 / 12 / 13 / 13 / 14 / 15

Se (%)

/ 62 / 60 / 57 / 51 / 49 / 88 / 86 / 62 / 54 / 54 / 54 / 50 / 50 / 46 / 42
Sp (%) / 72 / 72 / 74 / 76 / 80 / 74 / 74 / 84 / 84 / 84 / 68 / 71 / 71 / 71 / 74
PPV (%) / 63 / 62 / 62 / 63 / 67 / 71 / 71 / 75 / 73 / 73 / 58 / 59 / 59 / 57 / 58
NPV (%) / 71 / 70 / 70 / 67 / 68 / 89 / 89 / 78 / 75 / 75 / 64 / 63 / 63 / 61 / 61
Acc (%) / 67 / 66 / 66 / 65 / 67 / 80 / 80 / 77 / 74 / 74 / 61 / 61 / 61 / 60 / 60

*: cut-off value corresponding to maximal Youden's index. $ : ICS dose in mg equ BDP.day-1 . N and n+ are the total amount of cases and the amount of positive cases, respectively. A positive case is a decrease amplitude of ACQ score > 0.5. TP, TN, FP, FN are the amount of true positive, true negative, false positive, and false negatives cases, respectively. Se, Sp, PPV, NPV, and Acc are the sensitivity, specificity, positive predictive value, negative predictive value, and accuracy, respectively; p is the statistical significance of rejecting the hypothesis AUC=0.5. A true positive event is defined as FeNO change <cut-off (e.g. -30% vs -20%) associated with a positive case.

Table 5: Worsening (DACQ > 0.5) assessment of asthma control

Non-smokers
Total
(N=643, n+=161) / ICS dose £500$
(N=306, n+=64) / ICS dose 500$
(N=337, n+=97)
p<0.001 / p<0.001 / p<0.001
Cut-off
(%change) / 20 / 25 / 30* / 35 / 40 / 20 / 25 / 30* / 35 / 40 / 20 / 25 / 30* / 35 / 40

TP

/ 90 / 85 / 82 / 76 / 72 / 43 / 43 / 43 / 39 / 38 / 46 / 43 / 41 / 38 / 36

TN

/ 342 / 357 / 366 / 376 / 381 / 169 / 179 / 184 / 189 / 191 / 182 / 182 / 187 / 190 / 194

FP

/ 140 / 125 / 116 / 106 / 101 / 73 / 63 / 58 / 53 / 51 / 58 / 58 / 53 / 50 / 46

FN

/ 71 / 76 / 79 / 85 / 89 / 21 / 21 / 21 / 25 / 26 / 51 / 54 / 56 / 59 / 61

Se (%)

/ 56 / 53 / 51 / 47 / 45 / 67 / 67 / 67 / 61 / 59 / 47 / 44 / 42 / 39 / 37
Sp (%) / 71 / 74 / 76 / 78 / 79 / 70 / 74 / 76 / 78 / 79 / 76 / 76 / 78 / 79 / 81
PPV (%) / 37 / 40 / 37 / 42 / 42 / 37 / 41 / 43 / 42 / 43 / 44 / 43 / 44 / 43 / 44