Paperonium

← Back to the plain-language case

Abstract gives the headline smoking-initiation CI as 0.92–1.61; the body gives 0.92–1.16

PLoS Biol (2020) Β· doi:10.1371/journal.pbio.3000973 Β· PMC7728259

Summary

This two-sample Mendelian randomization study estimates the effect of genetically predicted smoking on multiple sclerosis (MS) risk. Its abstract gives the main smoking-initiation estimate as OR 1.03, 95% CI 0.92–1.61, but the Results and the Discussion give the same inverse-variance weighted (IVW) estimate as OR 1.03, 95% CI 0.92 to 1.16. On the log-odds scale, an OR of 1.03 with a lower limit of 0.92 implies an upper limit of 1.153, which matches 1.16. An interval of 0.92–1.61 would be centred on 1.22, and its upper arm would be 3.95 times as long as its lower arm, whereas every other interval in the paper is symmetric to within rounding. The published peer-review history shows that a reviewer raised this value and the response to reviewers called it a typo for 1.16, yet the version of record still prints 1.61. The null conclusion is unchanged, but the abstract interval is about 2.4 times too wide on the log scale. It also contains the observational OR of 1.5 that the paper's own Discussion says the MR result contradicts.

What the paper reports

Abstract:

There was no clear evidence for an effect of smoking on the risk of developing MS (smoking initiation: odds ratio [OR] 1.03, 95% confidence interval [CI] 0.92–1.61; lifetime smoking: OR 1.10, 95% CI 0.87–1.40). These findings suggest that smoking does not have a detrimental consequence on MS susceptibility.

Results, "Smoking initiation", first sentence:

The inverse-variance weighted (IVW) MR estimate (odds ratio [OR] 1.03, 95% confidence interval [CI] 0.92 to 1.16) revealed no strong evidence for a causal effect of the genetic risk of smoking initiation on the incidence of MS (Fig 2).

Results, "Smoking initiation", the sensitivity estimates:

the consistent MR–Egger estimate (OR 1.13, 95% CI 0.67 to 1.91)

gave a similar causal estimate (OR 1.05, 95% CI 0.93 to 1.17)

The MR-PRESSO outlier corrected causal estimate was 1.040 (95% CI 1.040 to 1.041).

Results, "Lifetime smoking":

estimate (OR 1.10, 95% CI 0.87 to 1.40) revealed no strong evidence for a causal effect of the genetic risk of lifetime smoking

which still overlapped the null (OR 1.34, 95% CI 0.49 to 3.65)

Discussion:

Although a small effect cannot be entirely excluded, the relatively narrow CIs, particularly for smoking initiation (0.92 to 1.16), make a clinically relevant effect less likely.

This contradicts previously reported observational studies that show an association with MS risk among smokers, compared to nonsmokers, of a meta-analysed effect estimate OR of 1.5

Methods, "Mendelian randomization analyses":

For smoking initiation, the ORs are expressed per unit increase in log odds of ever smoking regularly (smoking initiation)

Peer-review history published with the article (Decision Letter 1, reviewer comment, from the Europe PMC full-text XML):

Is the confidence interval for smoking cessation in the abstract correct? 1.03 (0.92, 1.61) - either the lower or upper bound seem incorrect.

Response to reviewers (attachment pbio.3000973.s002, "Response to reviewers .pdf", dated 2 Oct 2020):

Response: Thank you for pointing out this typo. The confidence intervals should instead read (0.92, 1.16). We have corrected this in the abstract of the revised manuscript.

The reviewer wrote "smoking cessation", but the only abstract interval matching "1.03 (0.92, 1.61)" is the smoking-initiation estimate, and the response treats it as that one.

What the paper's own numbers show

Method. MR estimates such as IVW, MR-Egger and MR-RAPS are computed on the log-odds scale, and their 95% intervals are exp(b Β± 1.96Β·SE). Three things follow. ln(OR) should sit at the midpoint of ln(lower) and ln(upper). The two arms, ln(OR/lower) and ln(upper/OR), should imply the same SE, so their ratio should be close to 1. And the OR and the lower limit together fix the upper limit at exp(2Β·ln OR βˆ’ ln lower). I applied these checks to every interval the text reports with an OR. MR-PRESSO is left out because its lower arm (1.040 to 1.040) is zero at the printed precision. All numbers below are printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Smoking initiation IVW, upper 95% limit Abstract 1.61 1.153, implied by OR 1.03 and lower limit 0.92
Smoking initiation IVW, upper 95% limit Results; Discussion 1.16 1.153, consistent
Implied upper limit over rounding of OR (1.025–1.035) and lower limit (0.915–0.925) derived 1.16 / 1.61 1.136–1.171: contains 1.16, excludes 1.61
Log-midpoint of 0.92–1.61 Abstract OR 1.03 1.217
Log-midpoint of 0.92–1.16 Results; Discussion OR 1.03 1.033
OR at which 0.92–1.61 would be symmetric (over rounding of the lower limit) derived OR 1.03 1.214–1.220
Upper arm Γ· lower arm, 0.92–1.61 Abstract β€” 3.95
Upper arm Γ· lower arm, 0.92–1.16 Results; Discussion β€” 1.05
Upper arm Γ· lower arm, MR-Egger 1.13 (0.67–1.91) Results β€” 1.00 (log-midpoint 1.131)
Upper arm Γ· lower arm, MR-RAPS 1.05 (0.93–1.17) Results β€” 0.89 (log-midpoint 1.043)
Upper arm Γ· lower arm, lifetime IVW 1.10 (0.87–1.40) Abstract; Results β€” 1.03 (log-midpoint 1.104)
Upper arm Γ· lower arm, lifetime MR-Egger 1.34 (0.49–3.65) Results β€” 1.00 (log-midpoint 1.337)
SE of log OR implied by the interval Abstract (0.92–1.61) vs Results (0.92–1.16) β€” 0.1428 vs 0.0591, a ratio of 2.41
Observational OR 1.5 inside the interval? Discussion; Abstract vs Results β€” inside 0.92–1.61; outside 0.92–1.16
OR 1 inside the interval? both β€” yes for both
Approximate Wald P for OR 1.03 with SE 0.0591 derived not reported 0.62

Every interval in the body is symmetric on the log scale to within rounding (arm ratios 0.89–1.05), as is the lifetime-smoking interval in the abstract. The abstract's smoking-initiation interval is the only exception. Its OR and lower limit match the Results exactly, and its upper limit has the same digits as the Results' 1.16 in reverse order.

Figure

Smoking-initiation IVW estimate as printed in the Abstract and in the Results/Discussion, and the interval implied by the OR and lower limit

Figure 1. The smoking-initiation IVW estimate on a log odds-ratio axis. Orange: the two versions printed in the paper. The Abstract prints OR 1.03 (0.92–1.61), and the Results and Discussion print OR 1.03 (0.92–1.16). Blue: the interval implied by the printed OR 1.03 and lower limit 0.92 under a symmetric interval on the log scale, with an upper limit of 1.153. The open diamond marks the log-scale midpoint of the Abstract interval (1.22), far from the printed OR. The solid line marks OR = 1. The dashed line marks the observational meta-analysed OR of 1.5 that the Discussion compares against. It lies inside the Abstract interval and outside the Results interval.

Arm-length ratio for every interval reported in the text

Figure 2. For each odds ratio and 95% CI printed in the text, the ratio of the upper arm to the lower arm on the log scale, ln(upper/OR) Γ· ln(OR/lower). A Wald interval gives 1. The six intervals from the Results and the Abstract's lifetime-smoking interval fall between 0.89 and 1.05, which is what rounding to two decimals produces. The Abstract's smoking-initiation interval, 0.92–1.61, gives 3.95.

Why it matters

The abstract is where most readers meet this result, and the 0.92–1.61 interval tells a different story from 0.92–1.16 in three ways.

The direction and statistical significance do not change. The point estimate is 1.03 in both places, and both intervals contain 1. Figures, tables, the lifetime-smoking estimate and the analyses themselves are unaffected. The error is in how one number in the abstract is reported.

Explanations considered

The digits of 1.61 are those of 1.16 in reverse order, which fits a transcription slip. The response to reviewers describes the value as a typo. No further explanation of the cause is offered here.

Status

Suggested correction

  1. In the Abstract, change "95% confidence interval [CI] 0.92–1.61" to "95% confidence interval [CI] 0.92–1.16" for smoking initiation. This is the correction the response to reviewers says was intended.
  2. No change is needed to the Results, the Discussion, the figures, the lifetime-smoking estimate or the conclusions as stated in the body. They already use 0.92 to 1.16.