Paperonium

← Back to the plain-language case

Key Points and abstract call the JAKi cancer risk significant though the paper's 95% CI includes 1

JAMA Netw Open (2024) · doi:10.1001/jamanetworkopen.2024.46336 · PMC11579790

Summary

The Key Points and the abstract's Conclusions and Relevance say that cancer risk was "statistically significantly" higher in people starting Janus kinase inhibitors (JAKis) than in people starting tumor necrosis factor inhibitors (TNFis). The paper's own estimate for that comparison is an adjusted hazard ratio (HR) of 1.36 with a 95% CI of 0.94-1.96. The interval includes 1. Working back from it gives a two-sided P of about 0.10, above the paper's stated threshold of 2-sided P < .05. The propensity-matched estimate (HR 1.79, 0.91-3.56; P about 0.09; shown as 1.8, 0.9-3.6 in eTable 10 of Supplement 1) and the crude rate ratio from Table 2 (1.21, 0.77-1.88) are not significant either, and the paper's Results, Discussion and main-text Conclusions all describe the JAKi result as not significant. The HR, its CI and its direction are reported correctly throughout. What is wrong is the significance label in the two most-read summaries, on the comparison of most safety interest (the JAKi cancer signal from the ORAL Surveillance trial).

What the paper reports

All passages are verbatim from the article as archived in PubMed Central.

The two summary statements:

"In this cohort study consisting of 25 305 individuals who initiated treatment with biologic or targeted synthetic disease-modifying antirheumatic drugs, a statistically significantly higher risk of incident cancer was seen with rituximab, abatacept, and JAKis compared with TNFis." (Key Points, Findings)

"In this cohort study of individuals with RA and new biologic or targeted synthetic DMARD exposures, individuals initiating rituximab, abatacept, and JAKis demonstrated higher incidence rates and statistically significantly increased risks of incident cancers compared with those initiating TNFis in the first 2 years after initiation of biologic or targeted synthetic DMARDs." (Abstract, Conclusions and Relevance)

The estimate those statements rest on, which appears in the abstract Results, the main Results and Table 2:

"Multivariable Cox proportional hazards regression analysis showed that rituximab was associated with a higher risk of incident cancer compared with TNFis (hazard ratio [HR], 1.91; 95% CI, 1.17-3.14), followed by abatacept (HR, 1.47; 95% CI, 1.03-2.11), and JAKis (HR, 1.36; 95% CI, 0.94-1.96)." (Abstract, Results)

"JAKi | 2570 (9) | 276 (169-351) | 22 (0.9) | 94 (59-143) | 1.36 (0.94-1.96)" (Table 2, JAKi row: exposures, median days to diagnosis, crude cancer diagnoses, diagnoses per 10 000 person-years, adjusted HR)

The significance convention:

"A 2-sided P < .05 was considered statistically significant." (Methods, Statistical Analysis)

The body of the paper describes the same result as not significant:

"Also like the primary analysis, we found that those treated with JAKis did not have a statistically significant increase in risk of cancer (HR, 1.79; 95% CI, 0.91-3.56) (eTable 10 and eFigure 4 in Supplement 1)." (Results, propensity-matched analysis paragraph)

"We observed that, among patients with RA, rituximab, abatacept, and JAKi initiations were associated with increased risks of incident cancer after 90 days and within 2 years of initiation, compared with TNFi initiation, although only rituximab and abatacept initiations had statistically significant associations." (Discussion, first paragraph)

"Our findings of increased incidence rate and risk of cancer with JAKi exposure compared with TNFi exposure are consistent with the results of ORAL Surveillance,6 although the results were not statistically significant." (Discussion, paragraph on prior JAKi studies)

"In this cohort study of individuals with RA and new biologic or targeted synthetic DMARD exposures, we found that new rituximab and abatacept initiators demonstrated higher incidence rates and statistically significantly increased risks of incident cancers compared with TNFi initiators in the first 2 years after biologic or targeted synthetic DMARD initiation." (Conclusions, main text)

What the paper's own numbers show

Method. Cox-model HRs come with Wald intervals on the log scale, so a reported HR and 95% CI fix the standard error: SE = [ln(upper) − ln(lower)] / (2 × 1.960). Then z = ln(HR) / SE and the two-sided P = 2 × [1 − Φ(|z|)]. This uses only the published numbers; the result is approximate because the bounds are rounded to two decimals. Before relying on it, I checked that the JAKi interval is symmetric about ln(1.36) on the log scale (0.369 below, 0.365 above), as a Wald interval should be. I applied the same calculation to every HR the paper reports, so the comparisons the paper correctly calls significant serve as a check. Separately, I recomputed the crude rates in Table 2 from their event counts. The person-years each rate implies give exact Poisson intervals that match the reported ones to within 1-2 units, so the inputs are internally consistent. From those I formed a crude JAKi/TNFi rate ratio and an exact conditional test. I did the same for the propensity-matched rates in eTable 10 of Supplement 1. All numbers come from recompute.py.txt.

Quantity Location Reported Recomputed
JAKi vs TNFi adjusted HR (95% CI) Abstract Results; Results; Table 2 1.36 (0.94-1.96) CI includes 1; geometric midpoint of the bounds 1.36 (consistent)
JAKi SE(ln HR), z derived from Table 2 not reported SE 0.187; z 1.64 (z ≥ 1.96 needed for P < .05)
JAKi two-sided P derived from Table 2 not reported; Key Points Findings and abstract Conclusions and Relevance say "statistically significantly" 0.10 (not < .05)
Smallest HR that would be significant at this SE derived n/a 1.44
JAKi one-sided P (not the stated convention) derived Methods: "2-sided P < .05" 0.050
JAKi propensity-matched HR (95% CI) Results 1.79 (0.91-3.56), "did not have a statistically significant increase" P 0.094 (not significant; agrees with the text)
JAKi propensity-matched HR (95% CI) Supplement 1, eTable 10 1.8 (0.9-3.6); 22 vs 14 cancers same as the Results value rounded to one decimal; P about 0.10 from the rounded bounds
Propensity-matched crude rates per 10 000 PY, JAKi vs TNFi Supplement 1, eTable 10 100 (63-151), 22 events; 60 (33-101), 14 events imply about 2200 and 2333 PY; exact Poisson CIs 63-151 and 33-101 (exact match); rate ratio 1.67 (0.85-3.26), exact conditional P 0.14
JAKi crude rate per 10 000 PY Table 2 94 (59-143), 22 events implies about 2340 PY; exact Poisson CI 59-142
TNFi crude rate per 10 000 PY Table 2 78 (66-91), 162 events implies about 20 769 PY; exact Poisson CI 66-91
Crude rate ratio JAKi/TNFi derived from Table 2 n/a 1.21 (0.77-1.88); exact conditional P 0.39
Rituximab vs TNFi HR Table 2 1.91 (1.17-3.14) P 0.010 (significant; agrees with the text)
Abatacept vs TNFi HR Table 2 1.47 (1.03-2.11) P 0.035 (significant; agrees with the text)
IL-6i vs TNFi HR Table 2 1.04 (0.57-1.92) P 0.90
Propensity-matched rituximab, abatacept, IL-6i HRs Results 4.37 (1.48-12.93); 3.12 (1.52-6.44); 5.65 (1.11-28.79), all called significant P 0.008; 0.002; 0.037 (agree with the text)

Every HR the paper calls significant has a back-calculated P below .05 and a CI that excludes 1. The JAKi primary estimate is the only one the Key Points and abstract call significant that does not meet either criterion. No JAKi analysis reported anywhere in the paper or in Supplement 1 does.

Figure

Two text panels contrast the Key Points and abstract significance claim for JAKis with the body's estimate and statements

Figure 1. The left panel (Key Points and Abstract) quotes the Key Points Findings and the abstract's Conclusions and Relevance, which group JAKis with rituximab and abatacept as having a "statistically significantly" increased cancer risk. The right panel gives the estimate those statements rest on (Table 2 and Results) and the body's own description of it: the Discussion says only rituximab and abatacept had significant associations, and the main-text Conclusions name only those two drugs. The strip below gives the reported HR and CI (orange) and the z and two-sided P recomputed from that CI (blue). All quotes are verbatim from the PMC version; the recomputed values come from recompute.py.txt.

Forest plot of every reported HR on a log axis with recomputed two-sided P values

Figure 2. Every adjusted HR versus TNFi that the paper reports, primary (Table 2) and propensity-matched (Results), drawn as the reported point estimate and 95% CI (orange) on one log-scale axis. The thin vertical line marks HR = 1. The right-hand columns give the reported values and the two-sided P recomputed from each CI (blue). Both JAKi rows (shaded) cross 1, with P = 0.101 and 0.094; the label on the primary JAKi row marks the estimate that the Key Points and abstract call "statistically significantly" higher. Every row the paper calls significant has P < .05.

Why it matters

The JAKi versus TNFi comparison is the question that motivates the study: the Importance section opens with the increased cancer risk with tofacitinib versus TNFis in ORAL Surveillance. The Key Points Findings sentence is the paper's headline result and is what many readers, abstracting services and news summaries carry forward. As written, it tells a reader that this US claims cohort found a statistically significant JAKi cancer signal. It did not. The point estimate (HR 1.36) is compatible with anything from a 6% lower hazard to a 96% higher hazard, and the paper's other JAKi analyses agree. A reader who takes only the Key Points or the abstract would count this study as independent statistical support for the ORAL Surveillance finding, when the paper's own Discussion calls the JAKi result "not statistically significant".

The rest of the paper is not affected. The HR, its CI and the direction of the JAKi estimate are reported correctly, the rituximab and abatacept results are significant as stated, and the authors' interpretation (channeling bias as a likely explanation) does not depend on the JAKi label. The error is confined to two sentences (one in the Key Points, one in the abstract), which makes it easy to correct.

Explanations considered

None of these accounts for the wording of the Key Points and abstract. Since the body of the paper states the result correctly, those two sentences are inconsistent with the paper itself.

Status

As of 2026-10-23:

Suggested correction

Bring the Key Points Findings sentence and the abstract's Conclusions and Relevance sentence into line with the Results, Discussion and main-text Conclusions. For example:

The numerical results need no change.