← Back to the plain-language case
Key Points and abstract call the JAKi cancer risk significant though the paper's 95% CI includes 1
JAMA Netw Open (2024) · doi:10.1001/jamanetworkopen.2024.46336 · PMC11579790
Summary
The Key Points and the abstract's Conclusions and Relevance say that cancer risk was "statistically significantly" higher in people starting Janus kinase inhibitors (JAKis) than in people starting tumor necrosis factor inhibitors (TNFis). The paper's own estimate for that comparison is an adjusted hazard ratio (HR) of 1.36 with a 95% CI of 0.94-1.96. The interval includes 1. Working back from it gives a two-sided P of about 0.10, above the paper's stated threshold of 2-sided P < .05. The propensity-matched estimate (HR 1.79, 0.91-3.56; P about 0.09; shown as 1.8, 0.9-3.6 in eTable 10 of Supplement 1) and the crude rate ratio from Table 2 (1.21, 0.77-1.88) are not significant either, and the paper's Results, Discussion and main-text Conclusions all describe the JAKi result as not significant. The HR, its CI and its direction are reported correctly throughout. What is wrong is the significance label in the two most-read summaries, on the comparison of most safety interest (the JAKi cancer signal from the ORAL Surveillance trial).
What the paper reports
All passages are verbatim from the article as archived in PubMed Central.
The two summary statements:
"In this cohort study consisting of 25 305 individuals who initiated treatment with biologic or targeted synthetic disease-modifying antirheumatic drugs, a statistically significantly higher risk of incident cancer was seen with rituximab, abatacept, and JAKis compared with TNFis." (Key Points, Findings)
"In this cohort study of individuals with RA and new biologic or targeted synthetic DMARD exposures, individuals initiating rituximab, abatacept, and JAKis demonstrated higher incidence rates and statistically significantly increased risks of incident cancers compared with those initiating TNFis in the first 2 years after initiation of biologic or targeted synthetic DMARDs." (Abstract, Conclusions and Relevance)
The estimate those statements rest on, which appears in the abstract Results, the main Results and Table 2:
"Multivariable Cox proportional hazards regression analysis showed that rituximab was associated with a higher risk of incident cancer compared with TNFis (hazard ratio [HR], 1.91; 95% CI, 1.17-3.14), followed by abatacept (HR, 1.47; 95% CI, 1.03-2.11), and JAKis (HR, 1.36; 95% CI, 0.94-1.96)." (Abstract, Results)
"JAKi | 2570 (9) | 276 (169-351) | 22 (0.9) | 94 (59-143) | 1.36 (0.94-1.96)" (Table 2, JAKi row: exposures, median days to diagnosis, crude cancer diagnoses, diagnoses per 10 000 person-years, adjusted HR)
The significance convention:
"A 2-sided P < .05 was considered statistically significant." (Methods, Statistical Analysis)
The body of the paper describes the same result as not significant:
"Also like the primary analysis, we found that those treated with JAKis did not have a statistically significant increase in risk of cancer (HR, 1.79; 95% CI, 0.91-3.56) (eTable 10 and eFigure 4 in Supplement 1)." (Results, propensity-matched analysis paragraph)
"We observed that, among patients with RA, rituximab, abatacept, and JAKi initiations were associated with increased risks of incident cancer after 90 days and within 2 years of initiation, compared with TNFi initiation, although only rituximab and abatacept initiations had statistically significant associations." (Discussion, first paragraph)
"Our findings of increased incidence rate and risk of cancer with JAKi exposure compared with TNFi exposure are consistent with the results of ORAL Surveillance,6 although the results were not statistically significant." (Discussion, paragraph on prior JAKi studies)
"In this cohort study of individuals with RA and new biologic or targeted synthetic DMARD exposures, we found that new rituximab and abatacept initiators demonstrated higher incidence rates and statistically significantly increased risks of incident cancers compared with TNFi initiators in the first 2 years after biologic or targeted synthetic DMARD initiation." (Conclusions, main text)
What the paper's own numbers show
Method. Cox-model HRs come with Wald intervals on the log scale, so a reported HR and 95% CI fix the standard error: SE = [ln(upper) − ln(lower)] / (2 × 1.960). Then z = ln(HR) / SE and the two-sided P = 2 × [1 − Φ(|z|)]. This uses only the published numbers; the result is approximate because the bounds are rounded to two decimals. Before relying on it, I checked that the JAKi interval is symmetric about ln(1.36) on the log scale (0.369 below, 0.365 above), as a Wald interval should be. I applied the same calculation to every HR the paper reports, so the comparisons the paper correctly calls significant serve as a check. Separately, I recomputed the crude rates in Table 2 from their event counts. The person-years each rate implies give exact Poisson intervals that match the reported ones to within 1-2 units, so the inputs are internally consistent. From those I formed a crude JAKi/TNFi rate ratio and an exact conditional test. I did the same for the propensity-matched rates in eTable 10 of Supplement 1. All numbers come from recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| JAKi vs TNFi adjusted HR (95% CI) | Abstract Results; Results; Table 2 | 1.36 (0.94-1.96) | CI includes 1; geometric midpoint of the bounds 1.36 (consistent) |
| JAKi SE(ln HR), z | derived from Table 2 | not reported | SE 0.187; z 1.64 (z ≥ 1.96 needed for P < .05) |
| JAKi two-sided P | derived from Table 2 | not reported; Key Points Findings and abstract Conclusions and Relevance say "statistically significantly" | 0.10 (not < .05) |
| Smallest HR that would be significant at this SE | derived | n/a | 1.44 |
| JAKi one-sided P (not the stated convention) | derived | Methods: "2-sided P < .05" | 0.050 |
| JAKi propensity-matched HR (95% CI) | Results | 1.79 (0.91-3.56), "did not have a statistically significant increase" | P 0.094 (not significant; agrees with the text) |
| JAKi propensity-matched HR (95% CI) | Supplement 1, eTable 10 | 1.8 (0.9-3.6); 22 vs 14 cancers | same as the Results value rounded to one decimal; P about 0.10 from the rounded bounds |
| Propensity-matched crude rates per 10 000 PY, JAKi vs TNFi | Supplement 1, eTable 10 | 100 (63-151), 22 events; 60 (33-101), 14 events | imply about 2200 and 2333 PY; exact Poisson CIs 63-151 and 33-101 (exact match); rate ratio 1.67 (0.85-3.26), exact conditional P 0.14 |
| JAKi crude rate per 10 000 PY | Table 2 | 94 (59-143), 22 events | implies about 2340 PY; exact Poisson CI 59-142 |
| TNFi crude rate per 10 000 PY | Table 2 | 78 (66-91), 162 events | implies about 20 769 PY; exact Poisson CI 66-91 |
| Crude rate ratio JAKi/TNFi | derived from Table 2 | n/a | 1.21 (0.77-1.88); exact conditional P 0.39 |
| Rituximab vs TNFi HR | Table 2 | 1.91 (1.17-3.14) | P 0.010 (significant; agrees with the text) |
| Abatacept vs TNFi HR | Table 2 | 1.47 (1.03-2.11) | P 0.035 (significant; agrees with the text) |
| IL-6i vs TNFi HR | Table 2 | 1.04 (0.57-1.92) | P 0.90 |
| Propensity-matched rituximab, abatacept, IL-6i HRs | Results | 4.37 (1.48-12.93); 3.12 (1.52-6.44); 5.65 (1.11-28.79), all called significant | P 0.008; 0.002; 0.037 (agree with the text) |
Every HR the paper calls significant has a back-calculated P below .05 and a CI that excludes 1. The JAKi primary estimate is the only one the Key Points and abstract call significant that does not meet either criterion. No JAKi analysis reported anywhere in the paper or in Supplement 1 does.
Figure
Figure 1. The left panel (Key Points and Abstract) quotes the Key Points Findings and the abstract's Conclusions and Relevance, which group JAKis with rituximab and abatacept as having a "statistically significantly" increased cancer risk. The right panel gives the estimate those statements rest on (Table 2 and Results) and the body's own description of it: the Discussion says only rituximab and abatacept had significant associations, and the main-text Conclusions name only those two drugs. The strip below gives the reported HR and CI (orange) and the z and two-sided P recomputed from that CI (blue). All quotes are verbatim from the PMC version; the recomputed values come from recompute.py.txt.
Figure 2. Every adjusted HR versus TNFi that the paper reports, primary (Table 2) and propensity-matched (Results), drawn as the reported point estimate and 95% CI (orange) on one log-scale axis. The thin vertical line marks HR = 1. The right-hand columns give the reported values and the two-sided P recomputed from each CI (blue). Both JAKi rows (shaded) cross 1, with P = 0.101 and 0.094; the label on the primary JAKi row marks the estimate that the Key Points and abstract call "statistically significantly" higher. Every row the paper calls significant has P < .05.
Why it matters
The JAKi versus TNFi comparison is the question that motivates the study: the Importance section opens with the increased cancer risk with tofacitinib versus TNFis in ORAL Surveillance. The Key Points Findings sentence is the paper's headline result and is what many readers, abstracting services and news summaries carry forward. As written, it tells a reader that this US claims cohort found a statistically significant JAKi cancer signal. It did not. The point estimate (HR 1.36) is compatible with anything from a 6% lower hazard to a 96% higher hazard, and the paper's other JAKi analyses agree. A reader who takes only the Key Points or the abstract would count this study as independent statistical support for the ORAL Surveillance finding, when the paper's own Discussion calls the JAKi result "not statistically significant".
The rest of the paper is not affected. The HR, its CI and the direction of the JAKi estimate are reported correctly, the rituximab and abatacept results are significant as stated, and the authors' interpretation (channeling bias as a likely explanation) does not depend on the JAKi label. The error is confined to two sentences (one in the Key Points, one in the abstract), which makes it easy to correct.
Explanations considered
- Another JAKi analysis is being described. Not supported. The propensity-matched HR, 1.79 (0.91-3.56), has P about 0.09, and the Results call it not significant. Supplement 1, eTable 10, gives the same matched result as 1.8 (0.9-3.6), from 22 vs 14 cancers; the matched crude rate ratio it implies, 1.67 (0.85-3.26), has exact P 0.14. The crude rate ratio implied by Table 2, 1.21 (0.77-1.88), has P 0.39. No JAKi estimate anywhere in the paper or Supplement 1 has a CI that excludes 1.
- A one-sided test or a looser alpha. Not supported. The Methods state "A 2-sided P < .05 was considered statistically significant." The back-calculated two-sided P is 0.10. Only a one-sided test brings it to 0.050, which is still not below .05.
- "Significant" refers to the higher incidence rates, not the HR. Not supported. The wording is "statistically significantly higher risk" and "statistically significantly increased risks". The JAKi crude rate, 94 (59-143), overlaps the TNFi rate, 78 (66-91), and the crude rate ratio is not significant either.
- Transcription or rounding error in the HR or CI. Not supported. HR 1.36 is the geometric midpoint of 0.94-1.96. The interval is symmetric on the log scale, and the same numbers appear in the abstract, the Results and Table 2. The crude rates match exact Poisson intervals from their event counts to within 1-2 units.
- A conversion artifact in the archived text. Ruled out. Both bounds are positive decimals, so no minus sign can have been lost, and nothing depends on superscripts or table alignment. The conflicting statements are in prose, and the same wording appears on the live PMC page, in the Europe PMC full-text XML and in the PubMed abstract.
- Rounding at the threshold. Not applicable. A lower bound of 0.94 is not a rounded 1.00, and z = 1.64 is far from 1.96. With this SE, an HR of at least 1.44 would have been needed for significance.
None of these accounts for the wording of the Key Points and abstract. Since the body of the paper states the result correctly, those two sentences are inconsistent with the paper itself.
Status
As of 2026-10-23:
- Version of record. The publisher's full-text page (jamanetwork.com, article 2826510) returned an automated-traffic check and could not be read directly. The wording quoted above was confirmed in three publisher-supplied copies: the live PMC page, the Europe PMC full-text JATS XML, and the PubMed abstract (PMID 39565623). The 0.94-1.96 interval appears three times in the XML (abstract, Results, Table 2).
- Supplement. Supplement 1 (eTables 1-11, eFigures 1-4) was downloaded through Europe PMC and opened. eTable 9 gives the matched TNFi vs JAKi cohort (2421 per arm; 14 vs 22 cancer outcomes). eTable 10 (page 14, extractable text) gives the propensity-matched JAKi vs TNFi result: 22 vs 14 cancers, 100 (63-151) vs 60 (33-101) per 10 000 person-years, adjusted HR 1.8 (0.9-3.6), consistent with the HR 1.79 (0.91-3.56) in the Results. The supplement contains no alternative JAKi estimate or P value that would support the wording of the Key Points and abstract.
- Corrections. Crossref, PMC and Europe PMC list no correction, erratum, expression of concern or retraction notice. A web search found no erratum.
- PubPeer. A web search found no PubPeer thread for this article. PubPeer's own search returned a human-verification challenge and could not be queried directly.
- Sources: PMC11579790; PubMed 39565623; doi:10.1001/jamanetworkopen.2024.46336; Europe PMC full-text and supplementary-files endpoints for PMC11579790; Crossref metadata for the DOI.
Suggested correction
Bring the Key Points Findings sentence and the abstract's Conclusions and Relevance sentence into line with the Results, Discussion and main-text Conclusions. For example:
- Key Points, Findings: "…a statistically significantly higher risk of incident cancer was seen with rituximab and abatacept compared with TNFis; the higher risk seen with JAKis was not statistically significant (HR, 1.36; 95% CI, 0.94-1.96)."
- Abstract, Conclusions and Relevance: "…individuals initiating rituximab and abatacept demonstrated higher incidence rates and statistically significantly increased risks of incident cancers compared with those initiating TNFis…; individuals initiating JAKis had a higher incidence rate, but the increase in risk was not statistically significant."
The numerical results need no change.