Paperonium

← Back to the plain-language case

Abstract reports whole-cohort estimates as sexual-minority results, making two non-significant subgroup findings appear significant

BMJ Glob Health (2022) · doi:10.1136/bmjgh-2022-010231 · PMC9791420

Summary

The paper's second aim is to compare the mental health of sexual minority adolescents with and without HIV, and the abstract reports that "Sexual minority adolescents living with HIV had significantly higher odds of high general anxiety (AOR: 2.42; 95% CI: 1.21 to 4.84; p=0.013)", along with three significant estimates for sexual minority adolescents with unknown HIV status. All four of these estimates, with both CI limits and the p-value, match the HIV-status rows of Table 1 exactly. Table 1 is the whole-cohort model (N=1247, heterosexual and sexual minority adolescents together), and the main text itself labels 2.42 as the result "In the overall cohort". The sexual-minority-only model in Table 2 (N=497) gives different values, and two of the four claims are not significant there: HIV-positive status and high anxiety, AOR 2.09 (0.93 to 4.69), p=0.073; unknown status and suicidal attempt/ideation, AOR 1.47 (0.98 to 2.22), p=0.064. The paper's headline comparison of sexual minority and heterosexual adolescents is unaffected, but the abstract's Results sentence on the second aim misstates which findings reached significance and contradicts the abstract's own Conclusion.

What the paper reports

Abstract, Background (the second aim):

and to determine associations between the mental health status of sexual minority adolescents living with and without HIV.

Abstract, Methods:

A second multivariate regression model was developed to establish associations between HIV status among sexual minority individuals and the dependent variables.

Abstract, Results:

Sexual minority adolescents living with HIV had significantly higher odds of high general anxiety (AOR: 2.42; 95% CI: 1.21 to 4.84; p=0.013), while those with unknown HIV status had significantly higher odds of depressive symptoms (AOR: 3.82, 95% CI: 2.78 to 5.27; p<0.001), high general anxiety (AOR: 3.09; 95% CI: 2.29 to 4.17; p<0.001) and suicidal attempt/ideation (AOR: 1.65; 95% CI: 1.22 to 2.24; p=0.001).

Abstract, Conclusion:

Although there was no significant difference in the mental health status of sexual minority adolescents living and not living with HIV, sexual minority adolescents with unknown HIV status reported worse mental health than their HIV-negative peers.

Methods, Data analysis:

The second set of multivariate regression analyses included data from only sexual minority individuals.

Significance was set at 5%.

Results, third paragraph (describing Table 1):

In the overall cohort, self-reported HIV-positive status among adolescents was associated with increased odds of high general anxiety (AOR: 2.42; 95% CI:1.21 to 4.84; p=0.013), but not depression or suicidal attempt/ideation.

Results, fourth paragraph (describing Table 2):

Having an unknown HIV status was associated with significantly higher odds of experiencing depressive symptoms (AOR: 3.11, 95% CI: 1.76 to 5.49, p<0.001) and high general anxiety (AOR: 2.32, 95% CI: 1.55 to 3.48, p<0.001).

Table 1, "Multivariate logistic regression analysis to determine association between sexual orientation, HIV status, mental health status of adolescents 13–19 years old in Nigeria (N=1247)", HIV-positive row (columns: total | depressive symptoms yes | no | AOR (95% CI) p | high general anxiety yes | no | AOR (95% CI) p):

Positive | 41(3.3) | 29(70.7) | 12(29.3) | 1.81(0.85 to 3.88)0.125 | 17(41.5) | 24(58.5) | 2.42(1.21 to 4.84)0.013

Table 2, "Multivariate logistic regression analysis to determine association between HIV status and mental health status among sexual minority adolescents 13–19 years old in Nigeria (N=497)", HIV-positive row (columns as above, then suicidal ideation yes | no | AOR (95% CI) p):

HIV positive | 29 (5.8) | 24(82.8) | 5(17.2) | 2.25(0.78 to 6.48)0.132 | 14(48.3) | 15(51.7) | 2.09(0.93 to 4.69)0.073 | 12(41.4) | 17(58.6) | 1.47(0.66 to 3.28)0.352

Table 2, unknown-status row:

Unknown HIV status | 222 (44.7) | 201(90.5) | 21(9.5) | 3.11(1.76 to 5.49)<0.001 | 130(58.6) | 92(41.4) | 2.32(1.55 to 3.48)<0.001 | 109(49.1) | 113(50.9) | 1.47(0.98 to 2.22)0.064

Results, fifth paragraph (exploratory analyses, describing supplementary Table S1, heterosexual adolescents only):

Exploratory analyses among heterosexual participants showed that, like sexual minority individuals, unknown HIV status was associated with an increased likelihood of reporting depressive and anxiety symptoms and self-harm/suicidal ideation in this study. The adjusted ORs were comparable in these latter multivariate models among heterosexual and sexual minority participants, though these were relatively larger in sexual minority individuals (see online supplemental table 1).

Supplementary Table S1 (N=750), unknown-status estimates for depressive symptoms, high general anxiety and suicidal ideation:

3.92 (2.66-5.78) <0.001 | 4.16 (2.63-6.56) <0.001 | 2.06 (1.28-3.30) 0.003

Discussion, first paragraph:

Among sexual minority adolescents, those with unknown HIV status had significantly higher odds of reporting depressive symptoms and generalised anxiety disorder. These findings partially support the hypotheses.

What the paper's own numbers show

Method. I matched each of the four estimates in the abstract's Results sentence on HIV status against the HIV-status rows of Table 1 (whole cohort) and Table 2 (sexual minorities only), comparing the AOR, both CI limits and the p-value. For every estimate I recomputed the Wald z and two-sided p implied by the printed 95% CI (SE = [ln(upper) − ln(lower)] / 3.92; z = ln(AOR)/SE), to confirm that each table is internally consistent and that the printed p-values are two-sided. As a model-free check on the key subgroup estimate, I computed crude odds ratios from the Table 2 counts. Finally, I checked that Table 2 and supplementary Table S1 (heterosexual adolescents only, N=750) partition the Table 1 cohort, which confirms which table covers which population. I also compared the unknown-status estimates in Table S1 with those in Table 2, as the Results' exploratory-analysis paragraph does. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
HIV-positive vs negative, high general anxiety, given for sexual minority adolescents Abstract, Results 2.42 (1.21 to 4.84), p=0.013 Identical to Table 1 (whole cohort); Wald p from CI 0.0125
Same estimate, whole cohort Table 1; Results ¶3 ("In the overall cohort") 2.42 (1.21 to 4.84), p=0.013 SE 0.354, z 2.50, p 0.0125; CI excludes 1
Same estimate, sexual minorities only Table 2 2.09 (0.93 to 4.69), p=0.073 SE 0.413, z 1.79, p 0.074; CI includes 1, not significant
Crude OR, same comparison, sexual minorities Table 2 counts, 14/29 vs 79/246 — 1.97 (0.91 to 4.29), p=0.086
Unknown vs negative, depressive symptoms, given for sexual minority adolescents Abstract, Results 3.82 (2.78 to 5.27), p<0.001 Identical to Table 1; Table 2 value is 3.11 (1.76 to 5.49), p<0.001
Unknown vs negative, high general anxiety, given for sexual minority adolescents Abstract, Results 3.09 (2.29 to 4.17), p<0.001 Identical to Table 1; Table 2 value is 2.32 (1.55 to 3.48), p<0.001
Unknown vs negative, suicidal attempt/ideation, given for sexual minority adolescents Abstract, Results 1.65 (1.22 to 2.24), p=0.001 Identical to Table 1 (Wald p 0.0012); Table 2 value is 1.47 (0.98 to 2.22), p=0.064 (Wald p 0.065); not significant
Abstract estimates that match Table 1 / Table 2 exactly Abstract vs Tables 1 and 2 — 4 of 4 / 0 of 4
Abstract estimates with CI excluding 1 Abstract vs Table 2 4 of 4 2 of 4 for the same estimates in Table 2
Ratio of abstract AOR to Table 2 AOR Abstract vs Table 2 — 1.16, 1.23, 1.33, 1.12 (the abstract's values are larger than the subgroup estimates for all four)
HIV-positive vs negative, depressive symptoms / suicidal attempt/ideation, sexual minorities Table 2 2.25 (0.78 to 6.48), p=0.132 / 1.47 (0.66 to 3.28), p=0.352 Wald p 0.133 / 0.346; neither significant (abstract does not report these)
Unknown vs negative, depressive symptoms / high general anxiety / suicidal attempt/ideation, heterosexuals vs sexual minorities Table S1 vs Table 2; Results ¶5 ("like sexual minority individuals ... self-harm/suicidal ideation"; AORs "relatively larger in sexual minority individuals") S1: 3.92 / 4.16 / 2.06 (p=0.003); Table 2: 3.11 / 2.32 / 1.47 (p=0.064) The Table S1 AOR is larger than the Table 2 AOR for 3 of 3 outcomes; the Table 2 suicidal-ideation CI includes 1
Sample sizes of the HIV-status rows Tables 1, 2 and S1 N=1247, 497, 750 Rows sum to 1247, 497 and 750; Table 2 + Table S1 = Table 1 for each status (246+549=795; 29+12=41; 222+189=411)

The Wald p-values recomputed from the printed CIs agree with the printed p-values for every estimate in both tables (largest difference 0.009, for a non-significant Table 1 estimate, 0.703 vs 0.712), so both tables are internally consistent, the printed p-values are two-sided, and the discrepancy lies only in the abstract's attribution of Table 1 values to the sexual-minority subgroup.

A second, smaller inconsistency in Table 1. In the depressive-symptom and high-anxiety columns of Table 1, the yes/no counts in the "Negative" and "Unknown" HIV rows do not add up to those rows' totals: "Negative" (n=795) shows 332/79 and 195/216, which sum to 411, and "Unknown" (n=411) shows 330/465 and 126/669, which sum to 795. Adding the Table 2 and Table S1 counts gives 330/465 and 126/669 for HIV-negative adolescents and 332/79 and 195/216 for those with unknown status, so these count cells (and their percentages, e.g. 332/411 = 80.8%) appear to have been placed in each other's rows. The suicidal-ideation column and the HIV-positive row add up correctly, and the column totals (691, 338, 309) match the Results text. This does not change any AOR, but it reverses the descriptive picture: as printed, Table 1 shows 80.8% of HIV-negative and 41.5% of unknown-status adolescents with depressive symptoms, when the subgroup tables imply the reverse.

Figure

The four estimates the abstract gives for sexual minority adolescents are the whole-cohort Table 1 estimates; in the sexual-minority model (Table 2) two of them are not significant.

Figure 1. Left: the abstract's Results sentence on sexual minority adolescents by HIV status, with two of its four estimates (orange). Top right: the same two rows of Table 1, the whole-cohort model (N=1247), which carry identical numbers; the other two abstract estimates (3.82 and 3.09) also match Table 1 exactly. Bottom right: the corresponding rows of Table 2, the model restricted to sexual minority adolescents (N=497), which is the population the abstract names (blue). Both Table 2 confidence intervals include 1, so neither association is significant at the paper's stated 5% level. All numbers are printed in the paper; recompute.py.txt checks each p-value against its CI.

Dot-and-interval plot of the four abstract estimates against the Table 2 sexual-minority estimates.

Figure 2. Each of the four estimates the abstract reports for sexual minority adolescents (orange circles, identical to Table 1, whole cohort) next to the same estimate from the sexual-minority-only model in Table 2 (blue squares), on a log odds-ratio axis with the null value (OR = 1) as a thin vertical reference line. All four Table 2 estimates are smaller, and the intervals for HIV-positive status with high general anxiety and for unknown status with suicidal attempt/ideation cross 1.

Why it matters

The comparison of sexual minority adolescents living with and without HIV is one of the paper's two stated aims and has its own model (Table 2). The abstract's Results sentence on that aim reports four significant associations, but all four come from the whole-cohort model, where heterosexual adolescents make up 60% of the sample. In the subgroup the abstract names, HIV-positive status is not significantly associated with any of the three outcomes, and unknown status is not significantly associated with suicidal attempt/ideation; the two effects that do remain significant are 19% to 25% smaller than the abstract states (3.11 vs 3.82 and 2.32 vs 3.09).

The abstract is the part of the paper most often read, indexed and quoted. A reader who relies on it would conclude that HIV-positive sexual minority adolescents in this sample had significantly higher odds of high anxiety, and that unknown HIV status was associated with suicidal attempt/ideation in sexual minority adolescents. Neither is what the paper's subgroup analysis found. The abstract also contradicts itself: its Conclusion ("no significant difference in the mental health status of sexual minority adolescents living and not living with HIV") matches Table 2, while its Results sentence does not. Results paragraphs 3 and 4, the Discussion's summary of findings (its first paragraph) and the tables report the subgroup findings correctly; the Discussion's later paragraph on HIV status agrees with Table 2 that HIV-positive status had no significant effect, although it describes the comparison as one with "heterosexual peers" rather than with HIV-negative sexual minority peers, which is the Table 2 comparison. One sentence in the exploratory-analysis paragraph (Results, fifth paragraph) repeats the suicidal-ideation claim ("like sexual minority individuals, unknown HIV status was associated with ... self-harm/suicidal ideation"), and the same paragraph says the adjusted ORs were "relatively larger in sexual minority individuals", although the heterosexual unknown-status AORs in Table S1 (3.92, 4.16, 2.06) are larger than the sexual-minority ones in Table 2 (3.11, 2.32, 1.47) for all three outcomes. The headline comparison of sexual minority with heterosexual adolescents (AORs 5.54, 3.56 and 2.95) is not affected.

Explanations considered

Status

Suggested correction

Replace the abstract's Results sentence on HIV status with the estimates from the sexual-minority model (Table 2), for example:

Among sexual minority adolescents, HIV-positive status was not significantly associated with depressive symptoms, high general anxiety (AOR: 2.09; 95% CI: 0.93 to 4.69; p=0.073) or suicidal attempt/ideation, while those with unknown HIV status had significantly higher odds of depressive symptoms (AOR: 3.11; 95% CI: 1.76 to 5.49; p<0.001) and high general anxiety (AOR: 2.32; 95% CI: 1.55 to 3.48; p<0.001), but not of suicidal attempt/ideation (AOR: 1.47; 95% CI: 0.98 to 2.22; p=0.064).

If the authors prefer to keep the whole-cohort estimates in the abstract, the sentence should say that they come from the whole cohort (sexual minority and heterosexual adolescents together, the model reported in Table 1), and the Table 2 results should be added for the sexual-minority comparison that the second aim and the Conclusion refer to.

In the exploratory-analysis paragraph of the Results, the phrase "like sexual minority individuals" should not extend to self-harm/suicidal ideation, which was not significantly associated with unknown status among sexual minority adolescents (Table 2: AOR 1.47; 95% CI: 0.98 to 2.22; p=0.064), and the statement that the adjusted ORs were "relatively larger in sexual minority individuals" should be checked against Table S1, whose unknown-status AORs are the larger ones.

In Table 1, the depressive-symptom and high-general-anxiety counts and percentages in the "Negative" and "Unknown" HIV-status rows should be checked against the data; the subgroup tables imply that they belong in each other's rows. Two CIs in the education rows also cannot contain their point estimates and look like typographical errors: Table 1, primary education, suicidal ideation, "0.28(1.13 to 0.64)", and Table 2, secondary education, depressive symptoms, "0.58(1.19 to 1.76)".