Paperonium

← Back to the plain-language case

Reported two-tailed p = 0.046 for t(18) = 1.75; that statistic gives p = 0.097

J Neurosci (2021) Ā· doi:10.1523/JNEUROSCI.3216-18.2021 Ā· PMC7984597

Summary

The paper reports that a high-fat diet reduced surface expression of the insulin receptor β subunit (IRβ) in the nucleus accumbens (NAc) by 21.7%, tested with a "two-tailed unpaired t test: t(18) = 1.75, p = 0.046". For t = 1.75 with 18 degrees of freedom, the two-tailed p is 0.097, and no t that rounds to 1.75 gives less than 0.096. The printed 0.046 is close to the one-tailed value (0.049), but the paper calls this test two-tailed in the Results, in the Fig. 6 legend and in the Methods. If t and df are correct, the reduction is not significant at the conventional 0.05 threshold that the figure legends use ("p < 0.05"), yet the abstract reports it as a finding and the Discussion uses it as a possible mechanism for the loss of insulin's effect after obesity.

What the paper reports

Results, section "Diet-induced obesity blunts insulin receptor-mediated increases in excitatory transmission and reduces NAc insulin receptor surface expression", second paragraph:

One potential explanation for the loss of insulin-induced increases in excitatory transmission is a reduction in NAc insulin receptor expression.

In NAc tissue from chow and high-fat groups, we found a 21.7% (±7.6%) reduction in IRβ surface expression in high-fat versus chow-fed groups (Fig. 6D; two-tailed unpaired t test: t(18) = 1.75, p = 0.046) without any changes in total IRβ (Fig. 6E).

This suggests that reductions in NAc insulin receptor expression may contribute to the loss of insulin-induced increases in eEPSC amplitude in the high-fat group.

Fig. 6 legend, title and last sentence:

High-fat diet-induced obesity results in a loss of insulin-induced increases in excitatory transmission and a reduction in NAc IRβ surface expression.

Statistical differences were determined by two-tailed unpaired t tests (A,B: **p < 0.001), two-way repeated-measures ANOVA comparing BL and treatment conditions (C: *chow group, main effect of treatment; #high-fat group, main effect of treatment, p < 0.05), and two-tailed unpaired t test (D: p = 0.046).

Materials and Methods, "Experimental design and statistical analysis":

Two-tailed t tests, one-way or two-way repeated measures ANOVAs, and Sidak's post hoc multiple comparisons tests were conducted using Prism 6-8 software (GraphPad).

Fig. 1 legend:

Data in all figures are shown as the mean ± SEM.

Abstract:

Furthermore, obesity resulted in a loss of insulin receptor-mediated increases in excitatory transmission and a reduction in NAc insulin receptor surface expression, while preserving reductions in transmission mediated by IGF receptors.

Introduction, third paragraph:

Finally, diet-induced obesity resulted in a loss of insulin-induced increases in NAc excitatory transmission and a reduction in NAc insulin receptor surface expression.

Discussion, "Loss of insulin-receptor mediated effects following obesity":

The loss of insulin-induced increases may be due in part to modest reductions in NAc insulin receptor expression, as surface expression of IRβ was reduced following high-fat diet (Fig. 6D).

A reduction in insulin receptor expression is consistent with the development of insulin resistance in the face of chronic elevations in circulating insulin resulting from obesity

What the paper's own numbers show

Method. The paper gives the test (unpaired, two-tailed), the statistic (t = 1.75) and the degrees of freedom (18). From these alone, the two-tailed p is 2Ā·P(Tā‚ā‚ˆ ≄ 1.75), computed with scipy.stats.t. To allow for rounding, I repeated the calculation for every t that rounds to 1.75 (1.745 to 1.755). I also computed the |t| that a two-tailed p of 0.046 would need, and the lowest two-tailed p that t = 1.75 can give at any df (the normal limit). As a check on the method, I recomputed the p values of all 36 other t and F statistics printed in the Results. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed
Two-tailed p, t(18) = 1.75 Results §"Diet-induced obesity…", para 2; Fig. 6 legend (D) p = 0.046 0.0971 (0.0963 to 0.0980 for t of 1.745 to 1.755)
One-tailed p, t(18) = 1.75 same not reported 0.0486 (0.0481 to 0.0490)
Lowest two-tailed p for t = 1.75 at any df derived — 0.0801 (df → āˆž)
Two-tailed p for t = 1.75 at df = 9 (smallest Welch df if the 20 samples were split 10 + 10; unequal splits give lower df and higher p) derived — 0.1140
Absolute t needed for two-tailed p = 0.046 at df = 18 derived t = 1.75 2.143
Absolute t needed for two-tailed p = 0.05 at df = 18 (critical value, α = 0.05) derived — 2.101
t needed for one-tailed p = 0.046 at df = 18 derived — 1.780
Reported p Ć· recomputed two-tailed p derived — 0.474
SE of the difference implied by 21.7 / 1.75 Results "±7.6%" 12.40 percentage points
t if ±7.6 were the SE of the difference (21.7 / 7.6) Results t = 1.75 2.855 (two-tailed p = 0.0105)
t if both group SEMs were 7.6 Results t = 1.75 2.019 (two-tailed p = 0.0586)
Same comparison in bioRxiv v1 (2019) and v2 (2020) Results, both versions "two-tailed unpaired t-test t18 =1.75, p=0.05" (v2: "t18=1.75") 0.0971
Other 34 t and F statistics in Results Results, Figs. 1–6 various all agree with the printed p after rounding
Fig. 3J, mEPSC amplitude with HNMPA, t(5) = 7.75 Results, "Insulin-induced changes…" p = 0.006 0.00057 (p = 0.006 would need t = 4.57)
Fig. 5F, (–)-naloxone alone, t(9) = 0.53 Results, "Insulin-induced increases… disinhibition" p = 0.53 0.61 (p = 0.53 would need t = 0.65)

Rounding of t cannot close the gap: the whole rounding range gives two-tailed p of 0.096 to 0.098. Nor can any choice of degrees of freedom: with t = 1.75 the two-tailed p never falls below 0.080. The reported 0.046 is not exactly the one-tailed value either (0.049; a one-tailed 0.046 would need t = 1.78), but it is far closer to it than to the two-tailed value. Of the paper's other 36 t and F statistics, 34 match their printed p after rounding. The remaining two (Fig. 3J and Fig. 5F) have printed p values that differ from the recomputed ones, but neither difference changes whether the result is significant.

The "±7.6%" cannot settle which of t, p or the test description is wrong. The paper does not say what ±7.6% is. If it were the SE of the group difference, t would be 2.86, which does not match the printed t either. If it is the SEM of one group, expressed as a percentage of the chow mean, the other group's SEM would have to be 9.80 for t = 1.75, which is plausible. The group sizes (they must total 20 for df = 18) and the group means are not given, and there are no supplementary data.

Figure

The paper prints p = 0.046 for a two-tailed t(18) = 1.75; the statistic gives a two-tailed p of 0.097, above 0.05

Figure 1. p values for the high-fat vs chow comparison of NAc IRβ surface expression, on a log scale. The vertical line marks 0.05. Orange: the values printed for this test, p = 0.046 in the published article (filled) and p = 0.05 in both bioRxiv versions (open), all with the same t18 = 1.75. Blue: values recomputed by recompute.py.txt from t = 1.75 and df = 18. These are the two-tailed p that the paper's stated test gives (filled), the one-tailed p (open), and the lowest two-tailed p that t = 1.75 can give at any degrees of freedom (open). The ranges in brackets cover every t that rounds to 1.75; they are narrower than the markers.

Student t distribution with 18 df: each tail beyond ±1.75 holds 0.049, so the two-tailed p is 0.097; a two-tailed p of 0.046 needs |t| = 2.14

Figure 2. Where the two p values come from. The curve is the Student t density with 18 degrees of freedom. The orange solid line is the reported t = 1.75, and the orange dashed line is its mirror, āˆ’1.75. Each shaded tail beyond ±1.75 holds 0.049. The right tail alone is the one-tailed p, and both tails together are the two-tailed p, 0.097. The blue dotted line marks |t| = 2.14, the value a two-tailed p of 0.046 would need. All areas and positions come from recompute.py.txt.

Why it matters

Explanations considered

Status

Suggested correction

The authors should check the original analysis of Fig. 6D and correct whichever element is wrong. There are three possibilities:

In every case, the group sizes for Fig. 6D and the meaning of "±7.6%" (SEM of which quantity) should be stated. Separately, the p values for Fig. 3J (t(5) = 7.75 gives p = 0.0006, not 0.006) and Fig. 5F (t(9) = 0.53 gives p = 0.61, not 0.53) could be corrected at the same time.