Paperonium

← Back to the plain-language case

Correlations printed as significant in this insulin-clearance study are too small to reach P < 0.05

Diabetes Care (2023) Β· doi:10.2337/dc23-1267 Β· PMC10698223

Summary

The paper reports that insulin clearance correlates negatively with liver lipid content (HCL) in type 2 diabetes (T2D) and with HbA1c, each at "P < 0.05", but prints coefficients such as r = βˆ’0.004, βˆ’0.005 and βˆ’0.037 in a T2D group of at most 489 people and r = βˆ’0.058 and βˆ’0.127 in a control group of at most 167. At those sizes |r| must be at least 0.089 (T2D) or 0.152 (controls) to reach two-sided P < 0.05; the recomputed P values for nine correlations described as significant run from 0.10 to 0.99, and seven of them underlie statements in the abstract. These correlations are the only correlational evidence for the abstract's conclusion that clearance "correlates negatively with liver lipid content rather in T2D". The regression lines drawn in Fig. 3 are far steeper than the printed coefficients allow; the paper does not show whether the coefficients, their labels or the significance statements are wrong.

What the paper reports

Abstract, Results:

HCL negatively associated with ICIVGTT and ICMMT in T2D (r = -0.005 and r = -0.037) and CON (r = -0.127 and r = -0.058, all P < 0.05).

In T1D, ICCLAMP, ICIVGTT, and ICMMT correlated negatively with HbA1c (all P < 0.05).

Abstract, Conclusions:

Insulin clearance is reduced in both T1D and T2D within the first year after diagnosis but correlates negatively with liver lipid content rather in T2D.

Results, Correlation Analyses (T2D):

In T2D, ICCLAMP correlated negatively with HbA1c (r = -0.004; P < 0.05) (Fig. 3B) but not with HCL or M value (Supplementary Fig. 3A and C).

On the other hand, ICIVGTT correlated positively with M value (r = -0.178; P < 0.05) (Supplementary Fig. 3F) and negatively with HCL (r = -0.005; P < 0.05) (Fig. 3D). ICMMT correlated negatively with HCL (r = -0.037; P < 0.05) (Fig. 3E).

Results, Correlation Analyses (controls, CON):

Similarly, ICMMT correlated positively with whole-body insulin sensitivity (r = 0.047; P < 0.05) (Supplementary Fig. 3I) and negatively with HCL (r = -0.058; P < 0.05) (Supplementary Fig. 3G) in CON.

Research Design and Methods, Statistics:

Pearson or Spearman correlations were used for numerical variables.

A two-tailed P value <0.05 was considered to indicate a statistically significant difference or correlation.

Figure 3 legend (Supplementary Fig. 3 uses similar wording):

Statistically significant correlations are depicted with linear regression lines and the respective coefficients (r).

Conclusions section (the article's discussion), final paragraph:

3) correlates with liver lipid content rather in T2D and with impaired glycemic control rather in T1D

The group sizes are CON 167, T1D 306 and T2D 489 (Abstract; Table 1). The Supplementary Fig. 3 legends (figshare 10.2337/figshare.24156732) print the same coefficients next to each regression line. They also give the three T1D coefficients behind the abstract's "In T1D ... all P < 0.05": r = βˆ’0.001 (ICCLAMP), βˆ’0.002 (ICIVGTT) and βˆ’0.012 (ICMMT), each against HbA1c.

What the paper's own numbers show

Method. The usual test of a correlation coefficient is t = r·√((n βˆ’ 2)/(1 βˆ’ rΒ²)) with n βˆ’ 2 degrees of freedom. I computed the two-sided P for each printed r at the full group size, which is the largest n any analysis could have had and so gives the smallest possible P. A smaller analysis sample only makes P larger. For the HCL correlations I also used the smaller number of participants with an HCL measurement, taken from the per-bar n printed under Fig. 2 (participants with plus without steatosis). Fisher's z, the usual large-sample test for Spearman's ρ, gives P values that differ from these by at most 0.001. I also inverted the test to find the critical |r| at each n.

Quantity Location Reported Recomputed
Critical |r| for two-sided P < 0.05 Statistics ("two-tailed") β€” T2D (n = 489): 0.089; T1D (n = 306): 0.112; CON (n = 167): 0.152; T2D with HCL (n = 129–143): 0.164–0.173
T2D ICIVGTT vs HCL Abstract; Results; Fig. 3C; Supp. Fig. 3D r = βˆ’0.005, P < 0.05 P = 0.91 (n = 489); 0.95 (n = 129)
T2D ICMMT vs HCL Abstract; Results; Fig. 3E; Supp. Fig. 3G r = βˆ’0.037, P < 0.05 P = 0.41 (n = 489); 0.66 (n = 143)
CON ICIVGTT vs HCL Abstract; Results; Supp. Fig. 3D r = βˆ’0.127, P < 0.05 P = 0.10 (n = 167); 0.20 (n = 105)
CON ICMMT vs HCL Abstract; Results; Supp. Fig. 3G r = βˆ’0.058, P < 0.05 P = 0.46 (n = 167); 0.53 (n = 120)
T2D ICCLAMP vs HbA1c Results; Fig. 3B; Supp. Fig. 3B r = βˆ’0.004, P < 0.05 P = 0.93 (n = 489)
T1D ICCLAMP vs HbA1c Abstract and Results ("all P < 0.05"); Fig. 3B; Supp. Fig. 3B r = βˆ’0.001 P = 0.99 (n = 306)
T1D ICIVGTT vs HbA1c Abstract and Results ("all P < 0.05"); Fig. 3D; Supp. Fig. 3E r = βˆ’0.002 P = 0.97 (n = 306)
T1D ICMMT vs HbA1c Abstract and Results ("all P < 0.05"); Fig. 3F (line drawn, no label); Supp. Fig. 3H r = βˆ’0.012 P = 0.83 (n = 306)
CON ICMMT vs M value Results; Supp. Fig. 3I r = 0.047, P < 0.05 P = 0.55 (n = 167)
n needed for P < 0.05 β€” β€” |r| = 0.004: 240,092; 0.005: 153,659; 0.037: 2,807; 0.058: 1,143; 0.127: 239
For comparison: CON ICIVGTT vs M value Abstract; Results; Supp. Fig. 3F r = 0.199, P < 0.05 P = 0.0099 (consistent)
For comparison: T2D ICIVGTT vs M value Abstract (r = 0.178); Results (r = βˆ’0.178, called "positively") P < 0.05 P = 7.6 Γ— 10⁻⁡ (consistent; the sign differs between Abstract and Results)
For comparison: CON ICMMT vs M value Abstract (r = 0.176); Results and Supp. Fig. 3I (r = 0.047) P < 0.05 P = 0.023 for 0.176; 0.55 for 0.047
For comparison: all groups ICIVGTT vs HbA1c Results ("negatively associated") r = 0.092, P < 0.05 P = 0.0043 (significance consistent; the positive sign contradicts "negatively")

The coefficients that do reach significance (r = 0.199, 0.178, 0.176, 0.092) show that the test applied here does not reject every correlation in the paper. Only the near-zero coefficients fail.

The plotted lines. Figure 3 draws the "significant" correlations as fitted lines. I measured those lines on the published image and converted the slope b to the coefficient it implies, r = bΒ·SD(x)/SD(y), using the group SDs printed in Table 1 and the Abstract (details in recompute.py.txt). The lines are much steeper than the printed coefficients allow. For T2D ICCLAMP vs HbA1c the line implies r β‰ˆ βˆ’0.27, against a printed βˆ’0.004. For T1D ICCLAMP vs HbA1c it implies β‰ˆ βˆ’0.32 (printed βˆ’0.001), for T2D ICMMT vs HCL β‰ˆ βˆ’0.18 (printed βˆ’0.037), and for T1D ICMMT vs HbA1c β‰ˆ βˆ’0.26 (printed βˆ’0.012). These estimates are approximate: besides the pixel measurement, the conversion assumes an untransformed least-squares (Pearson) line, whereas the Methods also mention log transforms and Spearman correlations, and it uses whole-group SDs as stand-ins for the SDs of the unreported analysis samples. They suggest that the plotted data carry associations of moderate size, and that the printed numbers are not the correlation coefficients of those data.

Figure

Recomputed two-sided P values for the correlations reported as P < 0.05, on a log axis.

Each row is one correlation that the paper describes as significant. The orange line and arrowhead show the paper's claim, P < 0.05. The filled blue dot is the two-sided P recomputed from the printed r at the full group size, which is the smallest P that coefficient could have. The open blue dot, for the liver-fat rows, is the same calculation at the number of participants with an HCL measurement (Fig. 2). The thin vertical line marks P = 0.05. The nine upper rows all fall to the right of it, between P = 0.10 and 0.99. The four lower rows are correlations from the same paper whose printed r does reach P < 0.05, shown for comparison.

Printed coefficients compared with the coefficients implied by the regression lines drawn in Fig. 3.

For the four lines that could be converted (an SD is reported for both variables), the orange dot is the printed r and the open blue diamond is the r implied by the slope of the line drawn in Fig. 3 (slope Γ— SDx/SDy, measured from the image, approximate). The gray band marks |r| values too small for P < 0.05: at the full group size for the HbA1c rows and at the HCL subset size for the HCL row. Every printed value lies inside the band, and every line lies outside it. The small blue squares test one possible explanation, discussed below: they show the |r| that each printed value would imply if it were a P value rather than r, at the Fig. 2 subset size.

Why it matters

Explanations considered

None of these readings accounts for every printed number. Only the authors' analysis output can settle what each number is.

Status

As of 2026-10-24.

Other inconsistencies in the same section, not needed for the main finding:

Suggested correction