Paperonium

← Back to the plain-language case

NRSF gain stated as 299% more binding sites; Table 1 counts give 199% more

Nucleic Acids Res (2008) Β· doi:10.1093/nar/gkn488 Β· PMC2532738

Summary

The paper says that its method, SISSRs, found "32, 299 and 78% more" binding sites than previous analyses of the same CTCF, NRSF and STAT1 datasets. It makes this statement in the abstract, the introduction and the Results, and the Discussion repeats it as the range "32–299%". Table 1 gives the counts behind the NRSF figure: 5813 sites from SISSRs and 1946 from the previous approach. That is (5813 βˆ’ 1946)/1946 = 198.7% more, or 2.99 times as many. "299" is the new count expressed as a percentage of the old one (298.7%), while the CTCF and STAT1 figures in the same sentence are correct "% more" values (32.3% and 77.9%). The NRSF gain is therefore overstated by 100 percentage points in the paper's headline sensitivity claim. SISSRs still finds about three times as many NRSF sites, not about four times as many.

What the paper reports

Abstract:

We identified 26 814, 5813 and 73 956 binding sites for CTCF, NRSF and STAT1 proteins, respectively, which is 32, 299 and 78% more than that inferred previously for the respective proteins.

Introduction, second paragraph:

Using SISSRs, we identified a total of 26 814, 5813 and 73 956 binding sites for CTCF, NRSF and STAT1, respectively, which is 32, 299 and 78% more than that inferred previously for the respective proteins (3–5).

Results, section "ChIP-Seq datasets and binding sites":

Using an FDR threshold of 10-3, we identified a total of 26 814, 5813 and 73 956 binding sites for CTCF, NRSF and STAT1 proteins, respectively. The number of sites identified in this study is 32, 299 and 78% more than what was previously identified from the same datasets for the respective proteins (3–5) (Table 1).

Discussion, first paragraph:

SISSRs’ ability to identify binding sites with high resolution helped it achieve unprecedented sensitivity and specificity, as evidenced by its ability to identify 32–299% more binding sites than that by previous approaches using the same dataset (3–5).

Table 1 ("Data and results summary"), two of its rows:

CTCF NRSF STAT1
Binding sites identified by SISSRs 26 814 5813 73 956
Binding sites identified by previous approaches 20 262 1946 41 582

What the paper's own numbers show

Method. For each protein I took the two counts in Table 1 and computed the increase as (SISSRs βˆ’ previous)/previous Γ— 100, which is what "X% more than" means. I also computed the ratio SISSRs/previous Γ— 100, the new count as a percentage of the old one, to see which convention each reported figure matches. As a check on the table, I compared the SISSRs counts with the subset sizes the paper gives elsewhere, and the previous NRSF count with the source the paper cites for it. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's own numbers
CTCF, % more sites (26 814 vs 20 262) Abstract; Introduction; Results 32% 32.3% (matches)
NRSF, % more sites (5813 vs 1946) Abstract; Introduction; Results 299% 198.7% (rounds to 199%)
STAT1, % more sites (73 956 vs 41 582) Abstract; Introduction; Results 78% 77.9% (matches)
NRSF, SISSRs count as % of previous count β€” β€” 298.7% (ratio 2.987)
CTCF and STAT1, same ratio as % β€” β€” 132.3% and 177.9% (do not match 32 and 78)
Range of improvement Discussion, paragraph 1 32–299% 32–199%
Fold change implied by "299% more" β€” 3.99 Table 1 gives 2.987
Previous NRSF count that would make "299% more" correct β€” 1946 (Table 1) 1456.9 needed
SISSRs NRSF count that would make "299% more" correct β€” 5813 (Table 1) 7765 needed
Top 20% of NRSF sites Methods; Results, "Accuracy" 1160 20% of 5813 = 1163 (consistent)
10% of NRSF and STAT1 sites Methods, "Motif analysis" 581 and 7396 581.3 and 7395.6 (consistent)

The CTCF and STAT1 figures fit "% more" and not "% of". The NRSF figure fits "% of" and not "% more". The subset sizes in Methods and Results are consistent with the SISSRs counts in Table 1, so the NRSF count of 5813 is used consistently across the paper. (The CTCF and STAT1 subsets of 2622 and 3825 are defined by a tag-count cut-off and are approximately, not exactly, 10% and 5% of the sites; they do not bear on the NRSF figure.)

Figure

The text reports 32, 299 and 78% more binding sites; recomputed from Table 1 the values are 32.3, 198.7 and 77.9%

Figure 1. Percentage increase in binding sites found by SISSRs over previous approaches, for each of the three proteins. Each row is labelled with the two Table 1 counts it is computed from (SISSRs vs previous approaches). Open orange circles are the values the paper reports in the abstract, introduction and Results; filled blue dots are the values recomputed from Table 1 as (SISSRs βˆ’ previous)/previous, by recompute.py.txt. For CTCF and STAT1 the two coincide to within 0.4 percentage points. For NRSF the reported 299% lies 100 percentage points above the recomputed 198.7%; 299 matches 5813/1946 Γ— 100 = 298.7%, the new count as a percentage of the old one. The axis starts at zero.

Why it matters

Explanations considered

Status

Suggested correction

In the abstract, the introduction (second paragraph) and the Results ("ChIP-Seq datasets and binding sites"), replace

32, 299 and 78% more

with

32, 199 and 78% more

and in the Discussion (first paragraph), replace

32–299% more binding sites

with

32–199% more binding sites

Alternatively, the NRSF result could be stated as "about three times as many binding sites as previously identified (5813 vs 1946)". Table 1 needs no change.