Status complete (E12: 300 trees, 297 fitted; E12b: 30 trees); superseded by E46 (recovery) and E47 (selection, on an effect grid, since E46 runs only ±0.35) when they report (Estimability)
Package build library e8, installed at 4f42b27 on 2026-09-20; the build audit found that library identical to a clean build except ned on the thinning proposal
Driver / analysis sim/E12-power.R; figures fig-power and fig-s-ned-recovery
Data results/E12/power.csv (300 rows), results/E12b/power.csv (30 rows)

Question

E8 found \(\beta_{ED}\) estimable and AIC unwilling to select ned over dd, at one effect size (-0.15\(\lambda\)) and one tree size. That was being stated as a boundary, but it is one point. E12 sweeps effect and size to find where selection starts to work, which turns a negative result into a statement about where the method has power.

Design

parameter value meaning
generator ned, linear link \(\lambda_s = \beta_0 + \beta_N N + \beta_{ED}\,\mathrm{ED}_s\)
ED effect 0, -0.05, -0.15, -0.35, -0.60 \(\lambda\) \(\beta_{ED}\) as a fraction of \(\beta_0 = 0.5\) at one clade-mean unit of ED
tips 100, 200, 400 \(K\) calibrated per cell to the target; crown 15
turnover 0.10 fixed by E7: at 0.25 the strong cells fall below ESS 10 by 50 tips
trees per cell 20 15 cells, 300 trees
candidates dd and ned only the question is whether the ED term pays for its two parameters
draws 800 a 100-draw pilot left ESS 1.2-6.6 in the strong cells
iterations max_iter 100
clock 3600 s as launched (the driver default; E12b 14400 s), not recorded in the CSV
proposal, box surrogate; sim/box.R

Generating \(\beta_{ED}\) per cell (raw units): -0.06/-0.03/-0.02/-0.005 at 100 tips; -0.07/-0.04/-0.02/-0.01 at 200; -0.09/-0.06/-0.02/-0.01 at 400 (-0.60 to -0.05\(\lambda\)). At 400 tips and -0.60\(\lambda\) the capacity sits at the search ceiling (≈ 10000) and 3 of 20 trees could not be drawn ("no ned tree in [240, 640] tips in 400 attempts"); the 17 drawn have a median of 267 tips.

Pre-set rule

Per cell, the fraction of trees with \(\mathrm{AIC}_{dd} - \mathrm{AIC}_{ned} > 2\). The deliverable is the contour where that fraction reaches 0.8 (the selection threshold of E2's rule) and the false-positive rate at effect 0, which must be \(\le 5\%\) for the curve to mean anything. Any cell whose median effective sample is below 10 is reported with its selection rate but excluded from the contour, marked sampler-limited. A cell that produces no trees is reported as a cell with no answer, not dropped.

Results

Selection (share of trees with \(\Delta\)AIC > 2):

tips 0 (FP) -0.05\(\lambda\) -0.15\(\lambda\) -0.35\(\lambda\) -0.60\(\lambda\)
100 0.10 0.05 0.55 0.75 1.00
200 0.00 0.00 0.05 0.70 1.00
400 0.00 0.00 0.15 0.75 1.00 (17 trees, sampler-limited)

False positives over the 60 null trees: 2 of 60 = 3.3%, within the 5% ceiling (2 of 20 at 100 tips).

Recovery of \(\beta_{ED}\) (median fitted/generating; right-sign share; median ESS of 800; fits with ESS < 10; stops converged/cap/clock):

tips -0.60\(\lambda\) -0.35\(\lambda\) -0.15\(\lambda\) -0.05\(\lambda\) 0 (median \(\hat\beta\))
100 0.69; 1.00; 24.9; 4; 0/2/18 0.865; 1.00; 61.5; 3; 0/5/15 1.25; 0.95; 83.8; 0; 1/5/14 0.34; 0.60; 131.9; 0; 0/8/12 -0.0013; 133.9; 1; 1/12/7
200 0.40; 1.00; 20.9; 5; 0/0/20 0.47; 1.00; 31.8; 5; 0/0/20 0.92; 0.95; 24.8; 6; 0/0/20 1.14; 0.80; 18.9; 7; 0/0/20 -0.0005; 20.0; 6; 0/0/20
400 0.19; 1.00; 4.6; 14 of 17; 0/0/17 (sampler-limited) 0.43; 1.00; 27.8; 3; 0/0/20 0.94; 0.95; 51.2; 5; 0/0/20 1.87; 0.90; 24.7; 5; 0/0/20 -0.012; 32.8; 4; 0/0/20

Every ned fit at 200 and 400 tips ended on its clock (120 of 120 at an effect, and every null fit); at 100 tips 35% to 90% did. Clock-limited: by the convention adopted later (clock ending more than a tenth of the fits), every cell of E12 is clock-limited, so every rate here is a floor. The supplement rounds 0.865 to 0.87; the figure prints 0.86.

E12 power
Figure 1. Selection rate of ned over dd by effect size, one line per tree size. All three sizes cross 0.80 between -0.35\(\lambda\) (0.70-0.75) and -0.60\(\lambda\) (1.00); the open point at 400 tips, -0.60\(\lambda\), is sampler-limited and excluded from the contour.
E12 recovery
Figure 2. Fitted against generating \(\beta_{ED}\), one fit per tree, with E16's turnover-0.25 cell on the right. At 100 tips the cell medians (red diamonds) sit near the dashed identity line; at 200 and 400 tips the strong effects are attenuated toward zero while keeping their sign.

E12b — the -0.35\(\lambda\) row with 300 iterations

results/E12b/power.csv: E12's design at -0.35\(\lambda\) only, 10 trees a size, 600 draws, max_iter 300, traces saved (no design entry of its own).

tips selection ratio sign median ESS of 600 fits < 10 stops conv/cap/clock
100 0.80 0.96 1.00 26.4 1 0/5/5
200 0.90 (sampler-limited) 0.67 1.00 6.1 6 0/0/10
400 0.80 0.72 1.00 10.7 5 0/0/10

No E12b fit met the stopping rule. The supplement's "half the fits finished" at 100 tips means 5 of 10 reached the 300-iteration cap. A sentence in methods-study.tex and paper.tex ("where the fit is allowed to finish (a median 262 iterations at -0.35\(\lambda\)) it returns at 0.97 of its value") has no source among E12b's files: power.csv has no iteration column, and its 100-tip ratio is 0.96 with no fit converged.

Conclusion

By E12's rule the 0.80 contour lies between -0.35\(\lambda\) and -0.60\(\lambda\) at every size (0.70-0.75 at -0.35\(\lambda\), 1.00 at -0.60\(\lambda\)), with false positives at 3.3%; the 400-tip -0.60\(\lambda\) cell is excluded. Every rate is a floor: no fit at 200 or 400 tips finished, and an unfinished ned carries a worse AIC than it would reach. With 300 iterations (E12b) -0.35\(\lambda\) reaches 0.80 at 100 and 400 tips (the 200-tip cell, 0.90, is sampler-limited). The coefficient keeps its sign (≥ 0.95 at |effect| ≥ 0.15 at every size) while its size is attenuated on unfinished fits — 0.69, 0.40 and 0.19 of the value at -0.60\(\lambda\) from 100 to 400 tips — and comes back toward its value with more iterations (0.96, 0.67, 0.72 at -0.35\(\lambda\) in E12b against 0.865, 0.47, 0.43). E12 does not measure turnover above 0.10 (E16 does 0.25), sampling below 1, or a fit that converged.

Its trees come from linear-link ned generators and the raw-ED simulator before f30a791, whose effect on these generators is not recorded, and its fits are unconditioned and unfinished; E46 measures recovery under the conditioned likelihood and E47 selection on a grid of effects (Estimability).