These are the pre-registered experiments behind the emphasis package: each tests the Monte Carlo EM estimator against an exact answer or a generating value, or asks what a dated tree can estimate. The validation series V1–V12 validates every method the two papers use, under one protocol: calibrated generators with no size window, the likelihood conditioned on survival, the exponential link, the package's default stopping rule, and a fit on the 1000-iteration cap read as the estimate (build 1.1.0.9000, b21083f, for the runs since 7 November). The experiments E1–E48 before it are the earlier record; each row below says what of it holds now. The conclusions that hold now are collected in What we know, what a tree of a given size estimates in Estimability, and the terms in the Glossary.

Identifiable, estimable, informative

These pages keep three properties apart, because each belongs to a different thing.

The simulation experiments report the second, the real-tree analyses (E26, E28, E40, E48) the third. Experiment pages say "identified at \(n\) tips" in the second sense.

Reading the status

status meaning
Holds the conclusion is a current result
Holds within a scope it is a current result within the scope its page states
Partly holds part of it is a current result; the page says which part
Superseded by E<n> the result will be replaced when E<n> reports; until then it is the measurement on record
Historical a method-development step, not cited as a result
Merged into E<n> its record is carried by another page's table
Running started; results so far are on its page
Planned registered or designed, not started

Conflicts between the record and the data are listed in Open issues.

At a glance

Validation series: complete 4 · complete for two rules 1 · running 4 · registered 3.

E1–E48: Holds 9 · Holds within a scope 5 · Partly holds 15 · Superseded 13 · Historical 6 · Merged 1 · Running 1.

Reading a page

Every experiment page has the same headings: Question, Design, Pre-set rule (as registered before the data existed), Results, Conclusion, Related. Variants (E8b, E10b, E12b, E19b, E24b, E34r, E36c and others) are sections of the parent page. Three markings recur:

The validation series V1–V12

Registered 2026-10-31. Each experiment validates one method of the papers; the series page gives the common protocol. Findings are as of 12 November 2026.

Experiment Question Status Finding so far
V1 On complete trees, is each model's likelihood its simulator's? Complete All ten models, 2560 complete trees: pooled over each cell's 40 trees, 138 of 150 coefficients lie within 10% of their generating values; the per-tree rule holds in 29 of 64 cells, the rest being small-sample error except \(\beta_N\) of the \(N\) models (0.85–1.21).
V2 Is each proposal's likelihood estimate unbiased, and how many effective draws does it keep? Complete Both densities are their samplers': the surrogate reproduces the exact likelihood (cr exactly; dd within 1% in six of seven cells), and the two proposals converge where both are well sampled (median difference 0.03 nats above 400 effective draws). The surrogate keeps hundreds of draws for cr, dd, nd, d; ed, ned, nedc keep a handful on either proposal, and there \(\log\hat L\) falls below the truth.
V3 Does each stopping rule stop where the estimate has settled, inside the box? Complete for two rules The default rule stops every cr fit, 11 of 20 dd and 28 of 280 lineage-level fits; the capped fits' speciation coefficients are within 2% of the box span of the reference for 7 of 10 models at turnover 0.10 and 1 of 8 read at 0.40; extinction coefficients drift. A fit on the cap is read as the estimate.
V4 On real and simulated trees, does MCEM reach the exact conditioned maximum? Complete On the 175 real trees the rule holds for cr and dd under both stopping rules (median shortfall -0.01 to -0.16); mc_error lands closer on 75% of trees; Picidae under dd stops 14.8 below. On simulated trees (E45, repaired dd reference, complete sampling) the rule holds in 11 of 14 dd cells.
V5 From how many tips is each lineage-level parameter reliable? Running 423 of 1,710 fits (12 November), no cell read: over 414 finished fits the covariate slope is within a factor of 2 on 74%, the right sign on 92%; the extinction intercept within a factor of 2 on 11%. First pass: E46.
V6 What do incomplete sampling and a misstated sampling fraction cost? Running Rerun under b21083f and projected submitted 8 November (3,200 jobs, behind V7); first pass archived.
V7 Does model choice name the generating model, and how often a richer one? Complete 22 of 32 cells read at 100 tips and ±0.35. No lineage-level model on any of 54 null trees (false choice holds in the cell decided). Power fails in 18 of 19 cells: the ladder names the generating covariate on at most 12 of 20 trees (nk) and 10 of 19 (nd).
V8 Does the covariate path name the generating covariates at a fitted estimate? Registered Runs on V7's trees once their dd fits are written.
V9 On trees with no lineage-level effect, how often is one chosen? Running Calibrated after the amendment of 1 November: 15 of 16 cells within 70-140 tips (the latent rate at \(\sigma = 0.40\), turnover 0.40, reaches 35 and enters no rule); rerun under b21083f and projected submitted 8 November (2,400 jobs, behind V7); first pass archived.
V10 Where does a tree identify extinction? Registered A reading of E45, E46 and E38.
V11 The analysis of record on real phylogenies Running Submitted 11 November (875 fits: 175 trees × the ladder, b21083f, projected), ahead of the readings of V5–V10.
V12 What a fit costs Registered A reading of the fits' recorded times.

1. The estimator against the exact answer, and the proposals

Experiment Question Status Finding
E1 Does MCEM reach the exact MLE under cr and dd? Holds within a scope (unconditioned, linear link, 20–100 tips; replaced by V4) MCEM at the exact maximum (median deficit ≥ -0.5 nats) on 27 of 30 cell × proposal rows; surrogate shortfall a median 0.00-0.32 nats.
S The same comparison from 25 to 800 tips Superseded by E45 and V4 Under cr MCEM lies within a tenth of the exact MLE's spread, 1-6% high; cells from 100 tips clock-limited.
E3 ED-aware or mean-field thinning for ned? Holds within a scope (where thinning is used) ED-aware is ahead on 77% of trees, short of the 95% required; it completes on 23 trees where mean-field fails.
E4 What does one E-step cost? Partly holds 0.06-4 s from 25 to 400 tips; the surrogate costs 4.7-7.0× thinning under cr, less per effective draw from 100 tips.
E6 Which proposal keeps more draws as sampling falls? Partly holds Linear link, older build: the ranking holds in 17 of its 20 cells on E6b; where they differ, E6b's reading replaces it.
E6b Does E6's proposal ranking hold on the current build and the exponential link? Holds Same winner in 17 of 20 cells; ed and ned cross between \(\rho\) 0.4 and 0.2; ED models below 10 effective draws on both proposals; the surrogate leads dd 11-68×, d 22-80×, nd 18-40×.
E7 Where is the surrogate proposal usable? Partly holds Surrogate ESS below 10 in 7 of 67 cells, all with strong negative ED effects; thinning below 10 in 54.
E9 Does the mc_error rule stop once the estimate has settled? Holds within a scope It never stopped while the trace moved (0 of 17) and lands about 2% from a 400-iteration reference.
E10 Which relaxation step converges the surrogate's mean field? Holds At 1000 sweeps a fixed ⅛ converges 107 of 139 diverging units, "auto" 77.
E10c Does a relaxation step of ⅛ win at the shipped sweep budget? Holds Its rule is met, but at \(\rho\) = 1 ⅛ converges 0 of 192 units in 20 sweeps against 131 for "auto"; the default is "auto" at \(\rho\) = 1 and ⅛ below.
E13 How noisy is one iterate, and when does a fit settle? Partly holds (the noise floor) Noise falls about as 1/√draws (1-7%); the settling iteration depends on the criterion and does not scale.
E15 Where does the time of a ned iteration go? Historical One iteration from the box takes 77 s, 78 min and 5.1 h at 100, 200, 400 tips; its parts under 2 min.
E42 MCEM against the exact MLE on real trees Partly holds (cr; replaced by V4) On 75 trees the median deficit is -0.079 nats under cr and -0.005 under dd on complete trees.

2. Recovery and selection of the ED and single-covariate models

Experiment Question Status Finding
E2 ned recovery and selection over size, \(\rho\) and turnover Historical \(\beta_{ED}\) recovered and detected in 1 of 16 effect cells; turnover 0.5 biases \(\beta_{ED}\) by 1-4 SD.
E14 Do emphasis and evesim simulate the same trees? Holds No KS rejection in 30 comparisons; completion rates match once max_tries = 0.
E11 Does the estimator invent an ED effect on evesim null trees? Partly holds (selection; fits unfinished) ned is selected on 0.05 and 0.00 of null trees.
E12 Where does the ED term become selectable? Superseded by V5 and V7 The 0.80 selection contour lies between -0.35\(\lambda\) and -0.60\(\lambda\) at 100-400 tips; false positives 3.3%.
E16 The same at turnover 0.25 Superseded by V5 and V7 Selection 0.67 at -0.35\(\lambda\) and 1.00 at -0.60\(\lambda\); no false positive.
E18 Does centring ED lift selection? Superseded by V5 and V7 No: 0.10-0.20 at -0.15\(\lambda\) against 0.5 required; nedc behaves as ned.
E24 Recovery and selection for ed, edc, d and k Partly holds edc meets every half (ratio 0.96, 1.02; selection 0.95, 0.85); d has 0.15 false positives.

3. The pendant-age model and the ancestral-splits covariate

Experiment Question Status Finding
E19 Is \(\beta_D\) recovered, and nd selected over dd? Superseded by V5 Sign recovered, size not: ratio 0.30-0.47 on thinning and 0.43-0.63 on the surrogate at ±0.35.
E30 E19b's design with 300 iterations and a 4-hour clock Superseded by V5 Ratio 0.67 at -0.35; 0 of 39 fits met the stopping rule.
E37 Does the pendant-age fit finish within 1000 iterations? Historical Stopped when the D change altered the likelihood it maximised; 6 of 40 fits written.
E31 Can the ladder tell nd from ned at 200 tips? Superseded by V5 and V7 No: nd chosen on 5 of 20 of its own trees, the pair on 9; \(\beta_D\) at 0.68.
E21 Is \(\beta_K\) recovered, by fitting and on the path? Superseded by V5 and V7 \(\beta_K\) at 0.95 and 0.92 at ∓0.35; selection 0.15 and 0.60; on the path ED carries K's signal.

4. Model choice: the ladder and the covariate path

Experiment Question Status Finding
E8 Can Monte Carlo bias reorder an AIC ranking? Partly holds Correcting it moves rankings by 0.01-0.25 AIC and reverses 1 of 101 wrong choices, a 0.003-AIC tie.
E20 Does one dgLARS path select the generating set? Superseded by V7 and V8 Met for cr only (0.90) at 100 tips; D and ED are not separable.
E22 The path at 50, 200 and 400 tips Superseded by V7 and V8 Null chosen on 0.85-1.00 of trees; dd's N alone on 0.40-0.70; ned's N + ED never on 0.80.
E23 What does the one-call ladder choose? Superseded by V7 and V8 The generator is chosen on 0.80 of cr and 0.90 of dd trees; nk 0.70, ned 0.20, nd 0.00.
E27 The ladder across eight conditions Superseded by V5 and V7 nk chosen on 11-15 of 20 of its trees, ned on at most 2; turnover 0.25-0.40 moves null trees to cr.
E33 The path at a fitted estimate; the extinction path Superseded by V7 and V8 At a fitted estimate dd's exact set falls to 0.35 and 0.25; an extinction driver is not detected.

5. Extinction, conditioning and what a tree carries

Experiment Question Status Finding
E29 On finished fits at 200 tips, is extinction recovered? Partly holds No: the fitted rate is a median 26× below 0.05; \(\beta_{ED}\) returns at 1.02 on the 7 finished fits.
E32 Does a wider box or an exact estimator recover extinction? Holds No: estimates follow the box for a 0.01 log-likelihood gain; bd_ML puts \(\mu\) at zero on 37 of 40 trees.
E34 Is zero extinction an artefact of not conditioning? Holds No: conditioned extinction is within 2× of 0.05 on 1 of 20 trees (E34r: 0 of 20).
E36 How accurate is the survival emulator? Partly holds It labelled cap-passing clades as extinct; with that fixed (d3dee92) it meets its rule at 2000 training clades.
E35 A K driver at turnover 0.25; a misstated \(\rho\) Holds within a scope The negative K slope returns at 0.75 of its value; \(\rho\) off by 0.1 leaves the choice unchanged on ≥ 18 of 20.
E38 How much does a tree say about extinction, by turnover and size? Holds within a scope At turnover 0.10 the exact cr likelihood excludes zero on fewer than 80% of trees up to 1600 tips; dd from 100.

6. Real trees

The results on the 170 bird and mammal trees of E40, with figures, are on Real phylogenies.

Experiment Question Status Finding
E5 Does the workflow run on 47 family trees? Historical cr within 0.53 nats of bd_ML on all 46 fitted trees (linear link).
E17 Are the families' margins what the fitted model predicts? Historical All three real ned-over-dd margins lie inside the central 90% of their bootstrap.
E25 Three bird families under the ladder and the path Historical nk on Corvidae (12.0 AIC over ned), ned on Icteridae, dd on Cotingidae.
E26 The family trees under the whole workflow Merged into E28 36 of 46 families usable; by minimum AIC 21 choose a lineage-level model (nk 12, ned 9).
E28 The 29 families in the calibrated range, refitted Partly holds 25 usable; parsimonious choice cr 4, dd 5, nd 0, ned 8, nk 8.
E39 What do the covariates share within a tree? Holds ED-K rank correlation median -0.44 on family trees against -0.18 on simulated ones, stronger with imbalance.
E40 Birds and mammals cut into clade-sized trees Partly holds Bird clades choose ned or nk on 72 of 77; clade and family tree agree on 10 of 29 pairs (rule: 70%).
E41 Do E40's fits change when continued, with edges moved out? Partly holds Choice unchanged on 16 of 16 trees; the edge binds on 4 of 63 fits. The nd and nk fits kept their starting draws (the stopping rule could not grow them on that build), so their convergence is measured by V3.
E48 Does refitting nd under the corrected D likelihood change the real-tree readings? Holds Choice changes on 5 of 155 readable E40 trees, 1 of 25 E28 and 1 of 9 E26 (only that set crosses 10%); the pairs rule still fails; \(\beta_D < 0\) on 28 of 30 read family fits.

7. Estimability under the conditioned likelihood

How well each model's parameters are estimated, and from how many tips, with the likelihood conditioned on the survival of the clade: what these experiments and the validation series have measured so far is read together on Estimability.

Experiment Question Status Finding
E44 On complete trees, is the likelihood the simulator's? Partly holds (the \(D\) fix; replaced by V1) Yes, with D constant between events: d's \(\beta_D\) returns at 0.95 and 1.16 of its value, against 0.70 and 1.46 before.
E43 The survival factor of each model, exact or emulated Holds No emulator meets the rule for any lineage-level model (error 0.5-2.6 against 0.05), so they are conditioned by a direct Monte Carlo estimate of \(P(S)\); the reference matches the exact cr/dd values (median error 0.007, 0.008).
E45 Conditioned cr and dd against the exact estimate Partly holds (cr; dd on the repaired reference) 1720 of 1720 fits: MCEM at the exact maximum in all 24 cr cells, \(\lambda\) reliable from 25 tips at turnover 0.10; on the repaired dd reference at complete sampling the rule holds in 11 of 14 dd cells (all nine of 25–100 tips; misses at 400 tips and turnover 0.10, and 200 and 400 tips at 0.25, where most fits ended on the clock); 747 of 756 rescored trees have a valid maximum. Reliable sizes for dd: pending.
E46 The eight lineage-level models, conditioned, by size, turnover and effect Running (the series' V5) 57 cells at 50 and 100 tips, larger only where a parameter is not reliable; 19 of 57 cell–proposal pairs are sampler-limited; the lineage-level fits rerun under b21083f after V3.

Chronology