Status complete 2026-10-31: 1,720 fits (960 cr, 760 dd). cr read on both rules; dd rule 1 read on the repaired exact reference (756 of 760 trees rescored, the last four at 400 tips running); dd rule 2 pending
Package build emphasis e1c2e82
Driver / analysis sim/E45-trees.R, sim/E45-fit.R, sim/E45-analysis.R, sim/survival-exact.R
Data results/E45/ (generators, trees, one row per fit)

Question

Phase 1 of the estimability plan. For the two models with an exact likelihood: does Monte Carlo EM conditioned on survival reach the exact conditioned maximum, and from how many tips is each parameter of the exact estimate reliable?

Design

models cr; dd on the exponential link (\(\beta_N = \log\varepsilon / K\), \(\lambda_0 = 0.5\))
trees crown age 15, kept if both crown lineages survive, no size window, generators calibrated to the median surviving size
turnover 0.10, 0.25, 0.40
tips 25, 50, 100, 200, 400, 800 (cr); 25-400 (dd); unreachable: dd at 0.40 with 400 tips, and at 0.40, \(\rho = 0.8\), 200 tips
\(\rho\) 1; 0.8 at 100 and 200 tips
trees per cell 40
fits MCEM conditioned through the exact survival functions (surrogate proposal, 300 draws, mc_error, at most 1000 iterations, eight-hour clock); the exact conditioned MLE (bd_ML cond = 1; the replaced-rate DDD likelihood minus the exact \(\log P(S)\)); Wald intervals from the exact Hessian, profile intervals for cr

Checks before launch: bd_ML(cond = 1) equals the unconditioned exact likelihood minus \(\log P(S)\) to \(4.6 \cdot 10^{-14}\) on every cr fit; the MCEM cr likelihood equals the closed-form Bernoulli-sampling likelihood to \(10^{-9}\).

Pre-set rule

(1) MCEM agreement: median deficit \(\ge -0.5\) nats and 90th percentile of \(|\Delta\ell| \le 2\) per cell. (2) A parameter is reliable at a size when the exact estimate's median ratio to the truth lies in [0.8, 1.25], its central 80% within [0.5, 2], and its 95% interval covers the truth on at least 90% of trees. At \(\rho = 0.8\) rule (1) reads cr against the Bernoulli exact MLE (the two sampling models differ by 0.2-0.36 nats); dd's \(\rho = 0.8\) deficit is reported, not read.

Results

Rule 1, cr: met in all 24 cells (25–800 tips; turnover 0.10, 0.25, 0.40; \(\rho\) 1 and 0.8): median deficit −0.001 to −0.014 nats, 90th percentile of \(|\Delta\ell|\) at most 0.08. Fits stopped by the clock (up to 26 of 39 at 800 tips) also sit at the exact maximum.

Rule 2, cr, reliability by size (exact conditioned MLE; median ratio in [0.8, 1.25], central 80 % within [0.5, 2], interval covering the truth on ≥ 90 % of trees), read with the profile interval (percentiles interpolated, as in sim/E45-analysis.R):

turnover \(\lambda\) reliable from \(\mu\) reliable from
0.10 25 tips not up to 800 tips
0.25 25 tips not up to 800 tips
0.40 25 tips 200 tips
\(\rho\) 0.8: 0.10 / 0.25 / 0.40 100 / 100 / 100 not / not / 200

\(\mu\) fails on spread, not coverage: the exact estimate of \(\mu\) is zero on 10–50 % of trees at turnover 0.10 (0–33 % at 0.25), so the central 80 % of ratios cannot lie within [0.5, 2], while the profile interval covers the truth on 88–100 % of trees. With the Wald interval (none exists at a zero estimate) \(\mu\) is reliable only at turnover 0.40, from 400 tips with \(\rho\) 1 and from 200 with \(\rho\) 0.8.

Rule 1, dd, on the repaired exact reference. The exact dd reference used until 30 October returned \(-\infty\) through a guard wherever extinction rises with diversity and rescaled its probability vector only at branching times (results/accept-surface/dd-reference-repair.md); on the repaired one 747 of 756 rescored trees have a valid maximum. With \(\rho\) 1, where the rule gives a verdict, it holds in 11 of 14 cells: all nine of 25–100 tips (median deficit −0.001 to −0.182) and two of the five of 200–400 tips. It fails at 400 tips / turnover 0.10 (median −0.55) and 200 and 400 tips / turnover 0.25 (−0.72, −0.62), cells in which 30–39 of the 40 fits ended on the eight-hour clock, the exact survival term taking most of each fit's time. At \(\rho\) 0.8 the deficits are reported without a verdict; the thresholds are met in three of five cells.

Rule 2, dd: pending; the rescoring re-maximised but did not compute the Hessian at the new maxima.

Conclusion

Under constant rates, conditioned MCEM reaches the exact conditioned maximum at every size and turnover measured. The speciation rate is reliable from 25 tips with complete sampling (from 100, the smallest size run, at sampling 0.8); the extinction rate only at turnover 0.40, from 200 tips, and at no size up to 800 tips at turnover 0.10 or 0.25. Under diversity dependence MCEM reaches the exact maximum up to 100 tips; at 200–400 tips the failures are cells whose fits ended on the clock.