Status registered 2026-10-31; read 2026-11-01 on all 700 fits
Series the validation series (replaces E1, E42 (E45's dd cells are re-scored in E45))
Package build emphasis 182055b (1.0.0.9000), R 4.5.3
Driver sim/V4.R
Data results/V4/jobs/, results/V4/build.txt

Question

Where an exact likelihood exists (cr, and dd at full sampling), MCEM's estimate can be scored against the exact maximum. E42 did this on 75 of E40's trees with an unconditioned old build and a dd reference later repaired (c641dca); V4 refits all 175 trees on the current build, conditioned.

Design

trees E40's 175: 116 class-tree clades (\(\rho = 1\)), 59 Condamine families at their sampling fraction
fits cr, dd; default rule and mc_error; conditioned; auto_bounds() box
exact cr conditioned (DDD at \(\rho = 1\), Bernoulli closed form below); dd at \(\rho = 1\), validity checked on the truncation
jobs 700

Pre-set rules

  1. On trees with a valid exact maximum: median \(\Delta\ell \ge -0.5\) and 90th percentile of \(|\Delta\ell| \le 2\), per model, rule and set (E45's C1).

Results

sim/V4-analysis.py (results/V4/rule.csv, paired.csv): 700 fits, 610 with a valid exact conditioned maximum (cr on every tree; dd only at complete sampling, so 14 of the 59 family trees).

The rule holds in all eight groups. \(\Delta\ell = \ell(\hat\theta_{\text{MCEM}}) - \ell(\hat\theta)\):

model stopping rule tree set trees with an exact maximum median \(\Delta\ell\) 90th percentile of \(\lvert\Delta\ell\rvert\) worst on the iteration cap
cr mc_error family trees 59 -0.04 0.13 -0.44 14%
cr mc_error class-tree clades 116 -0.10 0.29 -0.44 36%
cr default family trees 59 -0.07 0.30 -0.72 0%
cr default class-tree clades 116 -0.12 0.34 -1.17 0%
dd mc_error family trees 14 -0.01 0.41 -14.84 0%
dd mc_error class-tree clades 116 -0.01 0.56 -2.89 1%
dd default family trees 14 -0.16 0.53 -14.85 7%
dd default class-tree clades 116 -0.06 0.65 -4.09 6%

Conclusion

On real phylogenies MCEM reaches the exact conditioned maximum for cr and dd; the mc_error rule gets closer than the default. Picidae under dd is the exception: with the extinction slope on the box's edge, MCEM from the default start reaches a low-turnover mode 14.8 below the exact maximum's high-turnover one.