Create a blog/post about performance gains

[jdkent]

The software has gotten faster over 4 years of development. That should be marked/celebrated.

Version	Dataset	Benchmark	Estimator	Time (s)	Median RSS (MiB)	Peak RSS (MiB)
0.0.11	20	fit	ALE	1.32	305.39	322.64
0.0.11	20	fit	MKDA	0.32	277.17	300.17
0.0.11	20	fit	MKDAChi2	4.23	268.99	307.15
0.0.11	200	fit	ALE	53.15	1136.92	1363.95
0.0.11	200	fit	MKDA	2.46	702.50	932.50
0.0.11	200	fit	MKDAChi2	3.88	271.71	1115.47
0.0.11	1000	fit	ALE	326.28	4602.50	5936.53
0.0.11	1000	fit	MKDA	12.23	2640.20	3735.64
0.0.11	1000	fit	MKDAChi2	12.05	2158.61	4732.24
0.0.11	20	fwe(10)	ALE	4.18	343.52	2611.65
0.0.11	20	fwe(10)	MKDA	2.64	345.64	2455.52
0.0.11	20	fwe(10)	MKDAChi2	20.40	317.39	2753.45
0.0.11	200	fwe(10)	ALE	53.79	1171.33	8354.70
0.0.11	200	fwe(10)	MKDA	8.06	949.38	6177.61
0.0.11	200	fwe(10)	MKDAChi2	25.44	464.39	4992.94
0.12.0	20	fit	ALE	0.45	367.17	382.56
0.12.0	20	fit	MKDA	0.34	408.58	410.43
0.12.0	20	fit	MKDAChi2	0.33	387.69	410.23
0.12.0	200	fit	ALE	5.09	564.30	667.23
0.12.0	200	fit	MKDA	0.43	377.01	386.42
0.12.0	200	fit	MKDAChi2	0.51	376.39	397.34
0.12.0	1000	fit	ALE	32.38	1066.11	1915.28
0.12.0	1000	fit	MKDA	1.15	417.02	513.14
0.12.0	1000	fit	MKDAChi2	0.74	387.84	413.14
0.12.0	20	fwe(10)	ALE	3.75	470.37	567.43
0.12.0	20	fwe(10)	MKDA	2.38	3239.07	3285.63
0.12.0	20	fwe(10)	MKDAChi2	4.65	3327.39	3457.18
0.12.0	200	fwe(10)	ALE	10.67	1064.38	2141.51
0.12.0	200	fwe(10)	MKDA	3.18	3414.02	3447.71
0.12.0	200	fwe(10)	MKDAChi2	5.48	3356.73	3539.95

[jdkent]

more metrics

Warm Start Summary Across Environments

Runs used:

• nimare_0_0_11_py310: full_matrix_20260408T060754Z
• nimare_0_15_0_py310: py310_matrix_20260408T064801Z
• nimare_0_15_0_py314: merged current state of py314_matrix_20260408T071045Z

Notes:

• Warm-start results only.
• Cold-start results are excluded.
• Time values are wall_time_sec mean +- standard deviation across warm repetitions.
• Memory values are peak RSS in GiB, mean +- standard deviation across warm repetitions.
• 0.15.0 speedups are relative to 0.0.11.
• SCALE at studies_1000 in 0.0.11 is reported as 132.0 +- 0.0 GiB by request.

Warm Time

Study Size	Case	0.0.11 (wall s)	0.15.0 py310 (wall s)	Speedup	0.15.0 py314 (wall s)	Speedup
20	ALE	1.591 +- 0.596	0.383 +- 0.050	4.16x	0.123 +- 0.001	12.93x
	ALE+FWE	30.288 +- 8.244	19.574 +- 1.873	1.55x	4.792 +- 0.010	6.32x
	MKDADensity	0.385 +- 0.095	0.438 +- 0.006	0.88x	0.104 +- 0.001	3.69x
	MKDAChi2	1.471 +- 0.054	0.449 +- 0.015	3.28x	0.161 +- 0.001	9.16x
	ALESubtraction	13.701 +- 0.874	1.230 +- 0.049	11.14x	0.818 +- 0.003	16.75x
	SCALE	29.707 +- 5.486	1.111 +- 0.050	26.74x	0.730 +- 0.004	40.67x
	CorrelationDecoder Fit	8.727 +- 1.186	0.627 +- 0.035	13.92x	0.354 +- 0.013	24.67x
	CorrelationDecoder Transform	9.004 +- 1.147	0.936 +- 0.103	9.62x	0.392 +- 0.005	22.97x
200	ALE	12.590 +- 0.401	1.023 +- 0.057	12.31x	0.704 +- 0.004	17.88x
	ALE+FWE	90.537 +- 3.389	27.735 +- 1.038	3.26x	13.068 +- 0.009	6.93x
	MKDADensity	3.925 +- 0.706	0.493 +- 0.008	7.97x	0.167 +- 0.001	23.48x
	MKDAChi2	3.362 +- 0.358	0.515 +- 0.033	6.53x	0.195 +- 0.001	17.24x
	ALESubtraction	23.495 +- 3.514	5.227 +- 0.633	4.49x	3.714 +- 0.083	6.33x
	SCALE	79.400 +- 0.740	5.799 +- 0.107	13.69x	4.891 +- 0.030	16.23x
	CorrelationDecoder Fit	7.461 +- 0.145	0.921 +- 0.068	8.10x	0.649 +- 0.024	11.50x
	CorrelationDecoder Transform	10.097 +- 3.264	1.124 +- 0.067	8.98x	0.763 +- 0.048	13.24x
1000	ALE	94.963 +- 11.774	4.835 +- 0.166	19.64x	4.623 +- 0.151	20.54x
	ALE+FWE	397.173 +- 16.231	67.446 +- 11.613	5.89x	74.097 +- 11.422	5.36x
	MKDADensity	9.684 +- 0.317	0.889 +- 0.017	10.89x	0.618 +- 0.033	15.67x
	MKDAChi2	11.456 +- 1.672	0.791 +- 0.057	14.48x	0.460 +- 0.019	24.93x
	ALESubtraction	97.864 +- 18.343	24.942 +- 1.920	3.92x	38.537 +- 3.878	2.54x
	SCALE	n/a	25.759 +- 2.912	n/a	36.218 +- 5.812	n/a
	CorrelationDecoder Fit	30.520 +- 1.464	1.714 +- 0.058	17.80x	1.743 +- 0.032	17.51x
	CorrelationDecoder Transform	28.708 +- 3.904	1.873 +- 0.167	15.33x	1.558 +- 0.110	18.42x

Warm Peak Memory

Study Size	Case	0.0.11 (GiB)	0.15.0 py310 (GiB)	RSS Ratio	0.15.0 py314 (GiB)	RSS Ratio
20	ALE	0.345 +- 0.000	0.377 +- 0.002	0.92x	0.415 +- 0.001	0.83x
	ALE+FWE	0.389 +- 0.001	0.446 +- 0.000	0.87x	0.483 +- 0.001	0.81x
	MKDADensity	0.367 +- 0.005	0.379 +- 0.001	0.97x	0.411 +- 0.000	0.89x
	MKDAChi2	0.327 +- 0.000	0.408 +- 0.000	0.80x	0.443 +- 0.000	0.74x
	ALESubtraction	0.664 +- 0.000	0.387 +- 0.002	1.72x	0.420 +- 0.000	1.58x
	SCALE	8.377 +- 0.000	0.385 +- 0.000	21.73x	0.419 +- 0.001	19.98x
	CorrelationDecoder Fit	0.703 +- 0.002	0.443 +- 0.002	1.59x	0.477 +- 0.002	1.48x
	CorrelationDecoder Transform	0.706 +- 0.001	0.479 +- 0.001	1.47x	0.512 +- 0.004	1.38x
200	ALE	1.155 +- 0.000	0.472 +- 0.011	2.45x	0.496 +- 0.017	2.33x
	ALE+FWE	1.197 +- 0.001	0.534 +- 0.002	2.24x	0.567 +- 0.000	2.11x
	MKDADensity	1.611 +- 0.002	0.385 +- 0.002	4.18x	0.421 +- 0.001	3.83x
	MKDAChi2	1.119 +- 0.023	0.410 +- 0.000	2.73x	0.444 +- 0.000	2.52x
	ALESubtraction	1.443 +- 0.003	0.440 +- 0.005	3.28x	0.472 +- 0.004	3.06x
	SCALE	42.381 +- 0.000	0.497 +- 0.001	85.35x	0.531 +- 0.001	79.85x
	CorrelationDecoder Fit	1.530 +- 0.008	0.448 +- 0.002	3.41x	0.484 +- 0.004	3.16x
	CorrelationDecoder Transform	1.555 +- 0.008	0.498 +- 0.001	3.12x	0.531 +- 0.001	2.93x
1000	ALE	4.743 +- 0.008	0.951 +- 0.000	4.99x	0.987 +- 0.000	4.81x
	ALE+FWE	4.774 +- 0.001	0.990 +- 0.005	4.82x	1.026 +- 0.005	4.65x
	MKDADensity	7.321 +- 0.005	0.423 +- 0.000	17.30x	0.460 +- 0.000	15.92x
	MKDAChi2	4.648 +- 0.024	0.417 +- 0.000	11.14x	0.454 +- 0.000	10.24x
	ALESubtraction	5.537 +- 0.004	0.741 +- 0.003	7.48x	0.800 +- 0.004	6.92x
	SCALE	132.0 +- 0.0	0.951 +- 0.000	138.84x	0.988 +- 0.001	133.65x
	CorrelationDecoder Fit	6.079 +- 0.005	0.501 +- 0.002	12.14x	0.540 +- 0.003	11.26x
	CorrelationDecoder Transform	6.072 +- 0.016	0.588 +- 0.001	10.33x	0.623 +- 0.003	9.75x

[lyraluoyu]

Hi @jdkent,

Apologies for the late start on this. I've summarized the performance gains between v0.0.11 and v0.12.0 into the plot below. Does this meet the requirements for being an intuitive visualization of the performance leap? Please let me know if you have any suggestions for improvement. I'll then move on to the "Warm Start Summary Across Environments" data.

Regarding the documentation/blog post, do you have a preferred location in the docs/ folder for this content, or should I share it in another way? Thanks for your guidance! :)

[lyraluoyu]

I've drawn another two plots for warm time metrics:

[jdkent]

Thank you for your work on this, let's touch base on what the goals of the blog would be before you put any more work!

[lyraluoyu]

Thank you for your work on this, let's touch base on what the goals of the blog would be before you put any more work!

Okay, I'll hold off for now and look forward to your thoughts on the goals. :)