performance: Use push_unchecked when building indices for element take in fixedsizedlist take #23201

Job	Run time
Python (test)	3m 27s
validate-workflow-yaml	6s
Python (lint)	1m 35s
Python (wheel build)	4m 56s
Rust (docs)	3m 17s
lint-toml	6s
Rust (lint, no default)	3m 18s
Rust (lint)	3m 18s
Rust tests (sanitizer)	3m 21s
CUDA tests	3m 15s
License Check and Audit Check (bans licenses sources)	50s
Rust tests (miri)	3m 20s
Check build with minimal dependencies	5m 0s
wasm-integration	2m 58s
Benchmark with Codspeed (Shard #7)	1m 35s
Rust tests (windows-x64)	8m 52s
Rust (semver checks)	5m 32s
Benchmark with Codspeed (Shard #4)	1m 54s
Benchmark with Codspeed (Shard #5)	3m 5s
C++ build	4m 14s
Benchmark with Codspeed (Shard #6)	2m 11s
Check generated source files are up to date	3m 43s
Rust build (wasm32 with default features)	3m 14s
Rust tests (coverage) (tests)	6m 50s
Java	3m 7s
Benchmark with Codspeed (Shard #8)	1m 47s
Rust tests (linux-arm64)	4m 5s
Rust build (all-features)	7m 0s
Rust build (default features)	7m 9s
Benchmark with Codspeed (Shard #3)	3m 40s
Benchmark with Codspeed (Shard #1)	1m 8s
License Check and Audit Check (advisories)	49s
Benchmark with Codspeed (Shard #2)	4m 29s
Rust build (with tokio dispatcher)	2m 7s
	1h 55m 18s

Provide feedback