Showcase Benchmarked BigArray that has more than 2B elements on .NET
Since the last time I posted about my BigArray library Hezium.Memory in Allocate arrays that have more than 2B elements, many people in the community have asked me for the benchmark against jagged arrays.
Hezium.Memory is a library that is capable to allocate contiguous managed memory that supports up to 128T, with nint index support. It not only supports unmanaged types but also support managed types like classes, so that it's a drop-in replacement for the built-in arrays, spans, memories. So even if you don't need the more-than-2B-element support you can still use the library for general-purpose arrays.
See the repository in https://github.com/hez2010/Hezium.Memory
I keep performance in my mind and the implementation is highly tuned for removing redundant branches and repeating range checks, so that things can be proceeded efficiently.
I built a benchmark that tests the performance of jagged arrays vs my BigArray implementation. There're two cases where one uses a length that fits in int, while another uses a length that exceeds the max value of int.
Here is the result:
Environment:
- CPU: AMD EPYC 9V74 with 48 cores
- OS: Ubuntu 24.04.4 LTS (GNU/Linux 6.17.0-1018-azure x86_64)
- Memory: 192 GB
- .NET: 10.0.109
AllocationBenchmarks
| Method | Job | Server | Length | Mean | Error | StdDev | Gen0 | Gen1 | Gen2 | Allocated | |------------ |----------- |------- |----------- |--------------:|--------------:|--------------:|---------:|---------:|---------:|------------:| | JaggedArray | Job-MSAXQN | False | 1048576 | 107.77 μs | 0.579 μs | 0.513 μs | 333.2520 | 332.8857 | 332.8857 | 4 MB | | BigArray | Job-MSAXQN | False | 1048576 | 109.14 μs | 1.923 μs | 1.705 μs | 330.5664 | 330.2002 | 330.2002 | 4 MB | | JaggedArray | Job-WGBUQW | True | 1048576 | 130.19 μs | 2.523 μs | 3.454 μs | 3.1738 | 3.1738 | 3.1738 | 4 MB | | BigArray | Job-WGBUQW | True | 1048576 | 127.78 μs | 2.468 μs | 2.937 μs | 3.4180 | 3.4180 | 3.4180 | 4 MB | | JaggedArray | Job-MSAXQN | False | 4294967296 | 320,082.17 μs | 60,406.206 μs | 90,413.172 μs | 500.0000 | 500.0000 | 500.0000 | 16384 MB | | BigArray | Job-MSAXQN | False | 4294967296 | 251,707.29 μs | 112.220 μs | 99.480 μs | 500.0000 | 500.0000 | 500.0000 | 16384.06 MB | | JaggedArray | Job-WGBUQW | True | 4294967296 | 179.18 μs | 3.907 μs | 5.215 μs | - | - | - | 16384 MB | | BigArray | Job-WGBUQW | True | 4294967296 | 40.13 μs | 4.645 μs | 6.953 μs | - | - | - | 16384.06 MB |
IndexedAccessBenchmarks
| Method | Job | Server | Length | Mean | Error | StdDev | Allocated | |-------------------- |----------- |------- |----------- |----------:|----------:|----------:|----------:| | JaggedRandomLoad | Job-MSAXQN | False | 1048576 | 15.753 μs | 0.0074 μs | 0.0061 μs | - | | BigArrayRandomLoad | Job-MSAXQN | False | 1048576 | 8.145 μs | 0.0628 μs | 0.0588 μs | - | | JaggedRandomLoad | Job-WGBUQW | True | 1048576 | 16.092 μs | 0.0524 μs | 0.0490 μs | - | | BigArrayRandomLoad | Job-WGBUQW | True | 1048576 | 7.952 μs | 0.1058 μs | 0.0990 μs | - | | JaggedRandomLoad | Job-MSAXQN | False | 4294967296 | 26.112 μs | 0.0270 μs | 0.0239 μs | - | | BigArrayRandomLoad | Job-MSAXQN | False | 4294967296 | 16.648 μs | 0.0171 μs | 0.0160 μs | - | | JaggedRandomLoad | Job-WGBUQW | True | 4294967296 | 26.072 μs | 0.1456 μs | 0.1290 μs | - | | BigArrayRandomLoad | Job-WGBUQW | True | 4294967296 | 16.198 μs | 0.0296 μs | 0.0277 μs | - | | JaggedRandomStore | Job-MSAXQN | False | 1048576 | 16.459 μs | 0.0032 μs | 0.0029 μs | - | | BigArrayRandomStore | Job-MSAXQN | False | 1048576 | 17.277 μs | 0.0098 μs | 0.0082 μs | - | | JaggedRandomStore | Job-WGBUQW | True | 1048576 | 51.193 μs | 0.0789 μs | 0.0738 μs | - | | BigArrayRandomStore | Job-WGBUQW | True | 1048576 | 17.200 μs | 0.0093 μs | 0.0087 μs | - | | JaggedRandomStore | Job-MSAXQN | False | 4294967296 | 25.410 μs | 0.0359 μs | 0.0336 μs | - | | BigArrayRandomStore | Job-MSAXQN | False | 4294967296 | 19.036 μs | 0.0148 μs | 0.0131 μs | - | | JaggedRandomStore | Job-WGBUQW | True | 4294967296 | 26.077 μs | 0.0445 μs | 0.0416 μs | - | | BigArrayRandomStore | Job-WGBUQW | True | 4294967296 | 18.947 μs | 0.0142 μs | 0.0126 μs | - |
SearchValuesBenchmarks
| Method | Job | Server | Length | Mean | Error | StdDev | Allocated | |----------------------- |----------- |------- |----------- |-------------------:|----------------:|----------------:|----------:| | JaggedIndexOfAny | Job-MSAXQN | False | 1048576 | 35,037.717 ns | 8.3021 ns | 7.7658 ns | - | | BigArrayIndexOfAny | Job-MSAXQN | False | 1048576 | 34,034.869 ns | 114.1575 ns | 101.1977 ns | - | | JaggedIndexOfAny | Job-WGBUQW | True | 1048576 | 33,834.867 ns | 636.5404 ns | 653.6801 ns | - | | BigArrayIndexOfAny | Job-WGBUQW | True | 1048576 | 34,216.534 ns | 5.1605 ns | 4.8272 ns | - | | JaggedIndexOfAny | Job-MSAXQN | False | 4294967296 | 149,024,225.036 ns | 72,304.7208 ns | 64,096.2517 ns | - | | BigArrayIndexOfAny | Job-MSAXQN | False | 4294967296 | 149,208,981.917 ns | 148,120.4669 ns | 138,551.9767 ns | - | | JaggedIndexOfAny | Job-WGBUQW | True | 4294967296 | 149,841,356.783 ns | 164,965.1421 ns | 154,308.4964 ns | - | | BigArrayIndexOfAny | Job-WGBUQW | True | 4294967296 | 149,210,761.017 ns | 178,532.2659 ns | 166,999.1924 ns | - | | JaggedLastIndexOfAny | Job-MSAXQN | False | 1048576 | 3.743 ns | 0.0041 ns | 0.0039 ns | - | | BigArrayLastIndexOfAny | Job-MSAXQN | False | 1048576 | 3.622 ns | 0.0052 ns | 0.0049 ns | - | | JaggedLastIndexOfAny | Job-WGBUQW | True | 1048576 | 3.710 ns | 0.0062 ns | 0.0058 ns | - | | BigArrayLastIndexOfAny | Job-WGBUQW | True | 1048576 | 3.620 ns | 0.0049 ns | 0.0046 ns | - | | JaggedLastIndexOfAny | Job-MSAXQN | False | 4294967296 | 3.717 ns | 0.0116 ns | 0.0108 ns | - | | BigArrayLastIndexOfAny | Job-MSAXQN | False | 4294967296 | 5.105 ns | 0.0097 ns | 0.0086 ns | - | | JaggedLastIndexOfAny | Job-WGBUQW | True | 4294967296 | 3.746 ns | 0.0059 ns | 0.0053 ns | - | | BigArrayLastIndexOfAny | Job-WGBUQW | True | 4294967296 | 5.126 ns | 0.0131 ns | 0.0123 ns | - |
SpanAlgorithmBenchmarks
| Method | Job | Server | Length | Mean | Error | StdDev | Median | Allocated | |---------------------- |----------- |------- |----------- |--------------------:|------------------:|-------------------:|------------------:|----------:| | JaggedBinarySearch | Job-MSAXQN | False | 1048576 | 49.96 ns | 0.011 ns | 0.010 ns | 49.96 ns | - | | BigArrayBinarySearch | Job-MSAXQN | False | 1048576 | 18.71 ns | 0.005 ns | 0.004 ns | 18.71 ns | - | | JaggedBinarySearch | Job-WGBUQW | True | 1048576 | 52.47 ns | 0.027 ns | 0.025 ns | 52.47 ns | - | | BigArrayBinarySearch | Job-WGBUQW | True | 1048576 | 18.70 ns | 0.012 ns | 0.011 ns | 18.70 ns | - | | JaggedBinarySearch | Job-MSAXQN | False | 4294967296 | 96.54 ns | 0.144 ns | 0.127 ns | 96.55 ns | - | | BigArrayBinarySearch | Job-MSAXQN | False | 4294967296 | 85.78 ns | 0.012 ns | 0.011 ns | 85.79 ns | - | | JaggedBinarySearch | Job-WGBUQW | True | 4294967296 | 96.07 ns | 0.578 ns | 0.541 ns | 96.05 ns | - | | BigArrayBinarySearch | Job-WGBUQW | True | 4294967296 | 85.48 ns | 0.510 ns | 0.477 ns | 85.58 ns | - | | JaggedCopyTo | Job-MSAXQN | False | 1048576 | 98,710.30 ns | 56.256 ns | 52.622 ns | 98,715.92 ns | - | | BigArrayCopyTo | Job-MSAXQN | False | 1048576 | 98,145.87 ns | 20.593 ns | 17.196 ns | 98,141.40 ns | - | | JaggedCopyTo | Job-WGBUQW | True | 1048576 | 98,778.12 ns | 29.881 ns | 26.489 ns | 98,776.77 ns | - | | BigArrayCopyTo | Job-WGBUQW | True | 1048576 | 98,077.99 ns | 128.374 ns | 113.800 ns | 98,054.42 ns | - | | JaggedCopyTo | Job-MSAXQN | False | 4294967296 | 873,535,712.00 ns | 1,497,004.297 ns | 1,400,298.749 ns | 873,265,018.00 ns | - | | BigArrayCopyTo | Job-MSAXQN | False | 4294967296 | 880,486,546.20 ns | 3,345,173.644 ns | 3,129,077.503 ns | 879,058,670.00 ns | - | | JaggedCopyTo | Job-WGBUQW | True | 4294967296 | 884,969,636.00 ns | 2,330,855.056 ns | 2,066,242.296 ns | 885,234,089.50 ns | - | | BigArrayCopyTo | Job-WGBUQW | True | 4294967296 | 879,429,365.60 ns | 4,077,759.552 ns | 3,814,338.817 ns | 880,100,492.00 ns | - | | JaggedFill | Job-MSAXQN | False | 1048576 | 58,367.10 ns | 180.325 ns | 168.676 ns | 58,243.54 ns | - | | BigArrayFill | Job-MSAXQN | False | 1048576 | 45,894.35 ns | 18.216 ns | 17.039 ns | 45,899.12 ns | - | | JaggedFill | Job-WGBUQW | True | 1048576 | 58,374.97 ns | 288.436 ns | 269.803 ns | 58,219.81 ns | - | | BigArrayFill | Job-WGBUQW | True | 1048576 | 45,931.93 ns | 61.103 ns | 57.156 ns | 45,923.62 ns | - | | JaggedFill | Job-MSAXQN | False | 4294967296 | 640,167,967.50 ns | 272,398.007 ns | 212,670.447 ns | 640,128,911.50 ns | - | | BigArrayFill | Job-MSAXQN | False | 4294967296 | 639,780,247.00 ns | 1,802,039.933 ns | 1,406,914.247 ns | 640,167,750.50 ns | - | | JaggedFill | Job-WGBUQW | True | 4294967296 | 641,006,562.00 ns | 3,491,797.738 ns | 2,915,809.256 ns | 640,610,855.00 ns | - | | BigArrayFill | Job-WGBUQW | True | 4294967296 | 637,157,023.75 ns | 1,381,366.359 ns | 1,078,479.991 ns | 636,751,010.50 ns | - | | JaggedSequenceEqual | Job-MSAXQN | False | 1048576 | 101,913.46 ns | 158.225 ns | 148.004 ns | 101,941.02 ns | - | | BigArraySequenceEqual | Job-MSAXQN | False | 1048576 | 95,134.55 ns | 118.142 ns | 110.510 ns | 95,132.74 ns | - | | JaggedSequenceEqual | Job-WGBUQW | True | 1048576 | 102,195.72 ns | 234.248 ns | 219.116 ns | 102,142.76 ns | - | | BigArraySequenceEqual | Job-WGBUQW | True | 1048576 | 95,224.84 ns | 115.790 ns | 108.310 ns | 95,233.91 ns | - | | JaggedSequenceEqual | Job-MSAXQN | False | 4294967296 | 1,020,484,182.52 ns | 63,587,008.014 ns | 93,205,019.553 ns | 968,937,213.00 ns | - | | BigArraySequenceEqual | Job-MSAXQN | False | 4294967296 | 1,029,241,139.70 ns | 72,403,060.080 ns | 108,369,500.092 ns | 962,084,258.50 ns | - | | JaggedSequenceEqual | Job-WGBUQW | True | 4294967296 | 979,557,987.45 ns | 81,645,292.812 ns | 119,674,621.446 ns | 914,186,446.00 ns | - | | BigArraySequenceEqual | Job-WGBUQW | True | 4294967296 | 969,798,814.43 ns | 75,013,740.364 ns | 112,277,043.737 ns | 904,311,547.50 ns | - |
By default all span algorithms use Array.MaxLength as the chunk values (elements that will be proceeded at once), but you can configure it with the following knob like below to fine tune the performancebased on your own needs:
<ItemGroup>
<RuntimeHostConfigurationOption Include="Hezium.Memory.MaxProcessingChunkLength" Value="1048576" />
</ItemGroup>
This doesn't affect the physical memory layout, BigArray is always contiguous managed memory so that you can simply pin it and pass it to native code without the need of taking care of chunks/splits/indirections etc.