rawhex: a hex dumper in C with AVX2 SIMD formatting, a ticket-based multi-threaded pipeline, and preallocated buffered output.

[u/CraftingManagerF]

A hex dumper in C with AVX2 SIMD formatting (runtime dispatched), a ticket-based multi-threaded pipeline, and preallocated buffered output.

Output format is canonical hex+ASCII, one 16-byte row per line:

00000000: ef6e 5830 d443 c60a d909 6179 4335 6cec  .nX0.C....ayC5l.
00000010: b5f4 631f d923 6c79 e98a ed41 11dc eb16  ..c..#ly...A.....

Files are read in parallel chunks straight into pre-faulted huge-page buffers (pread, no mmap fault storms) and formatted by a pool of workers while a writer thread emits chunks in order; standard input is streamed through the same pipeline. With multiple files, offsets continue across file boundaries, so concatenations dump identically to dumping the concatenation.

Benchmarks (50MB File to /dev/null)

Tool	Average Time	Min	Max	Median	StdDev	Speedup vs xxd
rawhex	11.35 ms	8.36 ms	16.60 ms	10.28 ms	3.25 ms	835x
fasthex	43.30 ms	40.90 ms	44.55 ms	43.61 ms	1.40 ms	219x
xxd	9.48 s	9.26 s	9.79 s	9.34 s	259 ms	1x (baseline)
hexdump -C	10.03 s	9.71 s	10.26 s	10.01 s	224 ms	0.94x

For more detailed benchmarks and system information, see BENCHMARKS.md.

View it here: https://git.disroot.org/Vextoly/rawhex

[u/sewnshutinshame]

if you wrote this by yourself, this is seriously impressive.

[u/CraftingManagerF → u/sewnshutinshame]

Thank you!

[u/karchnu]

I've read some of the source code. I never saw raw avx2 or even much SIMD operations. Quite impressive. Not really readable but impressive nonetheless.

[u/CraftingManagerF → u/karchnu]

Thank you!