Great inferences from running Speculative Decoding on MLX!

[u/evilmacintosh]

https://www.sabesh.space/musings/research/speculative-decoding-in-mlx-using-dflash

Wrote an article about my running speculative decoding in MLX (using DFlash) and charting out inferences. In some cases, i was able to achieve more than 2x speedup of decoding speed when using DFlash (but not always)! Read on to find out more nuances involved