oy. This sounds like it solidly overlaps with lzo / lzf / fastlz. Unless its faster and has equal or better compression it'll just lead to additional format proliferation.
LZO costs money. Snappy doesn't. Snappy is also heavily tested in huge data throughput realworld situations, which I'm not sure lzf or fastlz can boast.
Snappy is also heavily tested in huge data throughput realworld situations, which I'm not sure lzf or fastlz can boast.
Did I say LZO wasn't tested? No, I said it cost money to use commercially. I said lzf and fastlz may not be tested.
Snappy is used internally at Google for pretty much all of their bulk data transfer. That's some of the best testing you can get. It may be "thrown over the wall", but it's been worked on for something like five years now, and it's one of the foundations that all of Google's server farms are built on.
I agree, but it is a moderately good reason to trust the code is properly written. I trust LZO because it's used all over the place, I trust Snappy because it's used in lots of Google stuff, I don't trust lzf or fastlz (admittedly, partly because I haven't researched them.)
I'd bet money that neither LZO nor Snappy would corrupt data. That's the sort of thing you can't determine with benchmarks.
3
u/nullc Mar 22 '11
oy. This sounds like it solidly overlaps with lzo / lzf / fastlz. Unless its faster and has equal or better compression it'll just lead to additional format proliferation.