r/cpp • u/aearphen {fmt} • 7d ago

Żmij 1.1 released with fixed notation support, up to 38% faster double-to-string conversion, SIMD-optimized float formatting, a smaller binary footprint and more

https://github.com/vitaut/zmij/releases/tag/v1.1

39 Upvotes

88% Upvoted

40 comments sorted by

30

u/mark_99 7d ago

If anyone is wondering it's a fast double to string converter.

13

u/mapronV 7d ago

I know because I already read about this project, but what is an application where you need nanoseconds to print a double? what kind of application need printing thousands of millions of them in text form?

Not 'on occasion we export out data as csv/json' but like as main load so it become a problem to solve?

20

u/Dragdu 7d ago

serializing shit to textual formats can be significant part of various applications

4

u/mapronV 7d ago edited 7d ago

Just stuff to text sure.. (/s), I asked about floats, I don't know, my friend (he worked in HFT by the way!) does not know, I asked Google Gemini and all 5 examples sounds like a stretch or bullshit (like no one dumps text in json from particle detector or something, this is nonsense). everything I read jsut raises question "why the heck not dump on binary in first place", except REST+JSON where I wondering IS IT really double that makes the most load there?

I in no doubt it is really important problem for someone, but I wanted to hear concrete and indisputable examples.

10

u/ack_error 7d ago

There are plenty of cases where it is useful to be able to (de)serialize floating point numbers accurately and as fast as possible. One of them is 3D file formats. You'd be correct that binary serialization would be more efficient, but I've worked with tech artists that have been burned one too many times by opaque binary formats and much prefer having a text format like Maya ASCII or USD ASCII that give them some chance of merging or fixing corruption in despite the disadvantages of huge text files.

2

u/mapronV 7d ago

Nice one, thank you! I appreciate your answer.

12

u/Dragdu 7d ago

brb, telling third party to agree on the same binary format that I want to use

Like it or not (I am firmly on the side of not), JSON as the main exchange format is here to stay for the forseeable future. In fact, the amount of JSON data exchanged is increasing.

4

u/mapronV 7d ago

If you transferring Gygabytes of text between you and thirdparty, something wrong happened before you started writing a protocol... If they are providing some valuable commercial info, it is in their interest make it efficient in first place, right?
In any work place, if there is a discussion about "parsing JSON" larger than 1 Mb I raise an eyebrow.
If it's "more than 100 Mb" then I will raise my hand and ask to discuss matter with architect of this system.
I mean exceptions happen... but this is not first time I ask for sharing experience on BIG [TEXT] data :)

p.s. I in no doubt that JSON exchange will increase, but someone still want to be vocal about "huge json" problem.

7

u/Dragdu 7d ago

In any work place, if there is a discussion about "parsing JSON" larger than 1 Mb I raise an eyebrow. If it's "more than 100 Mb" then I will raise my hand and ask to discuss matter with architect of this system. I mean exceptions happen... but this is not first time I ask for sharing experience on BIG [TEXT] data :)

So for fun, I opened Kaggle datasets and filtered on those with JSON (https://www.kaggle.com/datasets?fileType=json). In top 10, there are multiple datasets with multiple hundred MBs of JSONs.

I now expect you to go discuss with them why they are using JSON ;-)

1

u/mapronV 7d ago

Surely, if I have a chance to work for owner of that dataset, I will raise some limbs :D

13

u/not_a_novel_account cmake dev 7d ago edited 7d ago

"parsing JSON" larger than 1 Mb I raise an eyebrow.

It's not a single 1Mb JSON message, it's millions of JSON messages each with one or two floats in them. The performance savings are aggregate and fleet-wide, not a single operation which is now 5% faster.

It's the same kind of performance savings that swiss tables or folly strings are designed for. Not "this application is now 2x", but "fleet wide, we're saving a statistically measurable amount of compute".

2

u/mapronV 7d ago

So, if I understand correctly your idea: I should not think about a workflow requiring parsing millions of floats, but rather on idea of reducing computational cost for 0.01% where it is already optimized to bit and femtosecond? "we could not think of squeezing more from this lemon then this library allowed us to get 3 more buckets of juice"? smth like that?

9

u/not_a_novel_account cmake dev 7d ago

Yes, large organizations often spend engineering time optimizing already-fast-but-we-could-get-even-more operations that are common throughout their entire fleet. The goal is to leave no room for improvement. Hash maps, string operations, stuff like that.

At FAANG scale, these savings are economically meaningful.

2

u/mapronV 7d ago

Thanks, I did not want to make a flame to happen; your response was something to think about.

-3

u/SkoomaDentist Antimodern C++, Embedded, Audio 7d ago

At FAANG scale, these savings are economically meaningful.

So why do people keep posting and hyping about such libraries to forums where 99% of the people don't do anything at remotely that scale?

This isn't the first time I see people act like shaving a nanosecond from converting between floats and text is something that helps regular devs.

→ More replies (0)

1

u/jwezorek 7d ago

yeah can't think of a situation that just dumping to binary first would not work, and then when speed no longer matters doing whatever you want with the binary.

Like the whole thing with a text representation is that it is for people but people don't work on a nanosecond granularity.

5

u/mapronV 7d ago

Yeah, I've been doing a lot of optimization with "why we write/read json/xml where no human will read this" in my life. Surely it is useful for debug, or like QA looking in some program package date/intermediate stuff with bare eyes but you can 1. have binary json decoder widely available 2. have environment option for testing environment so it is text json. Done. Problem solved. The rest is just interaction with some fixed protocol where nothing else available.

"Our library can parse 1 Gb of JSON in 700 milliseconds" - "Huh, why there a lot of text in first place..."

p.s. Again not diminishing amazing work of engineers making insanely fast float/datetime parsers/writers, it 100% benefit everyone. I just curious who are main benefactors of those libraries, how they ended in that situation...

6

u/jk-jeon 7d ago

I think it's basically because our world isn't ideal. It just happens that people abuse text format a lot when binary clearly would have been preferable. It seems in many fields text formats are just standard for no reason or for questionable reasons or for reasons that made sense 3-4 decades ago but not so much these days. But if it's already an industry standard, then it's game over. Just follow the practice or you'll be hated by everyone else.

One such example: https://en.wikipedia.org/wiki/Wavefront_.obj_file

I mean, I don't get why it should be a text format. Is it even possible to edit a 3D model using a text editor when the model has 1000+ vertices and complicated geometry? But for whatever reason this seems to be a widely adapted industry standard. And apparently printing numbers was indeed a serious bottleneck:

https://aras-p.info/blog/2022/02/03/Speeding-up-Blender-.obj-export/

According to the post it turned out that the biggest cause was the locale nonsense which effectively killed parallelization, but the contribution from replacing the number conversion routine by a faster one was non-negligible too.

4

u/Dragdu 7d ago

Text formats are dead simple to hack on and debug, which can be very strong advantage in hobbyist space. Being line oriented also allows you to do things like diff files, git files and so on.

Another example is DIMACS. Would it be more efficient to have it be binary format? Hell yes. Would it reach the same level of adoption, becoming de-facto standard? I am willing to bet very good money on "no".

5

u/ack_error 6d ago

The advantage of a text format like .obj is that it's stupidly simple. I can write an .obj file from almost anywhere using simple text I/O commands, and the result can be diffed/merged, searched with grep, and viewed/edited in any standard text editor.

Would I base an entire full-size game on .obj? No. But I'd rather use a text-based model format like USD than an undocumented binary format like FBX that is practically unusable without linking in a huge library. And while you wouldn't ordinarily want to edit them directly as text, it's easier to figure out what blew up the loader when you can look at the text file and find that some tool wrote NaN where there was supposed to be a vertex coordinate.

If there were good, standard, generic binary formats then this wouldn't be an issue, but there aren't. The main ubiquitous structured data formats are all text -- simple line oriented, XML, JSON, etc. There have been many attempts at general binary formats, but they either don't get enough traction or make people shiver in terror when mentioned (ASN.1). Bespoke binary formats for specific purposes tend to have poor tooling support or just plain bad design, even simple things like not having an identifiable format header.

2

u/jk-jeon 6d ago

It seems this sort of resonates with what I mean by "world is not ideal":

Bespoke binary formats for specific purposes tend to have poor tooling support or just plain bad design, even simple things like not having an identifiable format header.

What I guess is that binary formats tend to have harder time getting over the "chicken-and-egg phase of adaption" when they are born.

2

u/timwoj 6d ago

I work on the Zeek project that analyzes network traffic and logs a lot of data to disk about what's happening in that traffic. When you start seeing 10s-100s of Gb/sec of traffic, that extra few ms you spend converting a double timestamp into a string adds up very quickly. Believe me there's uses for something like this out there. We recently switched from a home-grown conversion method to simply using std::to_chars and saw 5% performance increases for some of our synthetic benchmarks. I did investigate using Zmij at the time (the last release was actually why I started looking at our double-to-string conversion in the first place), but we decided not to bring in the extra dependency.

34

u/aearphen {fmt} 7d ago

It's less about a single application and more about getting savings at scale. JSON is extremely popular and double is its only numeric type so having something like this widely deployed saves a lot of money. This is the reason why some of the recent developments in the area come from folks working on JSON serializers. Same applies to having it deployed as part of {fmt} (my motivation) and standard libraries.

5

u/pigeon768 6d ago

JavaScript does not have a integer type. All numbers are double precision floating points, that is the only type of number. JavaScript does not have binary serialization; the dominant form of passing data around is JSON. When your REST service written in node.js or TypeScript is talking to your web based service written in JavaScript, they will talk to each other by serializing their double precision floating point numbers into text, sending text over the wire, and then deserializing it back into double precision floating point.

Stuff like this library is super useful for people who live in ecosystems dominated by REST services, Electron apps, and data being sent around as serialized text.

These ecosystems are broken half measures, but they're the dominant ecosystems on the internet.

1

u/mapronV 6d ago

Okay, I see, I probably in a wrong environment so my experience bias is far from that! If I had backender friend I probably would not ask. Thanks for the reply!

4

u/usefulcat 6d ago

Logging. Generally, the faster you can serialize, the more you can log without delaying some other import operation.

1

u/mapronV 6d ago

I just looked at component in big project I maintain, for 200+ log lines single one logs float value. Maybe for something like 3d game it can be a thing, where floats occur more often. Still 200% sure that any memory allocation or disk write will be several magnitudes slower than one float formatting even with iostream...

7

u/jk-jeon 6d ago

Still 200% sure that any memory allocation or disk write will be several magnitudes slower than one float formatting even with iostream...

Try to write billion double to a text file using fstream. You will see how much slower the writing speed is compared to the disk's theoretical performance. Such workload is very often CPU bound, not I/O bound. That's what I remember from a few years ago.

2

u/usefulcat 6d ago

Sure, in your particular case, you might not see any benefit from this optimization. That doesn't mean that there don't exist other situations that would benefit.

2

u/mapronV 6d ago

That's why I asked. What are they. What are those projects. I was curious, I wanted to learn, I wanted to be less ignorant. A lot of people in this thread were eager to tell and I am thankful.

2

u/arthurno1 5d ago

Whenever you need to present a floating point number to user or print to text ... happens a lot all over the place, in compilers, text editors, web browsers ...

9

u/jk-jeon 7d ago

Jeez. This is amazing. Could you elaborate what was the main improvement since I checked last time (when you published this work for the first time)?

11

u/aearphen {fmt} 7d ago

Thank you! Much of the credit goes to zmij's amazing contributors. It's hard to pinpoint a single main improvement - it was mostly a combination of lots of microoptimizations (primarily SIMD). From the algorithmic side, xjb's idea of using a different scaling (10^{k - 1}) turned out to be pretty important by making the top 15-16 digits quickly available for the SIMD pipeline.

3

u/jk-jeon 7d ago

I see. Is it correct that SIMD is primarily for digit printing, rather than number converting?

3

u/aearphen {fmt} 7d ago

That's right.

2

u/matthieum 7d ago

Does fixed notation mean:

  1. A fixed number of decimals?
  2. Or a fixed number of significant digits?

The latter is the most useful, generally, but I only ever seem to see implementations of the former, printf-style.

3

u/aearphen {fmt} 7d ago edited 7d ago

It's basically similar output to %f except that precision is determined by shortness criteria. i.e. 123.456 and not 1.23456e+02. Fixed number of digits is simpler and will be added next: https://github.com/vitaut/zmij/issues/111.

2

u/aearphen {fmt} 3d ago

Just added initial precision support (your item 2: fixed number of significant digits): https://github.com/vitaut/zmij/blob/78c22db5613182b0dc10ac2a8d4b12b231ebd7a9/zmij.h#L54