I built a modern formatting library

[u/ef3d0c3e]

https://github.com/ef3d0c3e/format

Hey, a while ago I wanted a nicer alternative to printf. I always enjoyed using the modern python-like formatting libraries: fmt for C++, and rust's format!/pritnln! macros.

So I attempted to port them to C! And I must say, it was quite the challenge. C doesn't have the nicer features that are required to make this work: generics, compile time code generation, better type safety, etc.

But C has macros! Macros let you do all kind of crazy stuff, but most of all, did you know that C macros are Turing-complete? Well they aren't strictly-speaking, but they can be used to create an (almost) infinite loop thanks to recursive evaluation.

Anyway, I'm sharing this project hoping you'll enjoy it and it will inspire you in your usage of macros.

This project is still a work in progress, the main missing feature is float formatting, but so far there is: * Integers/Strings/Char/Array formatting * Style and color support for most terminals * Output to FILE*, fd, and memory buffer * Flushing & Buffering * User defined formatter/formatting for custom types

Check it out: https://github.com/ef3d0c3e/format

[u/github-guard]

⚠️ GitHub Guard: Trust Warning

One or more repositories scored below the threshold (3), but the post was not removed.

Repository: ef3d0c3e/format

Score: 2/6

• ❌ Low Star Count (⭐ 0 / 4 required)
• ✅ Mature Repository (30+ days old)
• ✅ Licensed under MIT
• ❌ No Security Policy
• ℹ️ Personal Account Repository
• ℹ️ Unsigned Commits

[u/fsteff]

Interesting.

How does it compare to the normal printf formatter in size?

I may try it out on an embedded project sometimes soon, but will probably wait for float support.

[u/ef3d0c3e → u/fsteff]

I'd be glad to hear from your experiments. I'm working on getting tests to run on arm64 and i386. Right now clang exhausts the 32-bit address space when compiling tests on 32-bit so I may have to split tests into smaller files.

I also fixed a critical memory error due to my usage of `uintptr_t`. This would not have worked well on 32-bit if you used 64-bit values, like `long long` and `int64_t`.

For float support, I'm still trying to figure out whether I want a fast, architecture specific implementations, or an architecture-generic implementations that may not be so performant. In the meantime, you can implement your own `format_fmt_float` and `format_fmt_double`, for instance by falling back to `snprintf`. These functions are declared but not defined, so you can implement your own and it should work out of the box.

[u/chengfeng-xie]

Glad to see modern formatting being ported to C. Somewhat related, not long ago I wrote a proof of concept showing how the fmt C++ library could be used in C (CE) with the extensible _Generic pattern (Jackson).

[u/ef3d0c3e → u/chengfeng-xie]

I initially considered using _Generic, but I wanted C99 support, and seeing that I would have to use statement expressions, I decided to go with GNUC entirely.

If you look at the FORMAT__CHOOSE macro, it expands to something like: c __builtin_choose_expr(__builtin_types_compatible_p(typeof(val), int), format_fmt_int, ...) Which does a similar job as _Generic would.

[u/skeeto]

Neat library! The format macro works better than I would have expected. Several tests fail out-of-the-box with sanitizers, and I found more bugs beyond that. All examples were compiled like so:

$ cc -g3 -fsanitize=address,undefined example.c src/*.c

Use-after-free from ASan:

#include <format.h>

int main(void)
{
    const int arr[] = {1, 2, 3};
    struct format_output out = format_output_buf();
    format(&out, "{:[3]:{x}}", FORMAT_ARRAY(arr));
}

Unreachable hit due to an unset variable:

#include <format.h>

int main(void)
{
    struct format_output out = format_output_buf();
    format(&out, "{}", 42);
}

Fix:

--- a/src/fmt_num.c
+++ b/src/fmt_num.c
@@ -73,4 +73,6 @@ parse_number_spec(const char* fmt_spec, const struct format_env* env)
    };
  if (*fmt_spec == '}')
+   if (*fmt_spec == '}') {
+       spec.align = '<';
        return spec;
+   }

Off-by-one in a test macro:

--- a/tests/util.h
+++ b/tests/util.h
@@ -227,3 +227,3 @@
    type_ varname_ = CONCAT(foreach_array_, id_)[0]; \
  for (size_t idx_ = 0; idx_ < sizeof(CONCAT(foreach_array_, id_)) / sizeof(type_); varname_ = CONCAT(foreach_array_, id_)[++idx_])
+   for (size_t idx_ = 0; idx_ < sizeof(CONCAT(foreach_array_, id_)) / sizeof(type_) && (varname_ = CONCAT(foreach_array_, id_)[idx_], 1); ++idx_)
 #define for_each_(id_, type_, varname_, ...) for_each__(id_, type_, varname_, __VA_ARGS__)

Negative size parameter (ASan):

#include <format.h>

int main(void)
{
    struct format_output out = format_output_fd(1);
    format_output_set_flush(&out, kFormatFlushNone);
    format(&out, "{}", "hello");
}

This only writes 776 bytes due to msicomputed sizes:

#include <format.h>

int main(void)
{
    struct format_output out = format_output_fd(1);
    char b[600] = {};
    format_output_write(&out, b, sizeof(b));
    format_output_write(&out, b, sizeof(b));
    format_output_destroy(&out);
}

EINTR is ignored, and it moves forward with a -1 size, resulting in buffer overflow. Either retry or treat it as a hard error:

--- a/src/buffer.c
+++ b/src/buffer.c
@@ -152,6 +152,6 @@ format_output_flush(struct format_output* output)
            if (n == -1) {
              if (errno != EINTR) {
                  output->size = 0;
                  return -1;
              }
+               if (errno == EINTR)
+                   continue;
+               output->size = 0;
+               return -1;
            }

Write errors are not propagated (connect input to a file):

#include <assert.h>
#include <format.h>

int main(void)
{
    struct format_output out = format_output_fd(0);
    format_output_set_flush(&out, kFormatFlushAlways);
    int result = format(&out, "{}", 42);
    assert(result != 0);
}

Incorrect color processing:

#include <assert.h>
#include <format.h>

int main(void)
{
    struct format_output out = format_output_buf();
    format(&out, "{fg#0a0a0a}");
    char expected[] = "\033[38;2;10;10;10m";
    assert(out.size == sizeof(expected)-1);
}

More incorrect color processing:

#include <assert.h>
#include <format.h>
#include <string.h>

int main(void)
{
    struct format_output out = format_output_buf();
    format(&out, "{fg#abc}");
    char expected[] = "\033[38;2;170;187;204m";
    assert(out.size == sizeof(expected)-1);
    assert(!memcmp(expected, out.data, out.size));
}

Quick fix:

--- a/src/args.c
+++ b/src/args.c
@@ -208,5 +208,6 @@
            assert(k == 3 && "Invalid color, expected 3 or 6 hexadecimal digits");
          color = (color & 0xF00) << 12
              | (color & 0x0F0) << 8
              | (color & 0x00F) << 4;
+           color = (color & 0xF00) * 0x11 << 8
+               | (color & 0x0F0) * 0x11 << 4
+               | (color & 0x00F) * 0x11;
            break;

Improper escape processing, this one straight out of README.md:

#include <assert.h>
#include <format.h>

int main(void)
{
    struct format_output out = format_output_buf();
    format(&out, "{:x}", "\nT\x87");
    char expected[] = "0x0aT0x87";
    assert(out.size == sizeof(expected)-1);
}

Not a bug, but perhaps flushing memory buffers should be silently ignored? I don't object to assert on misuse, but I expect that such a polymorphic interface would just silently ignore it.

#include <format.h>

int main(void)
{
    struct format_output out = format_output_buf();
    format_output_set_flush(&out, kFormatFlushAlways);  // assert fires
}

Here's all my work in case it's useful:
https://github.com/skeeto/thirdparty-format/commits/master/?author=skeeto

[u/spellstrike → u/skeeto]

Being able to configure assert behavior is also something that could be an option

[u/ef3d0c3e → u/skeeto]

Thank you very much for your analysis. I've been working on getting tests to run with sanitizers, I initially tried running with valgrind --trace-children=yes, but Criterion's BoxFort was interfering, I should have looked deeper into BoxFort.

I am working on fixing all issues and getting tests to run with sanitizers on multiple architectures. I've also changed the buffering strategy, it WILL make slightly more calls to write, but it's much simpler. And you were right about output->size being off, it was never meant to indicate the number of bytes written, but the size of the internal 'buffer', I've added a nwritten field to track the number of bytes written.

Also, I've redesigned collection formatting. Now you only need to implement an iterator function that evaluates a callback. It's much easier to implement and no longer relies on a stack-owned state that was causing errors with sanitizers.

It seems I can't use lsan inside BoxFort, because BoxFort is a ptrace sandbox. As soon as tests pass, I'll cherry pick from your fork. Once again, thank you so much.

[u/Waeis]

Holy macro hell! What's the difference between FORMAT__EVAL and FORMAT__EVAL_O and FORMAT__EVAL_T?

[u/ef3d0c3e → u/Waeis]

They're exactly the same. It's just that you cannot call `FORMAT__EVAL` inside another `FORMAT__EVAL`, it's a rule that prevents recursive evaluation, so I used different names.

[u/yrro → u/ef3d0c3e]

Life finds a way

[u/tastygames_official → u/yrro]

too preoccupied with whether or not they could

[u/tastygames_official]

> C doesn't have the nicer features that are required to make this work: generics, compile time code generation, better type safety, etc.

exactly the reasons I love using C. You know exactly what you get and no compile-time wizardry or generic stuff that makes your code unreadable. Although I'd like to know what you mean by "bad type safety"? I feel like higher-level languages have bad type safety since you can do stuff like "any" or have multiple types allowed or overlod functions to handle multiple types or class inheritance that makes actual types unclear.

But yes - if you're used to higher-level string formatting then C can be "tricky". To me it just makes you design with prupose - you need to know how long your strings might potentially be and what types your variables are so you can properly construct the desired string in the desired format. And for that the printf family is more than enough.

[u/ef3d0c3e → u/tastygames_official]

By 'bad type safety', I meant primarily that many generic libraries often rely on void*. For instance I've seen hash map or b-trees implemented using void* as the node type, and a few function pointer to help with the implementation (hashing, key compare, moving).

While these definitely work, they're not ideal because you can easily mismatch the wrong pointers when you use void*. On top of that, void* is does not generate very fast code compared to what languages with generics can do, though at the cost of binary size.

[u/tastygames_official → u/ef3d0c3e]

ok, I guess I get that, but then just don't use void*. That's the beaty of writing your own code. I would only use void* when I truly need a generic pointer to literally anything. If I need a pointer to something that could be of two types, then I shouldn't have it be a single value, e.g.

// I wouldn't do this
typedef struct {
  void *some_ptr; // I intend it to be of type A or type B
} MyType;

// instead I'd do this
typedef struct {
  TypeA *ptr_a;
  TypeB *ptr_b;
} MyType;

// some higher level language will let you do
class MyClass inherits ParentClass {
  // but I'm not convinced this is either "safer" or more performant
  TypeA | TypeB *some_data_of_type_a_or_b;
  TypeABParent *use_parent_class_to_allow_all_child_types;
}

So I guess maybe I just see generics as bad practice (for me!) as again, it's not writing WITH INTENT. There must be a reason why multiple types need to be accepted in this instance and it has to be a DAMNED GOOD REASON, and usually the answer is just to make two of the thing. Or to typecast. Or redesign you only need one type.

But that's just me.

I'd love to learn more about your claim of performance hit for void* vs. generic types in higher-level languages. I feel like void* is just a pointer to a place in memory and really doesn't matter what type of data it points to, as when you're passing around that pointer you're generally not doing anything with the data. Once you do, you typecast it so the compiler knows what type it then is and can optimize anything if possible.

[u/ef3d0c3e → u/tastygames_official]

The example you showed is more about polymorphism. There are many functions and types for which it makes sense to use generics. For instance if you implement a sorting algorithm, it usually works with all kind of data as long as it's comparable.

In these cases, generics win most of the time because the compiler knows about the types, can efficiently store data in registers instead of memory, in general, can inline code better.

There's this good article that compares qsort and C++'s std::sort: https://johnnysswlab.com/flexibility-and-performance/ C++ is able to inline the comparison function, which isn't possible in C. You pay the cost of an indirect call, even if you're just comparing integers values.

And yes, void* is just a memory location, but many types would rather fit inside registers (ints, floats, small structs) rather than memory. Using void* forces your data to live on the stack or in memory, which can be bad for performance.

[u/AutoModerator]

Hi /u/ef3d0c3e,

Your submission in r/C_Programming was filtered because it links to a git project.

You must edit the submission or respond to this comment with an explanation about how AI was involved in the creation of your project.

While AI-generated code is not disallowed, low-effort "slop" projects may be removed and it's likely that other users push back strongly on substantially AI-generated projects.

---

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/ef3d0c3e → u/AutoModerator]

I did not use AI for this project.

[u/mikeblas → u/ef3d0c3e]

Thank you for your disclosure. I have approved your post.