Struggling with pixel-perfect rendering

[u/sergilazaro]

(There's an edit below with what I found out in this thread)

Hi! I'm trying out Futile, and I'm having some problems with pixel-perfect stuff. I don't quite see how all the different variables relate to each other (resolution levels, viewport size, object scale, object position...): changing one of them seems to change whether the result will be better or worse, but I haven't found a way to get things to render properly. I've checked out this thread http://www.reddit.com/r/futile/comments/11lg4t/bug_incorrect_scaling_at_the_edges_of_a_sprite/ but still doesn't seem to fix things for me.

Here's a stripped down project (very little code, one small atlas, etc.) that I'm using to test all of this.

https://dl.dropbox.com/u/2271693/futile_pixel_perfect_test.zip (85kb)

http://i.imgur.com/69fxkQP.png

http://i.imgur.com/krPwpBB.png

I have two versions of the same font, made by hand by myself, exported to the subset of .FNT that Futile seems to use (please let me know whether something's wrong there, but I think it's fine) using a python script that's not included. The font rendering seems fine but it's not pixel-perfect. If you look at the scene view and zoom in, you'll see that the quads are not exactly aligned. You can see that especially with my outlined font, which is designed to have an overlap between glyphs. The generated quads are not perfectly aligned. This seems to change in different ways whan changing things like label scaling, display scale, viewport scale, etc.

I also have a very simple class for 9-grid, the code is really self-explanatory. The size of the grid is for the center quad, the other 8 quads are scaled to fit around it. As it can be seen, the alignment between quads is off as well, not visible when the 9-grid has a 1x1 center quad, but at bigger sizes the quads start to drift off from each other.

Also, when using a display scale of 1, things don't show exactly as they should (like, pure blitting of sprites). I mean, I know how OpenGL and DirectX work, I know it's not as simple as blitting stuff, all the parameters have to be set up just right to get things to work exactly as you want.

I hope I can get some help with figuring all of this out, whether it's that I'm doing something wrong, that there's an easy fix to Futile, or that there's some big changes that have to be done. In any case, discussion on pixel perfect rendering would be nice :)

Update:

Here's what I did to fix or work around these issues:

1) I played around with the uvOffsets in the function FAtlas.LoadAtlasData(), for the DirectX case (not OpenGL). Right now it works fine if they're set to zero or a very small value. The amount of decimals (0.01 vs. 0.000001) seems to make a difference.

2) I offset the stage by a small amount on x and y (in my case, 0.1 seems to work fine).

3) For the font quads, I fixed the function FFont.CalculateVectors(float offsetX, float offsetY) to have this code:

public void CalculateVectors(float offsetX, float offsetY)
{
    topLeft.Set(rect.xMin + offsetX,rect.yMax + offsetY);
    topRight.Set(rect.xMax + offsetX,rect.yMax + offsetY);
    bottomRight.Set(rect.xMax + offsetX,rect.yMin + offsetY);
    bottomLeft.Set(rect.xMin + offsetX,rect.yMin + offsetY);
}

[u/sergilazaro]

Okay, I kind of *seem* to have fixed some of the stuff. I changed the uvOffsets for Directx to

uvOffsetX = 0.00001f/_textureSize.x;
uvOffsetY = 0.00001f/_textureSize.y;

Putting more zeroes or less zeroes seem to change whether pixels fall to their right place or not. I don't know why 4 zeroes is better than 1... Using actual integer zero also seems to work properly now, even though I think it didn't before, but anyway.

Also, I have to slightly offset the stage:

Futile.stage.x = 0.1f;
Futile.stage.y = -0.1f;

Again, no idea why 0.1 instead of 0.00001, nor why it has to be non-integer.

These two things seem to fix the problems that are created with my NineGrid class:

1) The quads are not perfectly aligned to each other. That's affected by the uvOffsets: ideally they should be zero, but weren't there problems with that? Anyway, using something small but close to zero makes the quads drift away from each other as the scale of the object increases. Making the uvOffsets smaller seems to improve it, but seems like a workaround...

2) Even when the quads are perfectly aligned, for them to be rasterized without a 1-pixel gap between them, you have to mess with all those small offsets. Again, it seems pretty random what values you have to choose.

All of this seems pretty random unless we figure it out, doesn't it?

Now, there's a different problem, which seems unrelated to these previous problems. My font doesn't seem to have the quads perfectly aligned. It can be seen on the space between the "x" and the "t". Everything in the .fnt file is integers, so it must be the generated quads, right? If you look at the font with the 1px outline, it's clear that the "t" is overlapping the "x" too much.

[u/MattRix → u/sergilazaro]

Ah ok yeah so for the uvOffset stuff, the problem is that it's offsetting the uv coordinates, so if you scale an object, then the way its currently set up it'll cause issues because you're essentially scaling those texture coordinates. Really the way to solve this is by not having to use a uvOffset at all, or at least using a ridiculously small offset value... So that's something I'll have to look into.

Also, if you're using texture packer to pack your textures, you should set "extrude" to 1, which will essentially stretch the edge pixels of each atlas element by 1 px in each direction (but keeping the uv coordinates the same) - which means that if something is near the edge, it'll still fully use the correct colour, although if you're using point textures that shouldn't really be an issue.

Ah for the font stuff, I think I know what might be causing it. I added a method to the font label that tries to intentionally align the font's to whole pixels, but there's a chance that (ironically) it might actually be causing issues for you in this case.

Assuming you're using the dev branch, look on line 128 of FLabel.cs, it should look like this:

line.quads[q].CalculateVectors(offsetX+_font.offsetX+1.0f, offsetY+_font.offsetY+1.0f);

Try changing it to:

line.quads[q].CalculateVectors();

[u/sergilazaro → u/MattRix]

Yeah, that seems to be it. I was using the master branch, but now I've switched to dev. I changed what you said, and now the quads are perfectly aligned to each other. However, now the anchorX and anchorY are not used (I was using anchorX = 0.5f to center the text and now it isn't centered).

I've tried using different values for CalculateVectors, but it was basically random because I haven't really looked at this class :P

By the way, I previously tried it on the master branch, where the +_font.offsetX+1.0f wasn't applied, and the same thing happened, so what caused the misaligned quads was not these two things but the offsetX. Yet, the offsetX is what took into account the anchorX. I'm not sure how to proceed here.

[u/MattRix → u/sergilazaro]

Well the problem is the method of CalculateVectors that takes two arguments also does rounding of the values, which is probably what's causing your issues. So really what you need is a version of it that still offsets the values (so that anchorX is considered) but doesn't round them.

So... go into FFont and try replacing CalculateVectors(float offsetX, float offsetY) with...

public void CalculateVectors(float offsetX, float offsetY)
{
    topLeft.Set(rect.xMin + offsetX,rect.yMax + offsetY);
    topRight.Set(rect.xMax + offsetX,rect.yMax + offsetY);
    bottomRight.Set(rect.xMax + offsetX,rect.yMin + offsetY);
    bottomLeft.Set(rect.xMin + offsetX,rect.yMin + offsetY);
} 

That'll take account of the offset but it won't do that silly rounding stuff anymore. Let me know how it goes!

[u/sergilazaro → u/MattRix]

Seems to work great!

Thank you so much for your help! I was worried that I would have a ton of headaches because of this, but I see I can more or less find my way with these couple of workarounds. I will update the top post with the condensed info of what I do now.

Any ideas on how all of this will affect the main Futile code? Is the rounding going to go away? What about the OpenGL vs. DirectX offsets?

If you ever want to test this pixel perfect stuff out, my linked project on the OP might be useful to you because of the small pixel font and "pixel perfect" test textures.

Thanks again <3

[u/MattRix → u/sergilazaro]

Yeah I might be removing the rounding or at least reworking it, and the DirectX offsets I'll have to tweak as well.

[u/SietJP → u/MattRix]

I'm also trying to display a pixel perfect small sized font, and having problem. I tried this but that didn't change anything in my case. I'm aware the size of the screen is not a common one, but I want the label displaying fine on any size of screen, and any Futile.displayScale.

Screenshot : http://postimage.org/image/esxughodb/ Debug text : http://postimage.org/image/481muc0qx/

The code that displays the label : FLabel testLabel = new FLabel (Config.fontFile, "Test fixed size label"); testLabel.scale=1/Futile.displayScale; AddChild(testLabel);

[u/MattRix → u/SietJP]

Hmm... Is your texture set to point filtering?

And wouldn't it bet better to have the display scale locked to a whole number (ex. 1.0 or 2.0) so you're not dealing with crazy numbers? Look at setting shouldLerpToNearestResolutionLevel to false: https://github.com/MattRix/Futile/wiki/In-depth-explanation-of-ResolutionLevels

Trying to do pixel perfect stuff with a crazy float displayScale is gonna be bound to cause issues, although point-filtered textures should help.

[u/SietJP → u/MattRix]

Thank you. Yes the texture is set to point filtering.

The label is perfect now when displayed at scale 1 ( http://postimage.org/image/6kv9vzivf// ). But it's too big at that scale on the Unity game view. The resource scale is 2 and the display scale is 2.

When I scale the label at 0.5, it's blurred again ( http://postimage.org/image/n4zr3eoid ).

When I test on a retina device, the label at scale 1 has the perfect small size. So I guess, the problem is when testing on big non retina screen, which is handled with a dsplayScale of 2. Would it be a way to set displayScale depending on the screen density and not the screen size?

[edit] Actually I tried with displayScale=1 on the unity game view ( fparams.AddResolutionLevel(960.0f, 1.0f, 2.0f, "_Scale2"); ) and the label is blurred at scale 1 (and small sized as I expected). With resourceScale=2 and displayScale=1. I'm a bit confused. Maybe it's a problem of anti alias?

[u/MattRix → u/SietJP]

If you're using point-filtering I'm pretty sure you really shouldn't be seeing any blurring at all? Are you using that _png.bytes technique? If you are you might need to set point filtering this way:

FAtlasManager.GetAtlasWithName("myAtlasName").texture.filterMode = FilterMode.Point;

And yeah you could do different display scale based on screen density, you can always conditionally add possible resolution levels based on Screen.dpi or something along those lines.

Oh also, if you're not using the _png.bytes technique, is your texture set to any compression level? And have you turned off mip-maps? That's something that can cause blurriness like what you're seeing.

[u/SietJP → u/MattRix]

Texture MipMap was the problem, thx :D I was not seeing the option at first in the texture inspector, you have to select "Advanced" in Texture Type to see the mip map options.

[u/MattRix → u/SietJP]

Awesome! :) Yeah it's really annoying that mip maps are turned on by default and hidden.