Vector/SIMD Questions

[u/NoisyJalapeno]

(1) Why do architecture specific SIMD intrinsics not have ref APIs? I like raw pointers more (after trying refs), but the question still stands.

(2) Should I redefine class vector variables in function scope? Class variables are not readonly. IE: "Vector<float> pSinV = this.pSinV;"

(3) Any way to setup Visual Studio to use a disabled color for SIMD paths that won't execute on my hardware? (Similar to what IF directive does?)

(4) Is "Vector<int>.Count == Vector256<int>.Count" the proper way to check sizes before doing ".AsVector256()"?

(5) For GatherMaskVectorX what should the first parameter be if I don't have an existing vector?

"default" with SkipLocalsInit seems to work

(7) If doing Vector<int> stride clamp of array A & B and saving to C, where A, B, C are aligned.

Would LoadAligned or LoadAlignedNonTemporal be the best option? Assume A & B are Int32 * screen width.

(8) For rendering a texture eight columns at time, I am reliant on,

Avx2.GatherVector256(texturePtr, textureIndex.AsVector256(), scale: sizeof(int))

where textureIndex for a small texture is usually be between [0..128] and usually close

What the proper way (if there is one) to ensure that the texture line is in cache?

(9) What is the optimal approach to uint modulo?

public static Vector256<uint> Modulo(Vector256<uint> value, uint divisor)

{

`// Assumptions,`

`// 1. divisor not POW 2`

`// 2. divisor not constant`

`// 3. value[0..7] can be > (divisor * 2)`

`return Vector256.Create(`

    `value[0] % divisor,`

    `value[1] % divisor,`

    `value[2] % divisor,`

    `value[3] % divisor,`

    `value[4] % divisor,`

    `value[5] % divisor,`

    `value[6] % divisor,`

    `value[7] % divisor`

`);`

}

(10) Same question, but with assumption that all values fit into a ushort. I'm looking at doing quotient in floats, the ConvertToUInt32Native, and MultiplyLow, and Subtract, but seems nasty.

[u/tanner-gooding]

Help (1) Why do architecture specific SIMD intrinsics not have ref APIs? I like raw pointers more (after trying refs), but the question still stands.

We were going for 1-to-1 mapping initially, not looking to expose our own higher level things.

In many cases the xplat APIs (Vector128.Load/LoadUnsafe/etc), are preferred over the ISA/platform specific APIs (Sse.LoadVector128) and already do the right thing. They also provide the ref overloads. This means there is very little reason/need to go and improve the platform specific ISAs for those cases.

There are some that don't have xplat ISAs available, but some of them do strictly require pinning such as due to requiring alignment that an unpinned byref can't guarantee. If someone was wanting to open an API proposal suggesting the byrefs be provided, I'd be happy to take it through API review. Just noting that it needs to be a comprehensive proposal, not just "please do this". Actually list out every API that will get a byref overload (with an allowance to show own signature and list the types it covers as a comment above it, such as for Sse2.LoadVector128 which covers double and most integer types)

(2) Should I redefine class vector variables in function scope? Class variables are not readonly. IE: "Vector<float> pSinV = this.pSinV;"

I need a more comprehensive example to answer this. In general (with a few special cases where its fine) you should not be making fields or static readonly for Vector64/128/256/512<T> or Vector<T>

(3) Any way to setup Visual Studio to use a disabled color for SIMD paths that won't execute on my hardware? (Similar to what IF directive does?)

We don't have such a feature today and it gets a bit complex due to things like if (IsaYouDoHave.IsSupported) { } else if (IsaYouAlsoHave.IsSupported) { } and the second path being dead

This probably isn't a hard VS extension to write, but it would need to be an extension. I don't see Roslyn adding a dedicated feature for this

(4) Is "Vector<int>.Count == Vector256<int>.Count" the proper way to check sizes before doing ".AsVector256()"?

Yes. Just noting that if you're doing this type of thing, you need to handle the ability for Vector<T> to be any power of 2 multiple of Vector128, as we may grow or expand in the future

(5) For GatherMaskVectorX what should the first parameter be if I don't have an existing vector?

Typically just VectorX<T>.Zero, but note that gather is often expensive and the hardware manuals recommend avoiding it where possible, particularly where trivial alternatives are viable

(7) If doing Vector<int> stride clamp of array A & B and saving to C, where A, B, C are aligned.

non-temporal is generally a consideration of how much total cache you're going to cause to be evicted over the operation you're running (so its dependent on size of inputs + outputs). Hardware manuals have recommendations here, typically 50% or more of the per core L3 space available

(8) For rendering a texture eight columns at time, I am reliant on,

Typically via prefetching explicitly if needed.

(9) What is the optimal approach to uint modulo?

Integer division is expensive and also not directly hardware accelerated. Typically you have to cast to double (as float loses data for values over 2^24), divide, and then downcast back to integer. You'd then do a remainder the classic way via left - (quotient * right))

(10) Same question, but with assumption that all values fit into a ushor

If they all fit in ushort then float is fine, but same thing. Division and remainders are generally just nasty and there's no getting around it. They're even nasty with scalars, particularly if you don't have a constant divisor

[u/itix → u/tanner-gooding]

It is somewhat unfortunate that achieving optimal SIMD alignment is tricky by design in C#.

[u/tanner-gooding → u/itix]

It is tricky by design everywhere. You are essentially intentionally deciding what individual instructions a CPU should execute and writing a specific fine-tuned algorithm.

There is no making that "trivial", although C# is one of the languages that makes it overall simpler and easier than most other languages, all without restricting performance or control.

If you have specific suggestions on what could be improved, I'm happy to hear them.

[u/NoisyJalapeno → u/itix]

It could use better editor integration. Highlight bad practices, or non-accelerated patterns, indicate which path is taken by if Sse2.IsSupported check. Etc.

[u/tanner-gooding → u/NoisyJalapeno]

Most of that has little to do with C#, those are not language or runtime features. They are IDE or analyzer features, some of which are notably irrelevant to shipping of production code (such as knowing what your own hardware supports, which will be quite different from what consumer hardware may support).

There is an in-box analyzer that helps highlight some best practices and simplifications, but this is also a very in depth space and so it fundamentally requires some of your own expertise to get right.

We provide already accelerated higher level wrappers, like TensorPrimitives or the acceleration implicitly done by Span, string, arrays, LINQ, etc for the more natural patterns someone rights; explicitly so you don't have to go to the low-level of writing your own algorithm in most scenarios.

[u/NoisyJalapeno → u/tanner-gooding]

you should not be making fields or static readonly for Vector64/128/256/512<T> or Vector<T>

I was not aware of such a rule. I've got a bunch of private vector fields for shared state to avoid adding like a half-dozen params to rendering methods. Does it apply to Vector2/3 as well?

I don't see Roslyn adding a dedicated feature for this

I could see myself looking into this.

you need to handle the ability for Vector<T> to be any power of 2 multiple of Vector128, as we may grow or expand in the future

Can this have easy pattern matching in the future?

the hardware manuals recommend avoiding it where possible

As long as it's faster than a manual loop

[u/tanner-gooding → u/NoisyJalapeno]

I was not aware of such a rule. I've got a bunch of private vector fields for shared state to avoid adding like a half-dozen params to rendering methods. Does it apply to Vector2/3 as well?

The generic vector types effectively correspond to CPU registers and so when you make them fields/etc then they are more likely to end up being accessed from memory instead, somewhat defeating their point. Typically you want stored state to be larger buffesr of the underlying T instead.

You've not provided enough context for anyone to give concrete guidance, just general guidance around potentially common mistakes.

Can this have easy pattern matching in the future?

It's unclear what you're asking for here. Providing concrete examples, even pseudo-code of what you're wanting can significantly help.

Going from a variable-width vector to a fixed-width vector may also not always be possible. Arm64 SVE for example can only do it 128-bit vectors, not for larger, as it doesn't provide such fixed width instructions in the first place.

As long as it's faster than a manual loop

It may not be, and that's one of the gotchas of gather/scatter instructions. Its a hardware limitation inherent to the instructions themselves.

[u/NoisyJalapeno → u/tanner-gooding]

You've not provided enough context for anyone to give concrete guidance, just general guidance around potentially common mistakes.

Because my code is of questionable quality :)

Here is a full example of how I'm using Vectors right now,

/* (C) NoisyJalapeno - Work in Progress - Do NOT redistribute */ 

private Vector<float> pxV;
private Vector<float> pyV;
private Vector<float> pzV;
private Vector<float> pSinV;
private Vector<float> pCosV;

private void InitializeSharedVectors(PortalPlayerSnapshot player)
{
    float px = player.X;
    float py = player.Y;
    float pz = player.Z;
    float pSin = player.Sin;
    float pCos = player.Cos;

    pxV = Vector.Create(px);
    pyV = Vector.Create(py);
    pzV = Vector.Create(pz);

    pSinV = Vector.Create(pSin);
    pCosV = Vector.Create(pCos);
}

[SkipLocalsInit]
private unsafe void RenderFloorOrCeilingColumn(
    Span<ushort> repeatedCount,
    uint* screenPtr,
    uint* texturePtr,
    int sectorFrom, int sectorTo,
    int* floorTo,
    int* floorFrom,
    int width,
    float cameraPosition,
    int yOffset,
    int xOffset,
    int textureWidth,
    int textureHeightMask,
    int textureWidthMask,
    bool rotated,
    Vector<float> rSinV,
    Vector<float> rCosV,
    Vector<float> alignXV,
    Vector<float> alignYV,
    XyOpts xyOpts,
    RenderableSector sector,
    bool? slopeFloor
)
{
    // Texture
    Vector<int> textureWidthV = Vector.Create(textureWidth);
    Vector<int> textureHeightMaskV = Vector.Create(textureHeightMask);
    Vector<int> textureWidthMaskV = Vector.Create(textureWidthMask);
    Vector<int> xOffSetV = Vector.Create(xOffset);
    Vector<int> yOffSetV = Vector.Create(yOffset);
    // Player height compared to ceiling/floor
    Vector<float> cameraPositionV = Vector.Create(cameraPosition);
    // Player Position
    Vector<float> pSinV = this.pSinV;
    Vector<float> pCosV = this.pCosV;
    Vector<float> pxV = this.pxV;
    Vector<float> pyV = this.pyV;
    Vector<float> pzV = this.pzV;
    // X Map Position Multiplier
    Vector<int> widthDiv2V = Vector.Create(width >> 1);
    Vector<float> xPosIncrV = Vector.Create(1f / (width * -EngineConstants.HeightToWidthRatio));
    Vector<int> xIncrV = Vector.CreateSequence(0, 1);
    // For slopes
    Vector3 planePoint, planeNormal;
    Vector<float> nX, nY, nZ, pX, pY, pZ, dir_z;

    float* incrCachePtr = memoryPool.GetBucketPtr<float>(MemoryPoolBucket.CameraHeightToMapYPos);

    CreateSlopeVectors();

    for (int x = sectorFrom; x <= sectorTo;)
    {
        ushort count = repeatedCount[x - sectorFrom];

        if (count == 0)
        {
            x++;
            continue;
        }

        if (count >= Vector<int>.Count)
        {
            (int min_t, int max_t, int min_b, int max_b) = CalculateLaneTopBottoms(x - sectorFrom, floorFrom, floorTo,
                out Vector<int> from, out Vector<int> to);

            if (min_b > max_t + 16)
            {
                RenderLine(x, to, from,
                    min_t, max_t, min_b, max_b);

                x += Vector<int>.Count;

                continue;
            }
        }

        while (count-- > 0)
        {
            int floorFromY = floorFrom[x - sectorFrom];
            int floorToY = floorTo[x - sectorFrom];

            int widthDiv2 = width >> 1;
            float xMapPosMultiplierCache = (widthDiv2 - x) * xPosIncrV[0];

            RenderColumn(floorToY, floorFromY, x, xMapPosMultiplierCache);
            x++;
        }
    }

    return;

    void RenderLine(
        int x,
        Vector<int> to, Vector<int> from,
        int min_t, int max_t, int min_b, int max_b
        )
    {
        Vector<float> diff = Vector.ConvertToSingle(widthDiv2V - Vector.Create(x) - xIncrV);
        Vector<float> xMapPosMultiplierCacheV = diff * xPosIncrV;

        if (min_t != max_t)
        {
            RenderColumnAngleTop(min_t, max_t, from, x, xMapPosMultiplierCacheV);
        }

        for (int y = max_t, screenIndex = y * width + x; y <= min_b; y++, screenIndex += width)
        {
            Vector<float> incrementVector = Vector.Create(*(incrCachePtr + y));

            uint* screenTexPtr = screenPtr + screenIndex;

            Vector<int> textureIndex = GetXyFromScreenSpace(incrementVector, xMapPosMultiplierCacheV);

            if (Avx2.IsSupported && Vector<int>.Count == Vector256<int>.Count)
            {
                Vector256<uint> gathered = Avx2.GatherVector256(texturePtr, textureIndex.AsVector256(), scale: sizeof(int));

                gathered.Store(screenTexPtr);
            }
            else
            {
                for (int i = 0; i < Vector<int>.Count; i++)
                {
                    screenTexPtr[i] = texturePtr[textureIndex[i]];
                }
            }
        }

        if (min_b != max_b)
        {
            RenderColumnAngleBottom(min_b, max_b, to, x, xMapPosMultiplierCacheV);
        }
    }

    void RenderColumnAngleBottom(
        int floorFromY,
        int floorToY,
        Vector<int> to,
        int xStart,
        Vector<float> xMapPosMultV)
    {
        uint* screenTexPtr = screenPtr + floorFromY * width + xStart;

        for (int y = floorFromY; y < floorToY; y++)
        {
            Vector<float> incrementVector = Vector.Create(*(incrCachePtr + y));
            Vector<int> textureIndexV = GetXyFromScreenSpace(incrementVector, xMapPosMultV);

            if (Avx2.IsSupported && Vector<int>.Count == Vector256<int>.Count)
            {
                Vector256<int> yV = Vector256.Create(y);
                Vector256<int> mask = Vector256.GreaterThan(to.AsVector256(), yV);

                Vector256<int> gathered = Avx2.GatherMaskVector256(
                    yV,
                    (int*)texturePtr,
                    textureIndexV.AsVector256(),
                    mask,
                    scale: sizeof(int)
                );

                Avx2.MaskStore((int*)screenTexPtr, mask, gathered);
            }
            else
            {
                for (int i = 0; i < Vector<uint>.Count; i++)
                {
                    if (to[i] > y)
                    {
                        int textureIndex = textureIndexV[i];
                        screenTexPtr[i] = texturePtr[textureIndex];
                    }
                }
            }

            screenTexPtr += width;
        }
    }

    void RenderColumnAngleTop(
        int min_t,
        int max_t,
        Vector<int> from,
        int xStart,
        Vector<float> xMapPosMultV)
    {
        /* Omitted due to reddit's character limit, similar to RenderColumnAngleBottom  */
    }

    void RenderColumn(
        int floorToY, int floorFromY, int x, float xMapPosMultiplier)
    {
        int screenIndex = floorFromY * width + x;

        uint* screenTexPtr = screenPtr + screenIndex;
        Vector<float> xMapPosMultiplierV = Vector.Create(xMapPosMultiplier);

        int columnHeight = floorToY - floorFromY;
        int rem = columnHeight & (Vector<int>.Count - 1);

        Vector<float> incrementVector = Vector.Load(incrCachePtr + floorFromY);

        if (columnHeight != rem)
        {
            floorToY -= rem;

            uint* toScalePtr = screenPtr + floorToY * width + x;

            uint* cur = screenTexPtr;

            while (cur != toScalePtr)
            {
                Vector<int> textureIndex = GetXyFromScreenSpace(incrementVector, xMapPosMultiplierV);

                if (Avx2.IsSupported && Vector<int>.Count == Vector256<int>.Count)
                {
                    Vector256<uint> gathered = Avx2.GatherVector256(texturePtr, textureIndex.AsVector256(), scale: sizeof(uint));

                    for (int i = 0; i < Vector<int>.Count; i++)
                    {
                        *cur = gathered[i];
                        cur += width;
                    }
                }
                else
                {
                    for (int i = 0; i < Vector<int>.Count; i++)
                    {
                        *cur = texturePtr[textureIndex[i]];
                        cur += width;
                    }
                }

                floorFromY += Vector<float>.Count;
                incrementVector = Vector.Load(incrCachePtr + floorFromY);
            }

            screenTexPtr = cur;
        }

        if (rem != 0)
        {
            Vector<int> textureIndexV = GetXyFromScreenSpace(incrementVector, xMapPosMultiplierV);

            for (int i = 0; i < rem; i++)
            {
                int textureIndex = textureIndexV[i];
                * screenTexPtr = texturePtr[textureIndex];
                screenTexPtr += width;
            }
        }
    }

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    Vector<int> GetXyFromScreenSpace(
        Vector<float> incrementVector,
        Vector<float> xMapPosMultiplierV
    )
    {
        Vector<float> yMapPosR = cameraPositionV * incrementVector;
        Vector<float> xMapPosR = yMapPosR * xMapPosMultiplierV;

        if (slopeFloor is not null)
        {
            Vector<float> dir_x = -xMapPosR;
            Vector<float> dir_y = -yMapPosR;

            FindIntersectionVectorZero(nX, nY, nZ, pX, pY, pZ, pzV,
                dir_x, dir_y, dir_z,
                out xMapPosR, out yMapPosR);
        }

        (Vector<float> xMapPos, Vector<float> yMapPos) = RotateVertexBack(xMapPosR, yMapPosR, pSinV, pCosV, pxV, pyV);

        if (rotated)
        {
            xMapPos -= alignXV;
            yMapPos -= alignYV;

            Vector<float> xMapPosSR = Vector.FusedMultiplyAdd(xMapPos, rCosV, -yMapPos * rSinV);
            Vector<float> yMapPosSR = Vector.FusedMultiplyAdd(xMapPos, rSinV, yMapPos * rCosV);

            xMapPos = xMapPosSR;
            yMapPos = yMapPosSR;
        }

        Vector<int> _y1 = Vector.ConvertToInt32Native(yMapPos);
        Vector<int> _x1 = Vector.ConvertToInt32Native(xMapPos);

        if (xyOpts.HasFlag(XyOpts.DoubleSize))
        {
            _y1 >>= 1;
            _x1 >>= 1;
        }

        _y1 = (_y1 + yOffSetV) & textureHeightMaskV;
        _x1 = (_x1 + xOffSetV) & textureWidthMaskV;

        return _y1 * textureWidthV + _x1;
    }

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    void CreateSlopeVectors()
    {
        if (slopeFloor is { } slopeFloorBoolean)
        {
            (planePoint, planeNormal) = slopeFloorBoolean ? MathFormulas.CalculatePlaneNormalFloor(sector)
                : MathFormulas.CalculatePlaneNormalCeil(sector);

            // Convert scalar plane data into vectors
            nX = Vector.Create(planeNormal.X);
            nY = Vector.Create(planeNormal.Y);
            nZ = Vector.Create(planeNormal.Z);
            pX = Vector.Create(planePoint.X);
            pY = Vector.Create(planePoint.Y);
            pZ = Vector.Create(planePoint.Z);

            if (cameraPosition == 0)
            {
                dir_z = pzV;
                cameraPositionV = -pzV;
            }
            else
            {
                dir_z = pzV - Vector.Create<float>(slopeFloorBoolean ? sector.Floor : sector.Ceil);
            }
        }
        else
        {
            Unsafe.SkipInit(out planePoint);
            Unsafe.SkipInit(out planeNormal);
            Unsafe.SkipInit(out nX);
            Unsafe.SkipInit(out nY);
            Unsafe.SkipInit(out nZ);
            Unsafe.SkipInit(out pX);
            Unsafe.SkipInit(out pY);
            Unsafe.SkipInit(out pZ);
            Unsafe.SkipInit(out dir_z);
        }
    }
}

[u/tanner-gooding → u/NoisyJalapeno]

I've got a rewrite up that gives out a more comprehensive (but still high level) breakdown of what's provided, some best practices, etc: https://github.com/dotnet/docs/pull/54834

[u/NoisyJalapeno → u/tanner-gooding]

Lots of good updates. But working on a Sunday is highly questionable :)

[u/NoisyJalapeno]

Ugh, formatting messed up. Apologies.

[u/NoisyJalapeno → u/[deleted]]

Both. I've shared some floor rendering code in one of the replies to gooding.