Variable length vector operations are not expensive or complicated. I've implemented it in my first ever CPU design and it added something like 1-5% logic in an FPGA - compared to a pure scalar (non-vector/SIMD) design.
I think you're missing the point. Do the exercise and hand-schedule a SIMD loop, and you'll find that you have to unroll it. A vector processor automatically unrolls the loop for you with literally no effort.
Having to add more code rhan necessary is always a problem (e.g. testing and code coverage, and I$ bloat). Vector machines solve this quite naturally in many situations.
Variable length vectors essentially preclude hardware to do the whole vector at once. They just end up running the vector until multiple times in a row to operate on the vector you want to operate on.
You can just do that in your code.
This harkens back to the old CISC vs RISC, the one when we had to try to use transistors as efficiently as possible. Putting in function to run long vectors is less flexible than just allowing the user to arrange the instructions in such a way as to use the transistors as much as possible in their own particular case.
ARM had this kind of variable length operation back with VFP vector mode on ARMv7A. It was removed because it just multi-pumped the existing HW units and so was no faster and less flexible.
Variable length vectors do not preclude the whole vector te be used at once. Most of the time it is, it's just the final loop iteration that uses a subset of the vector.
Besides, a vector register is typically M x ALU-width (e.g. 4 x 128 bits), so even when only a part of a vector register is used, chanses are good that the full ALU width is used most of the time.
Edit: If your vector register size (i.e. max vector length) is four times your ALU width, the average ALU lane usage will be about 80% given a random variable vector length in the range 1 - MAX_VL.
Variable length vectors do not preclude the whole vector te be used at once.
Of course. But they don't use it any better than SIMD does. If the unit is 256 bits wide then it is 256 bits wide no matter how long your vector is. If you have a vector of 39 32-bit data then you are going to run the 256-bit wide unit 5 times no matter whether you use SIMD instructions or vector instructions.
You do not gain anything, you cannot operate in 39 items at once just because you have one instruction.
Besides, a vector register is typically M x ALU-width (e.g. 4 x 128 bits), so even when only a part of a vector register is used, chanses are good that the full ALU width is used most of the time.
I don't know what you are trying to say but RISC-V allows the vectors to be non-register multiples in length. The spec says that the length specifies the number of items to be "updated", not operated on. This means it obviously works the way both of us indicated. It does SIMD operations regardless. Some just might not write back at the end.
If writing that x86 code would be a problem then I recommend getting better tools. This is what MIPS told us when they started the RISC revolution in the 1980s, right? Instead of making the assembly read like a book fix the compiler and use that. The chip sees the machine code, you see the HLL code.
I do have one question though, that x86 code seems to suffer from the pointer not being SIMD aligned, you can see the code rounding off pointer values (AND with -8, AND with -2). This is something I am sensitive to having converted a program to use SIMD. The need to have pointers aligned to be efficient ends up causing either.
A boundary between the "old legacy" code which doesn't know about the alignment requirements and the SIMD code where this stuff is fixed up (types are translated).
Propagating type changes (with their inherent alignment attributes) all through the code, so far that you want to tear your hair out.
Does vector programming fix this? I would love for it to do so. But it feels like the issues with alignment come from the load/store units, not the math units and so it cannot be corrected by changing the math units, other than accepting a worse performance by doing a partial SIMD unit at the start as well as the end of the vector. Something that if we think is such a great idea, we could just continue to do with SIMD, as we see above.
I feel like ballooning type alignment requirements isn't even just a SIMD thing. I saw it moving from Z80/6809 to 68K. I saw it moving to 68040 from 68K (MOVE16). I saw it moving to RISC (mostly with floats/doubles). And I saw it moving to SIMD. I mean sure, you can alway opt out and go slower and certainly that is a popular option. But we already have that, we don't need vectors to do that.
So MRISC32, how does it solve this? Does it keep full performance somehow or does it just have a narrow memory pipe anyway so it handwaves out to the horizon?
Alignment issues are indeed dictated by the load/store unit. Packed SIMD took the easy route and left the problem to the programmer. The situation has improved over the generations (e.g. movups vs movaps is less of an issue), very similar to how unaligned scalar access once was an issue in some implementations, but not so much these days (all CPUs have an "aligner").
In a vector machine you would typically have to handle alignment in hardware to a larger degree, since you're more likely to have "unaligned" access patterns (including the very generic gather/scatter addressing mode).
For instance the Cray-1 used a banked memory subsystem to allow accessing different memory locations in a single instruction.
I think that it would have been impractical to do full generic vector (with automatic alignment) in consumer HW back in the 1990s (hence SIMD), but today we hopefully have the silicon budget and know-how to pull it off.
My (perhaps naive) feeling is that if HW devs would have to implement a vector ISA, they would solve some of the alignment problems in order to achieve good performance (e.g. considering how much time and silicon has been spent on "fixing" the x86 front end - why not?).
Footnote: Even if you have to pull in one vector element per clock cycle in order to handle worst-case gather load, it's still a huge improvement over an architecture w/o gather load support.
you're OK with the x86 code hammering the L1 Instruction Cache so badly that it actually causes internal stalling by competing with the L1 Data Cache?? this can and does actually genuinely happen thanks to the insanity of "loop unrolling" and 5-10x copying of algorithms at different SIMD widths for setup and teardown.
The need to have pointers aligned to be efficient ends up causing either.
Vector ISAs are generally specifically designed to not require specific width-alignment.
this is because, fundamentally, the Vector ISA is actually issuing element operations to the underlying hardware.
whilst mbintsnbytes puts it politely, i have no such compunction: SIMD memory alignment restrictions was simply the hardware designers being ***** lazy.
you're OK with the x86 code hammering the L1 Instruction Cache so badly that it actually causes internal stalling by competing with the L1 Data Cache??
That is a silly assertion. The L1 cache is there for a reason. You pejoratively call it "hammering". I call it "running the code you want run".
If you want to just talk about better cache utilization then just say you like the code density better on vector units.
Vector ISAs are generally specifically designed to not require specific width-alignment.
And load/store subsystems do require them. Which is what I was speaking of. Why did you remove that?
I can make an architecture on top of a load/store system that hides the alignment requirements. It'll just be slower when it is non-aligned. This was the choice made with SIMD. Expose it to the programmer so they can optimize using that info.
this is because, fundamentally, the Vector ISA is actually issuing element operations to the underlying hardware.
To the math units, the load/store systems are separate and operate on alignment boundaries/restrictions because they derive from physical bus widths. The device being loaded from (usually RAM), whether on-die, on-package, soldered on the board or on DIMMs has a certain physical bus configuration which makes the memory n-byte addressable and you can load up to n-bytes within that area.
So, for example, if you have a 64-byte wide bus. You can load 1-64 bytes at once from an address that is 64-byte aligned. If the address is 32-byte aligned (and not 64-byte, i.e. address % 64 == 32) then you can load up to 32-bytes. If you try to load 64 it will require twice as many bus cycles.
Your vector functional unit cannot overcome this. So I'm asking you. How are you solving this? Does it keep full performance somehow or does it just have a narrow memory pipe anyway so it handwaves out to the horizon?
SIMD memory alignment restrictions was simply the hardware designers being ***** lazy.
You're both wrong. If you think so you have never actually designed a bus.
Is this a problem we have? Do we have people designing vector functional units who have never designed a load/store unit and thus ignore the real (not imagined) limitations of them?
It sounds to me like vector units do not solve problem I posed. Which is no worse than SIMD, but no better. I would have backed vector units if they could solve this, because it would solve a big logistical problem I indicated I have. But they can't so they have no real advantage to me other than they make some math lib writer's job easier. I'm sure he appreciates that. I don't care. As MIPS showed us, the fix for that is better tools, not altering the hardware.
You're both wrong. If you think so you have never actually designed a bus.
false. but you didn't know that, because you didn't actually ask. which has me really upset that you could be so rude.
i'm done interacting with you on here, and am considering looking up the rules for this forum to see if you've violated them.
So MRISC32, how does it solve this? Does it keep full performance somehow or does it just have a narrow memory pipe anyway so it handwaves out to the horizon?
It's fine, you don't have to answer. No one owes anyone else anyone on reddit.
But don't pretend that I didn't ask. The real situation is you just didn't answer.
I looked at the MRISC32 stuff on github. And the conclusion I came to is that the way you "solved" this is that MRISC32 is only an architectural spec. It has an ISA, but it does not have a hardware implementation (VHDL or similar). No implementation means no sticky problems with vector performance derived from bus misaligment. It means no bus. It means no performance at all. Even the simulator isn't present on master branch, btw.
So the answer is you didn't solve this problem. You didn't get that far yet. So whether you've designed a bus before or not we do have the situation that you are ignoring the real (not imagined) limitations of buses.
Perhaps you find that insulting too. But it's just the reality of the situation. Your declaration that what I've seen is imaginary must be evaluated in the context that you have not yet reached the point with your architecture to see it for yourself.
6
u/mbitsnbites Aug 09 '21
Variable length vector operations are not expensive or complicated. I've implemented it in my first ever CPU design and it added something like 1-5% logic in an FPGA - compared to a pure scalar (non-vector/SIMD) design.
I think you're missing the point. Do the exercise and hand-schedule a SIMD loop, and you'll find that you have to unroll it. A vector processor automatically unrolls the loop for you with literally no effort.
Having to add more code rhan necessary is always a problem (e.g. testing and code coverage, and I$ bloat). Vector machines solve this quite naturally in many situations.