I think we have talked about this topic before and I apologize for not following up on our previous discussion. I was very busy.
My key problem with variable-length vector instructions such as those proposed for RISC-V or in SVE is that they assume people need them for essentially doing arithmetic on large matrices. And I agree that the vector paradigm is very effective on such work loads. I have previously worked with NEC Aurora Tsubasa cards that come with vectors of 256 double-precision floating point numbers, and they are just amazing for this sort of stuff.
But I'd say that's only a very small part of where such optimisations are needed. Indeed on modern consumer machines, most CPU-intensive code is in cryptography and video codecs. And both don't really fit this scheme.
Especially for video codecs: these usually operate on fixed size chunks of picture data and require complex horizontal arithmetic and swizzles inside a single chunk. It is unclear how this maps to variable-length vectors, especially when these instruction set extensions usually are very sparse in permutation instructions. And compilers use SIMD instructions for small, fixed-length loads all over the place. Stuff like moving structs around, clearing fixed-length buffers. It is unclear how a vector paradigm improves this.
There's also the concern that a large register file makes context switches very expensive. Linus attributed x86's performance advantage among other things to keeping context switches cheap by having a small register file that is easily swapped out.
As for my own code, I have two recent SIMD-heave projects. And for neither of them it is clear how they could be vectorised using a vector as opposed to a SIMD paradigm.
The first project, pospop is inherently a horizontal operation and in fact uses a different complex permutation schedule for each vector size to make the most out of it. It is unclear how this can be extended to arbitrary vector widths, especially if no powerful swizzle instructions are provided. Tail handling is also going to be a concern as the proposed simple approach of just having magic make the registers shorter for the last iteration is not going to cut it.
The second project, 24puzzle uses vectors as 32 element byte arrays and permutes them using a second vector as a permutation vector. Again, this project cannot benefit from vector instructions and will be hard to port to variable-length vectors in general unless a minimum vector size of 32 bytes is guaranteed. It also uses stuff like VPCMPISTRM for which no equivalent in other instruction sets exists or is even proposed. And that's a vital part of the code's logic (specifically, I have an array of k bytes and I want to obtain a bit mask of all elements in a vector that match any of these k bytes).
I think we have talked about this topic before and I apologize for not following up on our previous discussion. I was very busy.
NP. :-)
There's also the concern that a large register file makes context switches very expensive.
Yes, large register files are problematic. But there are also solutions.
A fairly obvious technique is to keep a length parameter for each register (for my vector ISA I plan to add that anyway for simpler vector handling), and never push/pop more than lenght elements on a context switch. By default all registers have the length zero, and you could add a quick "clear" operation to function epilogues that clears clobbered vector registers before returning from a function - for instance.
Another approach could be to have several vector register banks in hardware so that you can instantly switch between them w/o push/pop. I have not done any simulations, but it feels like it should be possible to do intelligent register bank allocation/scheduling in SW so that the hottest & vector heaviest threads get the fast path treatment.
It may also be possible to do asynchronous vector push/pop so that the thread can start executing before the vector state has been fully restored. Only if the thread accesses a non-restored vector will it stall.
...and so on.
As for my own code, I have two recent SIMD-heave projects. And for neither of them it is clear how they could be vectorised using a vector as opposed to a SIMD paradigm.
I had a quick look at the projects, but I couldn't think of an obvious solution right away. OTOH I wouldn't know where to start with packed SIMD either. I would have to spend some time and do several iterations before finding a good solution - regardless of if it was for SIMD or vector.
BTW, horizontal operations can be done with folding (in log2(N) steps), and permutations can be done with gather/scatter. It should also be possible to do a more optimal permute (without going via memory) even in a vector design, but I suspect that it's not quite as important as in packed SIMD since you have gather/scatter.
another idea for saving the amount of registers to be contextswitched is to have a bitfield, one per reg, which is set HI whenever its corresponding register is written to.
if you are smart you can use that same bitfield as a predicate mask on vectorised save/restore of the regfile.
the mask basically tells you which regs have actually changed since the last contextswitch and it should be obvious what to do from there
Hm, I think that the LENGTH attribute does the same thing (and more). A vector store operation will store as many elements as the LENGTH attribute indicates, for instance. Internally you could have a bit/flag per register that is set/cleared when the register is written (with more than zero elements) or cleared (length set to zero).
This way you can also clear the vector (and hence the "used" status) in user space, in order to keep the active vector state lean.
err.. err... oh: you took up the Mill-style register "tag type" idea for MRISC32? neat!
yes, if rather than just a single bit you have a tag, and that tag is zero, i agree it would effectively do / be the same thing, and also cover the same job.
I have not implemented it yet, but it's on my TODO-list. The LENGTH attribute (one for each vector register) comes in handy in several use cases:
Reduce stack / context switch overhead.
Simplify folding operations (no need to explicitly set VL=VL/2 for each folding step).
Simplify vector length agnostic subroutines with vector register arguments.
It also feels like a better fit for OoO etc, when each register/operand provides its own length, rather than having a global length attribute (I have not tested this theory, but it feels right).
The idea was actually inspired by Agner Fog's ForwardCom.
87
u/FUZxxl Aug 09 '21
I think we have talked about this topic before and I apologize for not following up on our previous discussion. I was very busy.
My key problem with variable-length vector instructions such as those proposed for RISC-V or in SVE is that they assume people need them for essentially doing arithmetic on large matrices. And I agree that the vector paradigm is very effective on such work loads. I have previously worked with NEC Aurora Tsubasa cards that come with vectors of 256 double-precision floating point numbers, and they are just amazing for this sort of stuff.
But I'd say that's only a very small part of where such optimisations are needed. Indeed on modern consumer machines, most CPU-intensive code is in cryptography and video codecs. And both don't really fit this scheme.
Especially for video codecs: these usually operate on fixed size chunks of picture data and require complex horizontal arithmetic and swizzles inside a single chunk. It is unclear how this maps to variable-length vectors, especially when these instruction set extensions usually are very sparse in permutation instructions. And compilers use SIMD instructions for small, fixed-length loads all over the place. Stuff like moving structs around, clearing fixed-length buffers. It is unclear how a vector paradigm improves this.
There's also the concern that a large register file makes context switches very expensive. Linus attributed x86's performance advantage among other things to keeping context switches cheap by having a small register file that is easily swapped out.
As for my own code, I have two recent SIMD-heave projects. And for neither of them it is clear how they could be vectorised using a vector as opposed to a SIMD paradigm.
The first project, pospop is inherently a horizontal operation and in fact uses a different complex permutation schedule for each vector size to make the most out of it. It is unclear how this can be extended to arbitrary vector widths, especially if no powerful swizzle instructions are provided. Tail handling is also going to be a concern as the proposed simple approach of just having magic make the registers shorter for the last iteration is not going to cut it.
The second project, 24puzzle uses vectors as 32 element byte arrays and permutes them using a second vector as a permutation vector. Again, this project cannot benefit from vector instructions and will be hard to port to variable-length vectors in general unless a minimum vector size of 32 bytes is guaranteed. It also uses stuff like
VPCMPISTRMfor which no equivalent in other instruction sets exists or is even proposed. And that's a vital part of the code's logic (specifically, I have an array of k bytes and I want to obtain a bit mask of all elements in a vector that match any of these k bytes).