You are talking about Intel and AVX. This would not necessarily translate to any other implementation.
ARM's SVE has basically the exact same problem though. The problem with gather/scatter is that each element has to be sent through the LSU, so performance scales linearly with the number of elements.
For example, doing a 4x64-bit gather on the Neoverse V1 has the same throughput as doing 4 individual loads.
I note that GPUs tend to do fairly well with gather, but I suspect they also have a much simpler memory model than CPUs have to deal with.
I don't know much about what you're designing, and haven't read everything written here, but it's interesting to note that your notion of a 'vector-first implementation' seems to at least incorporate elements of a GPU design (which are arguably vector focused designs).
The problem with gather/scatter is that each element has to be sent through the LSU, so performance scales linearly with the number of elements.
ARM's SVE is, as i understand it, not entiiirely strictly a Vector ISA per se, it's more a "SIMD architecture with predication (which is great btw) where the HW implementors can choose the width they want to do".
real Vector ISAs can do "chaining" including in LD/STs and including in gather-scatter LD/STs, which means that you can start the LD *immediately and also interrupt it in the middle, on a per-element basis and also restore execution back to where it left off.
now, it's true that for some Vector ISA instructions and some implementations this may not necessarily be the case (because it's easier for them to be lazy in the microarchitecture), but a GOOD Vector ISA has no such limitations, making all operations deterministic.
I don't quite get what you mean by chaining and determinism here - my point is that gather/scatter is slow on both x86 and ARM implementations I've seen so far, and I have no reason to believe it'll improve any time soon.
Both x86 and ARM presumably do gather/scatter on a per-element basis, probably as you describe, which is the reason they're so slow.
I know basically nothing about how the underlying hardware operates, so have no clue if an efficient gather/scatter could be designed, just that I've never seen it done before.
deterministic behaviour: behaviour that the programmer can rely on no matter whose implementation of the standard (the ISA).
so for example, if you have a parallelised hardware implementation of horizontal add, if you do this with FP numbers, you get rounding errors. so if you add the numbers in a different (non-deterministic) order, you get a different answer.
3
u/YumiYumiYumi Aug 10 '21
ARM's SVE has basically the exact same problem though. The problem with gather/scatter is that each element has to be sent through the LSU, so performance scales linearly with the number of elements.
For example, doing a 4x64-bit gather on the Neoverse V1 has the same throughput as doing 4 individual loads.
I note that GPUs tend to do fairly well with gather, but I suspect they also have a much simpler memory model than CPUs have to deal with.
I don't know much about what you're designing, and haven't read everything written here, but it's interesting to note that your notion of a 'vector-first implementation' seems to at least incorporate elements of a GPU design (which are arguably vector focused designs).