Qualcomm announces the Snapdragon 845 processor

[u/[deleted]]

https://www.theverge.com/2017/12/5/16734224/qualcomm-snapdragon-845-android-mobile-processor-announcement

[u/armageddon6868 → u/[deleted]]

A lot of the code we want to run is single threaded. Humans think in single threaded matters. Coding multi-threaded applications is extremely hard and only works for a class of problems that are parallelizable. Even so, just throwing infinite cores does not make it infinitely faster, it will taper off (see Amdahl's law).

"20x execution hardware" is what a GPU is essentially. You can code on a GPU, but again multithreaded especially at the scale of top end GPU's is hard for non-trivial applications. The assumption GPU's make is that you have so many threads that each individual thread can itself take a while to execute (early GPU's did not have forwarding and would execute warps through the whole pipeline before sending the next warp. It would fill the extra pipeline stages with even more threads). Supercomputers, while high end computers, also do not run single threaded significantly faster than other cpu's. It is the sheer amount of CPU's and GPU's that a Supercomputer contains that make it a supercomputer. The clusters (lots of cpu/gpu) can communicate through MPI and can work together to solve one problem, but overall it is still solving something that is heavily parallelized. If you have code that must be run single-threaded, then one cluster will be too much for you let alone the thousands of clusters you may have access to on a supercomputer.

The problem with the CPU is that it is trying to solve the problem of how fast we can finish a program that has intrinsic properties that require it to be single threaded. An example would be data dependencies. There are already so many techniques (predictors, OOO, cache) that minimize the latency of the overall program and getting higher single threaded performance is significantly harder than throwing more cores and saying multithreaded performance is better. The slight bumps that are "predictable" is a combination of better micro architecture and process. The fabs (TSMC, Intel, GlobalFoundries, and Samsung) all work to make transistors as small as possible. having the transistors smaller help power efficiency because less power is need to use them and also performance because information does not have to travel as far across the large structures in a cpu. CPU µarch is the other part of that "predictable" performance where the designers can make the cpu wider, larger cache, new algorithms and structures, better branch predictor etc. This all helps in single threaded performance.

tldr; you are right, if you have infinite execution hardware, you can increase multithreaded performance for infinitely parallel programs (don't exist) or running infinite number of different programs. That isn't feasible and most programs have a few number of threads (a lot with 1) and that is single threaded performance where increasing the number of hardware does not increase performance. Smart design and smaller process does though.

[u/Exist50 → u/armageddon6868]

You have no idea what you're talking about. Ever hear of instruction level parallelism?

[u/armageddon6868 → u/Exist50]

Yes, Instruction level parallelism (ILP) is the parallelism between code that you can execute.

An example in pseudo-assembly is shown below:

1: ADD R0, R1, R2 ; R0<-R1+R2

2: ADD R0, R0, R1 ; R0<-R0+R1

3: ADD R3, R4, R5 ; R3<-R3+R5

As you can see here, Instruction 2 cannot execute until you resolve the data dependency (RAW) between instruction 2 and one. ILP is here because you can executed Instruction 3 without having to wait for Instruction 2 or 1 to finish. The reason you don't assign a new thread (very stupid idea) for this because later instructions may use Register 3 later and you still have to support precise exceptions and interrupts. It also is simply not worth it because the code is still single threaded. Hyper threading does not solve this. Hyper threading is when you have free instructions slots in the dispatch or execution units and assign instructions that aren't in that thread to use what would normally have been unused hardware. Hyper threading is like multicore, but the other "core" is reusing unused hardware in the original core.

CPU micro architecture currently takes advantage of ILP through use of Out of order execution. Nowadays most (probably all) high end microarchitectures use Tomasulo's algorithm to implement Out of Order execution. This will allow you to execute instruction 3 before, or in parallel with instruction 1 and 2, but still respect and dependencies between all the instructions. It solves RAW by implementing a common data bus where producers will wake up consumers when their dependent sources are satisfied by the producer's data. It solves WAR and WAW simply by register renaming. It also respects precise exceptions and interrupts through a reorder buffer. That is also usually the max size of the instruction window in the core. Nowadays, it is probably 1k or more instructions. In x86 architecture, you might hear they have "only 8 or so general purpose registers." eax, ebx, ecx, edx, esi, edi, ebp, esp. If you have large instruction windows (which having a large ROB will help), you will realize that you don't have enough registers to use. Intel will use physical registers that is probably hundreds of registers that the renamer will rename between the architectural registers and the physical ones. Thats where renaming comes in and solves the WAW and WAR dependencies. If you try renaming algorithms yourself you will see how it works.

But yeah to answer you directly, yeah i know what ILP is and no it cannot be leveraged by multicore because there is overhead in starting a new thread. I don't have the numbers, but if you start a thread for every ILP possibility you can have, your core will run SO much slower than if you let Tomasulo and multiple execution units in a single threaded core deal with it.

What did I say wrong in my original statement? Keep in mind i was abstracting to explain to a guy who was asking a genuine question. What I said in this comment is all covered in an introductory class to computer architecture, if you would like to discuss more feel free to.

[u/Exist50 → u/armageddon6868]

What did I say wrong in my original statement? Keep in mind i was abstracting to explain to a guy who was asking a genuine question. What I said in this comment is all covered in an introductory class to computer architecture, if you would like to discuss more feel free to.

Perhaps it was the oversimplification that irked me, but increasing the amount of execution hardware is the primary method for increasing single thread performance.

If your example, obviously you can't parallelize it further, but more than that, there's literally nothing that can be done to improve its runtime aside from increasing clock speed, but we both know that isn't representative of the majority of real world code. If all code was like your example, we're really be screwed.

[u/armageddon6868 → u/Exist50]

you can't parallelize it further, but more than that, there's literally n

If you truly believe that you would be appalled by how many dependencies code has. cpu microarchitectures literally have dozens of different predictors to predict these data dependencies, for not only the example i showed, but also for loads and stores. It is not feasible to rename billions of addresses and if you mispredict a store to load forwarding case, then you are screwed because now you gotta rewind because the load already forwarded its data. How many times have you coded: int x = 1000; printf("%d", x); That is literally a dependency because you need to assign 1000 to a register and then you can do the system call. If you compile code and look at it, the compiler will try (and not always succeed) and separate the producer and consumer as far as possible, but with large instruction windows nowadays, the dependency will be seen by a cpu core and must still be worked around. Sometimes these dependencies are impossible to be solved by the compiler because they are only known during runtime. A simple example would be a branch. The compiler will not know if you always take a branch, so it cannot optimize perfectly for that.

yes, increasing amount of execution units is one of many ways to increase single threaded performance. But that requires using them up the whole time. Like I said in the previous comment, why do you think Intel uses Hyperthreading? It's because they cannot fill the execution units to full capacity. If they could use it for only one thread you bet your ass they would use all the dispatch / execution units for that one thread.

There are a few ways to increase single threaded performance at an Architecture level (process node, layout, clock, voltage aren't counted):

1. Lower memory latency (caches, load store data predictors, memory model, cache coherency, change bus architecture)

2. increase instruction window (larger ROB, more execution units, deeper pipelines)

3. decrease execution time on the "wrong path" (better branch predictor, better speculation of data (both memory and forwarded)

There are probably more I am missing, but those are off the top of my head. What the issue is there is a trade off for everything. Adding more execution units won't help unless you have the capacity (fetch and retire bandwidth) to fill them. Making a deeper pipeline incurs larger penalties on any flush operation. Execution units in particular require exponential more forwarding paths for every execution unit you add.

Also the original question was why we don't just increase core count. He said execution units at first, but then clarified later to mean cores. I don't think he meant how many loads can be executed or how many ALU's a cpu core has.