GPU based OS/Kernel

[btoews]

I've been researching the efficacy of using GPUs in medical imaging; from what I've found of the GPU there is very little the CPU can do better. Anyone think it would be possible -- with a complete redesign of how the kernel/OS currently works -- to do away with the CPU? There are always many processes in the background (i.e. parallelization), thus it seems that the CPU as it is now could be made obsolete to GPGPUs.

[EvanED]

The GPU is very good at highly parallel tasks, but the CPU is better at serial ones. Some tasks are fundamentally (we guess) not parallizable. (See, e.g., here: "It is unknown whether NC = P, but most researchers suspect this to be false, meaning that there are probably some tractable problems which are "inherently sequential" and cannot significantly be sped up by using parallelism")

[btoews]

While true there are tasks which cannot be made parallel, it seems to me that in most situations, the OS would not encounter many of those programs -- even if it were to, the processes would likely not be overly intensive -- and there would be little performance drop in allowing those processing to run on a single GPU core while leaving the many other cores available for other tasks. The only time when one is waiting for their computer is loading programs (where computing power is not your bottleneck), gaming/simulations (which are generally parallelizable), and when there are many things that the computer has to do at once (parallel processing again, assuming RAM isn't a problem ).

Or, an alternative to the original proposition, the CPU could be made secondary to the GPU and the OS utilizes it when a massive process that is not parallel wants computing time.

[korona]

How fast are branches on a state-of-the-art GPU?
There are many applications that rely on heavy use of branches.
How fast is random access to on-chip memory?

[btoews]

That would be one thing I forgot to consider: the Single-instruction, multiple thread nature of GPUs. This combined with memory access performance problems would be two hurdles to overcome. Is it possible to hypothetically deal with these problems without a complete redesign of GPUs?

[EvanED]

even if it were to, the processes would likely not be overly intensive -- and there would be little performance drop in allowing those processing to run on a single GPU core while leaving the many other cores available for other tasks.

Just like GPUs are way more powerful at the tasks they're good at, they're way less powerful at the tasks they're bad at. It wouldn't be a "little" performance drop, it would be a massive performance drop for those tasks.

The only time when one is waiting for their computer is loading programs (where computing power is not your bottleneck), gaming/simulations (which are generally parallelizable), and when there are many things that the computer has to do at once (parallel processing again, assuming RAM isn't a problem ).

Also: compiling, rendering, running poorly-optimized web browsers (and programs), scientific computing and other research experimentation, and lots of other things.

And for instance, compiler optimization is one of those kinds tasks that parallelizes really poorly. Heck, I'm not even sure you could effectively parallelize type checking in a language like C++ where that can take a while.

Now, that's not to say that the idea is completely half-baked; consider the cell processor. It's moving toward that picture. (Actually one more like the last thing you said, as there's one more powerful core.)

That would be one thing I forgot to consider: the Single-instruction, multiple thread nature of GPUs. This combined with memory access performance problems would be two hurdles to overcome. Is it possible to hypothetically deal with these problems without a complete redesign of GPUs?

So I don't think GPUs nowadays are really SIMD now that we've got shaders. But my impression is that the reason GPUs are so fast is exactly because of the assumptions about how memory works and stuff that makes those things slow. So I would say no. And if you did damage the actual GPU performance, the next thing you'd see is nVidia come along and say "let's make a processor that's better at graphics by exploiting all these assumptions about what you need for realtime rendering"

But I'm not an architecture person, so take what I say with a grain of salt.

[Carnildo]

I've been researching the efficacy of using GPUs in medical imaging; from what I've found of the GPU there is very little the CPU can do better. Anyone think it would be possible -- with a complete redesign of how the kernel/OS currently works -- to do away with the CPU? There are always many processes in the background (i.e. parallelization), thus it seems that the CPU as it is now could be made obsolete to GPGPUs.

It sounds like you've re-invented the vector supercomputer.

[nilspin]

Well check out the KGPU project.
http://gpgpu.org/2011/05/04/kgpu-gpu-computing-in-linux-kernel
http://code.google.com/p/kgpu/

[jareds]

Well check out the KGPU project.
http://gpgpu.org/2011/05/04/kgpu-gpu-co ... nux-kernel
http://code.google.com/p/kgpu/

Some tasks are better on CPUs, other tasks are better on GPUs. Many types of cryptography are better on GPUs. The Linux kernel does cryptography. For technical reasons, it could only use the CPU. This changes that.

It does not mean that getting rid of the CPU and/or running the bulk of the OS on the GPU would improve performance. It would not, as people have explained.

[troyp]

As a related question, how specific is GPU architecture to handling graphics? Given that more and more highly parallelizable tasks are being computed on GPUs, would we be better off (in the future) with more generic "Parallel Processing Units"? Or are GPUs already efficient at more general tasks?

[EvanED]

As a related question, how specific is GPU architecture to handling graphics? Given that more and more highly parallelizable tasks are being computed on GPUs, would we be better off (in the future) with more generic "Parallel Processing Units"? Or are GPUs already efficient at more general tasks?

So older GPUs were pretty specific to graphics. You could hijack them for other things with shader programming, but getting them to do GPGPU stuff was... interesting. (I never head to do any of this, but my impression is you basically had to encode the answer to whatever you were asking into the graphics it was outputting, then capture the current frame. Modern ones (last 4 years? that's a bit of a guess) are more general, even being designed with an eye toward GPGPU stuff.

However, that doesn't mean they're good at everything. You need a task that can be nicely farmed out to the different GPU cores in such a way that they can mostly work on their own independent portions of memory; you may even have to have a SIMD-friendly task.

So the answer I think is that to some extent we're there already. There's probably some more things that could be done to make the GPU more friendly for general purpose tasks, but my impression as an outsider to the field is that pushing things too much further will start to degrade their performance at the things they're really good at now, which probably isn't worth it.

[KnightExemplar]

As a related question, how specific is GPU architecture to handling graphics? Given that more and more highly parallelizable tasks are being computed on GPUs, would we be better off (in the future) with more generic "Parallel Processing Units"? Or are GPUs already efficient at more general tasks?

Its a matter of cost / performance tradeoffs. As the GPU becomes more programmable, it will either get slower or more expensive. Obivously, we all would like to have $2000 (yes, two thousand) Nvidia Teslas as our GPU, but consumers just can't afford that kind of stuff yet. The less programmable $200 GeForce 560Ti is more than suitable for most people, and even beats the Tesla in gaming performance.

So I ask you: would you rather get a generic GPGPU processor like the $2000 Tesla. Or would you rather get the faster and more efficient $200 GeForce 560Ti? I guess if you are building a supercomputer and you need that fast double-precision GPGPU functionality for analyzing weather patterns or folding proteins, the Tesla is better. But for most consumers, the card designed for gaming is the better pick.

All of the major manufacturers are working towards GPU computing. It is definitely an exciting time for High Performance Computing (HPC). Even Intel is comming out with something in this space (which has been typically dominated by AMD/ATI and Nvidia). And even the 560Ti gets decent performance with CUDA, so its not like you can't accelerate your stuff with a consumer video card.

-------

Day old EDIT:
I found this interesting article which introduces you to OpenCL computing on NVidia. The SIMT architecture really shows how conditional statements really mess up the parallelism on NVidia processors.

Normally, a GPU executes 32-threads with one processor (mine actually does 48-threads). Whenever a thread disagrees with the other threads on a branch, that thread is disabled and is executed at a later time. So if you have 31 threads take one branch, while the last thread takes another branch... the whole computation may take twice as long to finish. When you have two branches in a row, it may take four passes in the worst case.

Basically, code with conditionals is best executed serially. On the other hand, few conditionals and a large amount of parallelism make for a better GPU problem.