I've done research in writing tools to make parallel machines easier to use for non-(computer specialist)s. The consensus is still, largely, that you can't do it. There are special cases, and there is library code, but for most people, they haven't a clue how to write code that can run reliably on multiple processor systems.
I think almost everyone involved in the industry knows that we want faster uni-processors, and that we don't want multi-core machines. We're pretty sure that some problems that do have fast serial algorithms don't have efficient parallel algorithms (this is akin to P vs NP type questions, and is beyond my current knowledge) and these problems are stalled at uni-processor speeds.
Research continues, progress is made, and most people don't care because most people just write emails, use facebook and browse the web. Heavy computing with parallel or multi-processor machines is hard in general, and that's not changing fast.
I don't expect it to. People will simply have to learn how to devise parallel algorithms, and write good, parallel code.
I'm not so sure that multicore programming will be the future.
In the last couple of years, we have seen the rise of the netbook, the smartphone, and languages like Python and Ruby. Each of these are, apparently, "fast enough" to be highly succesful.
Applications are also changing: casual games, Farmville and even World of Warcraft have very modest system requirements but make a ton of money. Web applications are awful from a performance perspective: they are crippled by network latency and need a single server to do the work for many people. Still, they are "the modern way" to write an application. Again, the performance is "good enough".
Finally, even when performance is needed, there are different models for parallel programming than multicore. GPUs are very fast data-parallel "single core"-like devices. At the other end of the spectrum, real-world "web scale" systems scale horizontally and are thereby forced to adopt a multi-process model.
Yes, our OSes should be written for multicore, the newest Doom/Quake should be written for multicore, and our numerical models should be at least aware of multicore; but despite the fact that the C programmer in me will likely be happier in your multicore world than in the world I sketch above, I think most programmers will live in the latter.
GPUs are very fast data-parallel "single core"-like devices.
I don't think that's accurate - being "single core"-like. Besides literally having multiple execution cores, GPUs are radically data-parallel in ways that even those already familiar with data-parallel code (using languages such as OpenMP) must still adapt their thinking. Code that extracts high performance out of GPUs must be aware of the memory hierarchy and degree of parallelism - you can't pretend you're doing sequential programming.
Keep in mind Moore's law as originally stated was a law of economics, not engineering -- that decreasing feature size means more dice and thus more revenue per wafer, while also increasing the performance of the individual ICs.
These days I think energy usage is the overriding concern, both because of thermal limits in high performance systems, and energy budgets in low-power systems. A smartphone which is fast enough to play Farmville or run a python interpreter is cool and sells well, but there is still a huge economic incentive to cut energy usage further, since that would increase battery life.
The core of your argument seems to be that since things like netbooks, smartphones, Python and Ruby are currently "fast enough", there is little incentive to increase the core count. I disagree with that, because energy savings will (or would) still add massive economic value. Whether these kinds of systems will become reality is a different question, since of course programming them is a very hard problem.
But how cool would it be to have a smartphone with 100 cores, each running at a few MHz or so (and apps which use them efficiently) and battery life of a month or so? Also, think of how motivated companies with datacenters are to reduce energy costs.
In the last couple of years, we have seen the rise of the netbook, the smartphone, and languages like Python and Ruby. Each of these are, apparently, "fast enough" to be highly succesful.
For consumer purposes. However, a ginormous amount of the software in operation today is not aimed at consumers at all, and don't resemble Farmville in the slightest.
Most programmers are writing COBOL (even if they are using a slightly more modern language to do it in.)
I'm not sure I fully understand you, but I'll try to answer: yes, "enterprises" don't run their business off a smartphone, but if they move to (internal) web applications or don't spend the extra funds to make their CRUD desktop applications even faster, they don't use, or need, the multi-core parallel programming model either.
If the above is not an answer to what you meant, could you please clarify?
I mean that these "enterprise" applications (which are not CRUD desktop applications, and are definitely not moving to web applications in my lifetime) run on "big iron", which is multi-processor-based-- and that there will be lots of people working hard to make these platforms more parallel-friendly in the future.
I should add that it was 1990 that I was doing this, and although some things have improved, I think the overall picture hasn't changed a lot. Threads weren't common back then, and are now, as people realise they need to be able to run things in parallel.
The tool I have now allows people to write multi-process programs and have them communicate effectively. I'm still trying to get permission to release the system as open source, or at least publish the ideas, but the industry is notoriously secretive and conservative, so it'll be some time yet.
Essentially he was explaining the physics behind the collapse of Moore’s Law and the rise of many-core systems.
Moore's Law still holds; it is about the number of transistors on a chip, not processor frequency. The problem is that the way that we achieved better performance for a long time was to increase sequential performance, which meant: higher clock frequency, larger caches, longer and more elaborate instruction pipelines to extract instruction level parallelism. That's what is breaking down, not Moore's Law.
That we're now exploring multicore design is because of Moore's Law: our old approaches for improving performance no longer work, but we're still getting more transistors to work with.
I'm a little confused about what point the article is trying to make. I don't think that the many-core folks have ever said that they're building many-core because people want it; I've consistently heard that it's because of Moore's Law collapse, as admitted in the article.
If it were economical to produce faster uniprocessors instead, you can bet that chip companies would do it, because they'd make more money that way.
Reminding us that parallelism is not our original goal is just the prelude to casting doubt on the inevitability of the breakdown of Moore's law for unicore processing. That's how I interpret the article.
I think a major problem there are many parallel-programming models , which either do not make big news outside the academic world, or are deemed not suitable for industry(CUDA folks- not talking about you!).
Like Matt Welsh(the Harvard prof who left for Google) mentioned on his blog recently.. how many people risk running OpenMP in production, and watch some nodes fail..
Even the "Big" guys (Google,Facebook) seem to be risk-aversed to thread-level parallelism to earn their bread -BigTable,Hadoop,Cassandra are centered on multi- process architecture.. and basically built for horizontal scaling- add more nodes,assume they fail.
Where is this push towards parallel machines coming from? Not really from consumer web - Or any consumer oriented service that an entrepreneur can gain value, by utilizing multi-processor systems.
It's the organizations with big bucks that call the shots, or drive innovation.
Like NCEP/NOAA in weather prediction. Not the guy who moves to the Bay area who wants to hack something, get traction in user base, get funding,and finally mint money :-)
The demand for writing consumer facing software for multi-processor systems is simply too less now.
I think almost everyone involved in the industry knows that we want faster uni-processors, and that we don't want multi-core machines. We're pretty sure that some problems that do have fast serial algorithms don't have efficient parallel algorithms (this is akin to P vs NP type questions, and is beyond my current knowledge) and these problems are stalled at uni-processor speeds.
Research continues, progress is made, and most people don't care because most people just write emails, use facebook and browse the web. Heavy computing with parallel or multi-processor machines is hard in general, and that's not changing fast.
I don't expect it to. People will simply have to learn how to devise parallel algorithms, and write good, parallel code.