Physics wouldn't have progressed much if all research results had to be immediately usable by kids playing with Lego blocks. Most so called "programmers" are just fitting blocks together without any deep understanding. Just look at all the horrendous VB code out there.

C, for example, should never come within a mile of Joe Average Programmer. But the world needs C, it needs the Linux kernel, etc, etc. For other things, this is not so clear.

You have to be able to judge the prospects of research programs. Otherwise, how do you separate the dry holes from the promising ones? There is an infinite set of dry holes in the world, and any of them is happy to accept all the funding you can give it.

This still misses the point. C was "invented" well before the Linux Kernel or GCC was written in it. At the point of invention, it was not clear that C would become the roaring success it turned out to be. The utility (or otherwise) becomes obvious only after (sometimes long after) something is invented/discovered. IOW, no one knows whether an idea is a "dry hole" or not, until after it has been explored (though you can make some, usually inaccurate guesses more on this below). The history of Physics (for e.g) is full of discoveries that became usable centuries after the discovery.

Coming back to the C language, your article seems to be saying that the inventors of C need to justify it in terms of an yet to be conceived of (leave alone implemented) Linux kernel, before writing the code for the language compiler.

From wikipedia, (http://en.wikipedia.org/wiki/C_(programming_language)

"# The development of Unix was the result of programmers' desire to play the Space Travel computer game.[1] They had been playing it on their company's mainframe, but as it was underpowered and had to support about 100 users, Thompson and Ritchie found they did not have sufficient control over the spaceship to avoid collisions with the wandering space rocks. This led to the decision to port the game to an idle PDP-7 in the office. As this machine lacked an operating system, the two set out to develop one, based on several ideas from colleagues. Eventually it was decided to port the operating system to the office's PDP-11, but faced with the daunting task of translating a large body of custom-written assembly language code, the programmers began considering using a portable, high-level language so that the OS could be ported easily from one computer to another. They looked at using B, but it lacked functionality to take advantage of some of the PDP-11's advanced features. This led to the development of an early version of the C programming language."

Following your "dry hole" logic , at the point of invention, C would be a very dry hole. I can just see Kernighan and Richie "justifying" their new language by saying "umm so we want to play Space War"... :-)

"You have to be able to judge the prospects of research programs."

True. To a certain extent.

The point of research is that it is an exploration into the unknown. The unknown is, by definition unknowable. When allocation of finite resources (like money) is involved, researchers who expect to consume those finite resources (in the form of grants for e.g) expect grants should try to elucidate the benefits of their proposed research. And the people in charge of allocating those resources should try to examine those claims and place their bets wisely. So far, so good.

The fact remains though that this is all guesswork. No one knows what the outcome of a proposed line of research will be. Think of it as funding Colombus's expedition. (see http://en.wikipedia.org/wiki/Christopher_Columbus particularly the sections on background and "funding campaign"). Both Colombus and his funders had several "best guess" judgments about his "prospects" but they all turned out to be wrong. His voyage was revolutionary nonetheless.

"For other things, this is not so clear."

This is just the nature of exploring the unknown. Which is what research is all about. You do your thinking, allocate your resources and try like hell, knowing all the while that it could be a colossal waste of resources. Just the nature of the beast.

Now, if your point is that there are people who game the system, that is a valid point. But then people game every system. So ...

The problem is that the actual process by which this guesswork is done, actually right now in the real world, is nonsense. It is permeated by intentional distortions of reality. As a result, in my opinion, research is pointed in directions which a reasonable observer would guess are dry holes. The fundamental difficulty is that Congress is not a reasonable observer - it is a political one.

This is inextricable from the phenomenon of official research funding. I suspect that private philanthropic funding - the way it worked before WWII, a period in which there was no computer science, but quite a lot of other science - would do a much better job of "guesswork."

It is actually possible to make educated guesses as to what's interesting and what isn't. It is just very hard.

The management of the old Xerox PARC, for example, was very good at it. So were the people at DARPA - about 40 years ago. So, once, was Bell Labs. And so on.

Needless to say, the actual process of guesswork has nothing in common with the modern scientific budget process.

I'm willing to accept that you are more than capable than predicting what's interesting. Let's try a zero knowledge proof. You point out the interesting ones, and we'll check back in a few years to see your accuracy.

The reason outsourcing is used is because of a favorable exchange rate due to historical inertia, even as we nudge it closer to the midway point every time we engage other countries. Going by population alone, their complement of great minds surely outnumbers ours.