Given DARPA’s track record, I’ve often wondered why there wasn’t a domestic version. Apparently Steven Chu has been wondering about the same thing.
Fortunately, Chu’s wondering is more potent than mine; in 2006 he worked on an NAS project that proposed such an agency, and now that he’s Secretary of Energy DoE is rolling out ARPA-e, the Advanced Research Projects Agency – Energy, using some stimulus money to pay for it.
Early projects:  bacteria that will make gasoline, enzymes that will capture carbon dioxide to counter global warming, and batteries so cheap that they will allow the use of solar power all night long.
Of course, the whole point of such an agency is to fund a bunch of low-probability, high-impact bets. Â That means that most of the money spent by ARPA-e will be “wasted.” Â Fortunately, William Proxmire is no longer around handing out “Golden Fleece” awards to research projects he didn’t understand (one went to the paper that discovered the freon/ozone problem), but ARPA-E will be lucky if it escapes similar heckling from the current generation of obscurantist demagogues.
Now, how about starting something like this in the Department of Education? Â There’s a field that could use some out-of-the-box thinking.
It will be interesting to see how this works. The DARPA model isn't just about giving a fair amount of money to far-out stuff (and in fact DARPA also does a lot of fairly ordinary R and even D). It's also about giving program managers and principal investigators a fair amount of autonomy and encouraging researchers to take career risks. (This can have highly-unintended consequences, e.g. the founding of one of Wall Street's earlier super-quant firms after DARPA dumped a bunch of computer-architecture researchers whose work wasn't suited to what they thought their needs were.)
The other thing that will be interesting will be the infusion of nonmilitary money (which may mean fewer clearances and other cultural changes) into some subfields. One hopes main result won't be turf battles.
This "intense debate" seems to have lost its intensity. I ran into this space in the bloggosphere by looking for information on whether ARPA-E's enabling legislation has a sunset clause. It should, and this crowd-following organization should be allowed to sink quickly into the west. Perhaps someone would like to defend the claim by Paul: " encouraging researchers to take career risks."
Chu took the risk out of taking money for "R&D" with his priestly claim that only R&D can solve our problems. Not so. What we need to solve our problems is guts.
Now (fall 2014 FOA) ARPA-E is going to throw money at wannabee ideas for fusion power? The reason we do not already have operating fusion power systems is because of just this mentality. The "under the streetlight" syndrome has protected the hegemony of the military funding of inertial fusion, which is the real deal that is in the scary dark alley away from the streetlight.
ARPA-E is following the "build it and they will come" mentality of fund whatever will make black white and up down: biomass? the ethanol disaster is not enough proof? fund GE to develop windmill blades? the government needs to fund GE to tweak their windmills as well as subsidize their purchase?
"encouraging researchers to take career risks." You have got to be kidding. ARPA-E is about as much into taking risks as today's Wall Street.
Something like this in the Dept of Ed: There is FIPSE, which at times has funded very innovative stuff but has been at risk of being turned into an earmark shop — a move pioneered by a recent Pennsylvnia convert to the Democratic Party. See:
http://www.eric-goldscheider.com/id75.html
Absolutely, it would be interesting see more of this in education.
Dan Tompkins
I think we learn something from the case of Proxmire (just as we learn about valid inference by considering invalid arguments such as the ontological proof of the existence of God). Ridicule of low probability high payoff projects certainly reflects a simple failure of imagination, but I think it also reflects the damage caused by loss aversion. Spending millions for nothing absolutely nothing is an appalling prospect. It's only a loss of millions and if there were a 1% chance of gaining billions it would be a great investment for a deep pockets entity like the US government.
The huge wealth of venture capitalists shows just how gigantic a profit opportunity was created by the fear of total failue which is only twice as bad as 50% failure.
Attempting to link the comment thread on innovative approaches to global warming and the comment on "play nice" about Martin Luther stinking to high heaven, I note that one geoengineering proposal is to send Hydrogen Sulfide up 18 mile tubes to the stratosphere sending a stench up to high heavan. Currently disfavored by the superfreakeconomists who favor S02, but a surprising link between Luther and Myvrhold.
Speaking of independent investigators taking career risks spending public money on apparently pure research — NIH. The NIH has a huge gigantic budget for investigator initiated peer reviewed grants applications. Maybe the best shot would be to argue that better schools would reduce teenage smoking or something, and try to sell it to the National Cancer Institute.
Robert W: "The NIH has a huge gigantic budget for investigator initiated peer reviewed grants applications." Isn't peer review part of the problem here? It's an inherently conservative method. I thought DARPA and (some) Silicon Valley venture capitalists succeed by making bets on individuals whose ideas would not pass a conventional wisdom screen.
James Wimberley is spot on regarding the problem of peer review. It's an excellent method for sorting out high-quality projects in established fields, but it tends to screen out projects that fall outside the areas of expertise of the reviewers. Unfortunately, ARPA-E adopted the peer review model for reviewing its initial funding announcement. Two panels of experts reviewed proposals that made it beyond the abstract phase, and if you read the list of funded projects, none of them are truly new ideas; the majority are incremental improvements on technologies such as solar cells, thermoelectrics, and wind turbines. DARPA's success is due in large part to its venture capital-like structure, so it's somewhat sad that ARPA-E was so excited about their two rounds of peer review.
Good comments on peer review limiting "out of the box" R&D, and the federal support for fusion energy using the "star in a magnetic bottle" approach for over 50 yrs is a counter example to a properly implemented ARPA-E concept. I remember reading about the Stellarator and Tokamaks when I was in high school in the late 50s and since that time I would guess the US has poured 10s of billions down that hole with little to show. Unfortunately, the plasma physics lobby has dominated the high pay-off government energy research and suppressed any rational investment in "cold fusion" or more accurately Low Energy Nuclear Reaction (LENR). Given the hostility of attacks from the plasma fusion lobby on the legitimate science needed after the unfortunate premature publicity given the Fleishman-Pons experiments, there is no way the peer review process would have allocated a fraction of the resources justified on follow-up LENR work. Now we have the case where the major breakthrough could come from Japan, Italy or Israel because of the suppressive effect of one special interest group in the US defending their right to a billion dollar stream of rent from the taxpayers. An as yet unmentioned ground rule of the original ARPA concept would have prevented this–a sunset provision of about 5 years on the "high risk, high pay-off" R&D.