Why are research papers on things that have no practical benefit for society funded by the government? This topic came up in reference to how an ideal government should function.

Ideal government. Ideal for whom? One person's meat is famously another person's poison.

Averaging <5%, R&D is a minuscule part of government expenditure
It's misleading to debate government spending on research in general, and on so-called 'frivolous' research in particular without considering the total amount of government spending. Governments spend a lot of money on lots of things. What proportion is on R&D? Minuscule. That's as true in the US (2.85% of 2011 GDP spent on R&D) as it is in India (0.76% of 2007 GDP spent on R&D) (see table below), where the already meager amount the government spends on R&D is only decreasing, not increasing. By and large, governments, especially the larger, wealthier ones tend to spend astronomical amounts on stockpiling weapons and munitions even when millions of their citizens don't get enough to eat. Scope and scale of arguably frivolous public funded R&D literally fades into insignificance when examined from this perspective.

 
As for everyone having the basic necessities, is there even one country to have succeeded in this endeavor? With its share of the poor and homeless, even the wealthiest and most powerful country in the world, the US, is unable to ensure that every one of its citizens has the basic necessities. Should its government should stop funding basic R&D? That's tortured logic or more accurately, torturing logic.

Practical benefits of science are typically unforeseeable
How to assess the value of government-funded research? Is it only that which yields immediate practical use? We'd then have to include German Nobel prize winner, Fritz Haber, whose weaponization of chlorine gas was of obvious immediate practical benefit to his country's government during the First World War. Rather than laudable, the point of this inglorious and downright shameful example is to highlight the folly of focusing exclusively on foreseeable practical benefits of public funded R&D.

History also teaches that many things of immediate practical benefit turn out be long-term banes. Developed accidentally by Allied industrial chemists trying to help their side win the Second World War, the self-evident practical benefits of plastics such as polyethylene meant that the post-WWII human took to using it in unprecedented numbers and in arguably all aspects of life. Emblematic of long-term bane, toxic chemicals such as BPA (bisphenol-A) and phytoestrogens leaching from plastics cause long-term damage to the health of not just humans but other life forms as well while plastic refuse is quite literally choking ocean life itself to death as a vast toxic plastic landmass builds on the Pacific Ocean.

Thus immediate, foreseeable practical outcome alone is an inadequate benchmark to judge the value of public funded R&D. OTOH, public funded R&D has several, largely invisible long-term benefits to society. At the very least, it helps establish research infrastructure and culture, i.e., a scientific temper among its populace. An invaluable common good, can't put a price on scientific temper. It also provides stable, gainful, long-term employment to many. Even if their scientific output is middling, their consumption helps the country's economy. Educated people are also more likely to invest in their children's education. Predicated on the way in which social networks operate, this in turn further helps embed scientific temper within the larger society.

Another self-evident truth to consider is that smart people are unlikely to waste their talent and time on research projects that they find unfulfilling. If public funded R&D is indeed disproportionately spent on fruitless fishing expeditions, it's more likely to be the output of careerist scientists, and as we explored already, there's value to be mined from their ranks as well.

As for highly abstract notions with no foreseeable practical benefit, what could be more emblematic of that than Quantum mechanics? Its calling card, casual defiance of Newtonian mechanics, the very stuff of woolly-headed dreaminess. And yet from lasers and transistors to cancer diagnostics (MRI; Magnetic Resonance Imaging), a long line of practically unforeseen practical benefits have ensued from it over the past century.

Since innovation isn't a command function, the wild card of serendipity weaves the necessary magic
Practicalities aside, is innovation even a command function? Even a cursory look at the history of science leads to incontrovertibly conclude no. Rather, innovation through the ages shares the common thread of serendipity.

According to Royston M. Roberts, after reading the fairy tale adventures of 'The Three Princes of Serendip' (i.e., Serendib or Ceylon or Sri Lanka), who 'were always making discoveries, by accidents and sagacity, of things which they were not in quest of...', Horace Walpole coined this word in a letter to his friend Sir Horace Mann, 1st Baronet in 1754 to explain the basis of his own accidental discoveries (Amazon.com: Serendipity: Accidental Discoveries in Science (9780471602033): Royston M. Roberts: Books). The list of serendipitous discoveries is legendary, legion and grows by the day. From X-rays and Quinine to Penicillin and Velcro to Oral Contraceptives and Viagra.

However, serendipity is not synonymous with accident. As Pasteur famously said, 'chance favors only the prepared mind'. Thus, innovation can't be ordered into existence nor can one stumble into it by accident. Yet many of the most important discoveries were the product of serendipity, which is accidental discovery but not accident. What gives? Insatiable curiosity, and breadth and depth of knowledge certainly seem to be the common mainstays among these famous examples. What else?

When Carl Djerassi was hired in 1949 to lead a research team at Syntex, 'an oral contraceptive was not in anyone's plans' (Amazon.com: Serendipity: Accidental Discoveries in Science (9780471602033): Royston M. Roberts: Books). An ancillary goal of his team was to synthesize a molecule that could mimic the biological effect of estradiol, a female hormone used for treating puberty- and menopause-related disorders. Instead they accidentally ended up synthesizing one that mimicked progesterone. Progesterone, of course, is 'nature's contraceptive' since it inhibits ovulation during pregnancy. Thus, the first oral contraceptive resulted from serendipity, not from a utilitarian command directive from management on high. Freeing womanhood, indeed society itself, from the drudgery and curse of unplanned pregnancy, it can be argued that woman's liberation itself may not have been possible without the serendipitous oral contraceptive.

Djerassi's team's goal was synthetic estradiol. Rigidly sticking to their programmatic goal would have dictated throwing the artificial progesterone in the trash. A prepared mind means to be able to accurately appraise the value of accidental discoveries, as Dejrassi's team did. This isn't something programmable by a rationally designed utilitarian system. Rather it's inculcated within a larger cultural framework that values curiosity and the capacity to synthesize knowledge from information, and that imbues an intuitive sense of value.

I recently read an article on Rajaram Bhapkar. Much like the indomitable Dashrath Manjhi, made famous in a recent Bollywood film, who connected his village with the outside world through his own backbreaking labor, Bhapkar built roads in the craggy hillsides surrounding his village on his own initiative. The astonishing innovation? 'On a particularly challenging downhill road from Khandoba mandir, Bhapkar asked a few engineers for help. When they refused, he turned to two donkeys. He filled two bags with gravel and pierced holes in them, tied them to the donkeys and sent them downhill. He then constructed the road on the trail left by the gravel. “Now, structural engineers visit and tell me it is a marvel and ask who is the engineer behind it. I point to the donkeys,” said Bhapkar, with a laugh (Mountain man and donkeys). That's ingenuity of the highest order. Not a degree from Harvard or MIT or IIT but sheer undeniable native ingenuity.

As long as human societies continue to throw up the Rajaram Bhapkars in our midst, innovations will remain reliable engines. That's a takeaway for sure. Role of governments? Innovation not being a command function, it would suffice if governments are able to rein in the addictive and hypnotic hold of political expedience and power long enough to engender a culture that stably values and upholds scientific temper within its midst, come what may.



https://www.quora.com/Why-are-research-papers-on-things-that-have-no-practical-benefit-for-society-funded-by-the-government/answer/Tirumalai-Kamala