The puzzle of increased spending on science and R&D in advanced economies and falling rates of economic growth in the 21 first century

There is no mechanistic connection between scientific and technological progress and economic growth in the short and medium term and everything turns on the institutional structures of economies.

Over the last sixty years there has been a secular decline in the rate of economic growth in mature advanced economies. Whether the data is looked at by decade or by economic cycle growth in the rate of GDP has fallen. In the 21 century there have been two massive adverse shocks, the Great Financial Crisis and the Great Recession that followed it and then the covid public health emergency. While economies have weathered these massive shocks well, they have returned to growth but probably at a slower rate than before 2007, which itself was a period marked by disappointing productivity and GDP growth. This marked deceleration in the rate of economic growth is illustrated by a chart published in the US Economic Report of the President in 2023 showing GDP growth from 1790.

Average Annual U.S. Real GDP Growth since 1790, by Decade and Contributor

The US economy in recent decades has grown faster than other advanced economies, but still as this chart shows has exhibited a marked deceleration in the pace of trend economic growth.

Governments and opposition politicians across advanced economies are anxious to reverse this secular trend of slowing growth and have become evangelists for technology innovation and science to raise productivity and output. This desire to support scientific and technological innovation has become conflated with the green agenda of de-carbonisation, with technical innovation in energy production being acclaimed as exciting green growth opportunity. There are good reasons to be sceptical about policy makers’ claims in relation to science and technical progress and GDP growth in anything other than the very long-run and for all practical purposes outside the planning horizon of modern democratic governments. 

The rationale for public policy support of science and R&D

This is not to suggest in any way that policy makers should be put off from spending money on science or supporting research, given that in many respects it is a merit good deserving support that conventional market prices and processes may generate. Although some economists would point out that given imperfect markets and oligopoly competition where corporations use R&D as a form of non-price competition, the amount of corporate R&D may not be as suboptimal as policy makers sometimes think. This note does not criticise public support for science and technology but simply offers a reminder that there is not a swift mechanical causal relationship between science and technology spending and economic growth.

The IMF in a blog Why Basic Science Matters for Economic Growth, published in 2021, emphasising the importance of R&D and science in GDP growth, particularly for emerging market economies, notes that ‘surprisingly, productivity growth has been declining for decades in advanced economies despite steady increases in research and development (R&D), a proxy for innovation effort.

Spillovers are important for emerging markets and developing economies

Source IMF

The US Government’s long involvement with promoting science

The involvement of governments with science and technology to promote military and commercial objects has a long history. The President of the National Academy of Sciences (NAS), Marcia McNutt, for example, a lecture For the People: The Role of Science at Harvard reminded her audience that in the US government involvement with science began in the 19^th century. Indeed, the NAS was established by Abraham Lincoln in 1863. McNutt explained how it came about as a result of the American Civil War. ‘After witnessing two ironclad warships battle to a draw because “cannonballs just bounced off the sides of these ships,” Lincoln is said to have proclaimed he wanted scientists on his side.’ President Lincoln also established the Land Grant Colleges that have gone on to play a significant part in American research because he believed in the dark days of the conflict that people had to be given hope and something to look forward to. In the modern world governments in advanced economies support science, technology, and innovation in a range of spending on research, support for R&D and with tax expenditures such as tax reliefs as well as up-front cash in the form of tax credits for R&D.

UK spending of R&D as a ratio of GDP

The UK House of Commons published a research paper on Research and Development in 2023 that marshals interesting information on UK and international comparisons spending on R&D. The UK Government now estimates that R&D accounted for between 2.9 percent and 3.0 percent of GDP in 2020, with the OECD provisionally estimating the ratio of spending in that year at 2.93 per cent. In scoring and estimating R&D there have been revisions made to the methodology to improve it. The old methodology suggested R&D expenditure in the UK increased by 96 per cent in real terms between 1985 and 2019. Since 2017 the UK Government has a target to increase total R&D expenditure to 2.4 per cent of GDP by 2027. The research paper notes that ‘following the change in how R&D funding is measured it appears this target has been exceeded with the government estimating that R&D spending was equivalent to between 2.9 percent and 3 per cent of GDP in 2020’. The paper has charts and tables illustrating the rising trend in R&D spending in the UK and its composition.

In 2020 £12.8 billion was spent by the public and private sectors in the UK. The public sector accounted for £3 billion of this total and the private sector accounted for £3.8 billion of it. The largest source of spending was in higher education that spent £6 billion.

Changes made by ONS to the methodology for estimating UK R&D spending in 2022 that raise it

The House of Commons research paper provides a concise and useful explanation of the changed UK methodology used for measuring R&D in GDP. In 2022, the ONS changed its methods for producing GERD statistics. This ‘markedly’ increased the figures, for example the overall spending estimate for 2019 increased by 55 per cent. ‘There are two main changes: A change to improve the extent to which R&D by small businesses is counted.  A change to improve the estimate of R&D performed by the higher education sector – adding in R&D that is both funded and performed by the sector. The ONS says that more improvements will be made to estimates for business over the next couple of years but that it expects the changes in the estimates published in November 2022 and covering 2018 to 2020 to be the most substantial The changes to overall estimates for business have been welcomed by the statistics regulator which said it had been provided with “sufficient reassurance” that the “new method has produced the better estimates of total UK” business R&D’. 

The research paper notes ‘An editorial in the journal Nature suggested that the ONS could have taken more time, and consulted more widely, on the changes’. The ONS data on spending by businesses was increased to match HMRC data on tax credits paid to businesses for R&D. The House of Commons’ research paper noted that ‘But that some businesses claim these credits fraudulently, so estimates of R&D spending based on data on these credits may not be reliable’.

As well as setting out UK spending on R&D and its components the House of Commons paper helpfully draws on OECD data to make illustrative international comparisons and draws on EU Commission research to illustrate the principal international firms involved in R&D.

There is huge interest in science and R&D among policy makers around the world. The US dominates science research with huge federal government programmes, most of the world’s principal research universities, large well-resourced private research foundations and its corporations lead the world in terms of international business research. The European Union has put huge efforts into science and R&D with its Horizon 2020 programme, one of the world’s largest science efforts. The Chinese Government has identified progress in science and technology as a key priority. 

Among the G7 countries, the USA spent the most on R&D 3.45 percent of GDP, Japan 3.27 per cent and Germany 3.13 percent. The OECD estimated that the UK spent 2.93 per cent of GDP on R&D which is higher than average spending in the OECD and EU. Israel and South Korea were outliers within the OECD data set the highest spending on R&D as a proportion of national income.  

OECD figures show that between 2005 and 2020 spending on R&D rose in most advanced economies both in absolute terms and as a share of GDP. In real terms spending doubled in the US, in China it rose more than four times, in the EU U it rose by 50 per cent, in Germany by 52 per cent, France by 25 percent. As a proportion of national income, it increased from 2.5 per cent to 3.45 per cent in the US, in the EU the ratio increased from 1.68 per cent to 2.19 per cent and across the OECD it rose from 2.10 to 2.67 per cent.

Economic growth slows despite increased spending on science and R&D

Yet in terms of productivity growth and growth in GDP the performance of advanced economies has been disappointing. Despite the well-rehearsed platitudes about the importance of science and R&D to economic growth it is difficult to make a clear short- or medium-term connection between them. This is partly because of the complex process of the diffusion that embeds scientific and technical progress in economies and partly because progress in science is only part of the story having the right social, legal and economic institutions that can make best use of new technology. It does not matter how much fundamental science a country’s citizens many have a purchase on and how much technical knowhow it may have, little of it will be translated into economic capacity that improves the welfare of its residents with out properly functioning institutions, markets and prices. 

The long complex processes of the diffusion of technology

The process of technical diffusion is often complex, protracted and turns on unexpected serendipity. Steam offers an example of this. 

Steam 

The steam engine was first used in Cornwall to pump water out of mines by Thomas Newcomen in 1712. For a long time it was considered only as a technology for the extraction of water from flooded mines. It was not widely used in manufacturing and industrial processes until around 1790. Its impact as a technology was transformed by its use in transport with steam locomotion powering railways after the opening of the first public railway in Liverpool in 1834. Yet even the impact of railways powered by steam was nothing compared to the impact that steam would have when complemented by a new technology of electricity. At each stage of this processor perhaps more accurately described processes of technical diffusion the potential of the innovative technology was underestimated and how it would be deployed appears with hindsight to be extraordinarily naïve. The late Robert Fogel the pioneering economist and economic historian who studied the impact of railways on the transcontinental US economy in the 19^th century points out that their early promoters expected that their main use would be as a feeder form of freight transport to canals. 

Electricity 

Michael Faraday first demonstrated the principles of electromagnetic induction in 1831. Initially while it appeared interesting Faraday’s discovery had little practical use After electric light was demonstrated in the 1870s by Edison, giving rise to the need for steam powered dynamos to generate electricity, by serendipity the steam turbine was perfected in the 1880s. Thomas Parson invented the steam turbine in 1884 transforming the efficiency of steam power. and used it to drive dynamos. The result was plentiful electricity that lowered its price. 

Once electricity generation took off in the 1880s it took over forty years for its full impact to be felt on economies. Electricity had many more uses than steam power. Among its features were the ease with which it could be transmitted over long distances and that it could be made available in ‘fractionalised’ units and the flexibility of electrically powered equipment resulted in the eventual demise of the steam engine. The steam turbine also revolutionised marine transport and naval warfare. The pace of adaptation and diffusion of the new technology partly reflected the sunk capital costs that enterprises had from previous investment in the old steam technologies. The new more efficient technology was substituted for the old technology, as capital goods came to the end of their commercial lives. In 1910 25 percent of US factories used electric power, by 1935 75 percent of factories used electivity. The adaptation to the new technology came when the old capital stock needed replacement. The process of diffusion has been demonstrated over and over again as slow, idiosyncratic. 

Complementarity, unexpected use of novel technologies and compounding effect of accumulated innovation

This process is slow, and often results from unexpected uses, which are advanced by complementarity. This process of serendipity can be seen in the diffusion of many novel technologies. Many distinguished scientists and engineers have been wrong footed by the process and fail to appreciate the significance of the scope for both new invention and the potential of recent technical innovation. Lord Kelvin is famous for having judged that at the end of the 19^th century most things had been invented and in future innovation would yield diminishing returns. Marconi never expected the radio spectrum to be used for entertainment and news media. In the 1940s few people saw their potential for widespread adoption of computers outside specialist scientific and research activities. The Harvard physicist Howard Aiken, who played a pioneering role in the development of the computer is a good example of someone never thought that computers would be used beyond esoteric scientific research involving the solution to differential equations, such as being used in the accounts and bookkeeping division of a department store. Technically they were transformed by the replacement of values and the development of transistors and integrated circuits. There are difficulties in identifying uses for new technology, its use will often turn on complementary invention, inventors often only direct a new technology at specific problem-solving functions.  New technologies such as steam, electricity and computers, moreover, induce further innovation. Large numbers of innovation have cumulative effects that compound eventually to immense overall effects.

Technology and meeting economic need at an affordable cost compared to other technologies

Whatever merits of a new technology it has to pass the basic tests of cost effectiveness. The Anglo-French collaboration of Concorde was for example a technological triumph but a financial disaster for the governments and the taxpayers financing the supersonic plane because it failed cost benefit analysis. Given the uncertainty about the course of technical innovation there are good grounds for the government to be cautious about directing it and leaving a very large role for the private sector. It is moreover important to allow the normal processes of a hard budget constraint present in the private sector to a much greater extent than in the public sector to play the critical part of killing off dud projects. The private sector has powerful incentives to terminate swiftly and without sentiment research that was once thought likely to be productive when the evidence against their ultimate use and success accumulates.

Appropriate institutions and functioning markets with proper prices, and incentives 

Scientific progress, technological development and R&D will not result in commiserate economic progress without the right social, legal and economic institutions to deploy and use it. The late Dougals North emphasised the need for appropriated economic and social institutions if innovation  is to flourish. This requires property rights, the effective enforcement of the rule of law and markets with the appropriate incentives, price structures and flexibility both to deploy new technology and to end mistakes so that resources can be moved on to new projects.

The lesson of the Soviet Command Economy 

In the years of the Cold War the USSR was one of the largest public sector largest science and technological establishments in the world, probably second only to the USA. The achievements of Soviet technology were egregious. They included the swift development of the atomic bomb in the late 1940s, the launch of the first earth orbiting Sputnik satellite in October 1957, sending the first living organism into space a – a dog named Laika in November 1957 and sending the first person into space Yuri Gagarinin 1961. Yet this hugely impressive technical and scientific progress did not translate into an economy that could raise the living standards of Soviet citizens in the decades between 1960 and 1990. A socialist command economy operating without a price mechanism and effective investments to work, save and invest efficiently could not translate technology into economic welfare.

India’s Socialist Planning and Licence Raj vitiated the effort of its scientists and technologists

India in the post-colonial period offers us a similar example of huge scientific and technical effort in the first forty years of independence that did not translate into economic growth and comparable increased economic welfare for its citizens. Pandi Nehru the Indian Republic’s first prime minister is well known as a practising barrister who studied at the London bar. It is less well known that he read the Natural Science Tripos at Cambridge and had a lifelong interest in science and technology, spending much of his leisure time as prime minister reading science papers and journals and taking an acute interest in science policy. Nehru was instrumental in supporting science and establishing his country’s All India science and engineering research institutes. Yet Nehru and his Congress Party’s commitment to Fabian socialist planning and the replication of the Soviet five-year planning model with an emphasis on heavy industry in as mercantilist economy that shunned foreign trade and external investment resulted in a controlled and regulated economy that could make full use of India’s huge science base. The potential economic benefits of India’s science, technological and engineering base was vitiated by the Congress Party’s socialist planning controls and ‘licence raj.’  

Europe, economies need to be open to trade and open to modern scientific advice and innovation

In contemporary Europe despite the huge effort put into its Horizon science programme regulation and social institutions in its countries holds back innovation. Product and labour markets are not sufficient to attract investment in the way that the US can. Part of this is the inflexibility and delays in closing businesses and investment where companies find they have to cut their losses and move on to other projects. In the Financial Times, this week there are two examples of this explored in its news pages and in an Op Ed piece. The institutions of the EU have not helped innovation. The passage of the Biotechnology Directive banning GM crops in the 1990s has closed the EU to an important new technology and a decision of the European Court of Justice has extended this ban on innovation to a block on new gene editing techniques. Horizon, moreover, has not funded genetic research into things such as unravelling the genome of wheat. The Commission at the instigation of the European Parliament resulted in the removal of the Chief Scientific Adviser and the disbanding of the whole department because they would not modify their scientific judgement about the safety of GM crops. Moreover, the EU’s mercantilist instincts in trade cause it to promote EU science and to exclude external technical innovation where it can. This emphasis on EU autarky was in evidence in 2020 and 2021 when EU emphasis on a European covid vaccine delayed vaccination across the EU during the pandemic. Openness trade is critical if economies are to reap the benefits of international scientific progress.

In the US growth in GDP and productivity has slowed as it has in all advanced economies, but economic progress has held somewhat better. The tech sector benefits from huge domestic capital markets that can provide risk capital readily, and it has flexible labour markets that can invest swiftly and abandon failure just as expeditiously. Yet there are good reasons to explore whether some aspects of the institutional structure of the US economy have moved against making the best use and opportunities of new technology. First there is a question worth exploring about the extent that the US patent system may confer monopoly rights to readily and to extensively slow the diffusion of technology. Second labour and service markets in individual states have become increasingly burdened by local economic licensing regimes that restrict trade and competition. Third, the increased use of tariffs in relation to China in particular and a mercantilist ambition of Biden to redirect America’s trade into domestic production, will raise costs and potentially slow adaptation to new technology that may well be made more cheaply in China and other Asian emerging market economies. Fourth labour market incentives have been progressively blunted by complex tax and social security transfer programmes over the last thirty years. These include: the development of the Earned Income Tax Credit; benefits that phase in and phase out over the earnings distribution that have created a Manhattan skyline of marginal tax an withdrawal or tapers, aggravated by the complexity of the health care subsidy regime of the Affordable Care Act and the extent of Medicaid apply benefit withdrawal rates further up the earnings distribution,. This complex schedule of taxes and a high rate of withdrawal of benefits has been extensively explored by Laurence Kotlikoff. 

The lesson from this international experience is that while spending by governments on science and R&D is worthwhile and represents the support of what could be considered a merit good it does translate neatly and swiftly into faster economic growth and productivity in the short or near term. Moreover, if economies are to make the most of science and innovation, they need to maintain favourable institutional structures, properly functioning markets, and frameworks of incentives that reward risk, work, saving and investment, as well as an openness to trade and international competition. In most modern advanced economies that involves awkward and search questions about the dead weight costs of public expenditure, taxation and regulation. And in this process of uncomfortable interrogation of economic institutions the USA is not immune from scrutiny.

Warwick Lightfoot 

29 February 2024

Warwick Lightfoot is an economist and was Special Adviser to the Chancellor of the Exchequer between 1989 and 1992