The 2024 AI Index Report: AI’s Influence on Society Has Never Been More Pronounced

Image: Depositphotos

Source: Irving Wladawski-Berger, CogWorld think tank member

Earlier this year, the Stanford Institute for Human-Centered Artificial Intelligence released the 2024 AI Index Report, its seventh annual edition of the impact, progress, and trends in AI. At over 500 pages and 9 chapters, the report aims to be the world’s most authoritative source for data and insights about AI in order to help develop a more thorough understanding of the rapidly advancing field of AI.

Last year’s report concluded that AI had now moved into its era of deployment, with new large-scale models being released every month. While these systems demonstrated capabilities unimagined a decade ago, they were prone to so called hallucinations, that is, they sometimes generate responses which contain false or misleading information presented as facts, as well as being prone to bias and to their use for negative purposes.

“The 2024 Index is our most comprehensive to date and arrives at an important moment when AI’s influence on society has never been more pronounced,” said this year’s report. “As AI has improved, it has increasingly forced its way into our lives. Companies are racing to build AI-based products, and AI is increasingly being used by the general public. But current AI technology still has significant problems. It cannot reliably deal with facts, perform complex reasoning, or explain its conclusions.”

 The report starts out by summarizing its overall ten top takeaways.

  • AI beats humans on some tasks, but not on all. “AI has surpassed human performance on several benchmarks, including some in image classification, visual reasoning, and English understanding. Yet it trails behind on more complex tasks like competition-level mathematics, visual commonsense reasoning and planning.”

  • Industry continues to dominate frontier AI research. “In 2023, industry produced 51 notable machine learning models, while academia contributed only 15. There were also 21 notable models resulting from industry-academia collaborations in 2023, a new high.”

  • Frontier models get way more expensive. “According to AI Index estimates, the training costs of state-of-the-art AI models have reached unprecedented levels. For example, OpenAI’s GPT-4 used an estimated $78 million worth of compute to train, while Google’s Gemini Ultra cost $191 million for compute.”

  • The United States leads China, the EU, and the U.K. as the leading source of top AI models. “In 2023, 61 notable AI models originated from U.S.-based institutions, far outpacing the European Union’s 21 and China’s 15.”

  • Robust and standardized evaluations for LLM responsibility are seriously lacking. “New research from the AI Index reveals a significant lack of standardization in responsible AI reporting. Leading developers, including OpenAI, Google, and Anthropic, primarily test their models against different responsible AI benchmarks. This practice complicates efforts to systematically compare the risks and limitations of top AI models.”

  • Generative AI investment skyrockets. “Despite a decline in overall AI private investment last year, funding for generative AI surged, nearly octupling from 2022 to reach $25.2 billion. Major players in the generative AI space, including OpenAI, Anthropic, Hugging Face, and Inflection, reported substantial fundraising rounds.”

  • The data is in: AI makes workers more productive and leads to higher quality work. “In 2023, several studies assessed AI’s impact on labor, suggesting that AI enables workers to complete tasks more quickly and to improve the quality of their output. These studies also demonstrated AI’s potential to bridge the skill gap between low- and high-skilled workers. Still, other studies caution that using AI without proper oversight can lead to diminished performance.”

  • Scientific progress accelerates even further, thanks to AI. “In 2022, AI began to advance scientific discovery. 2023, however, saw the launch of even more significant science-related AI applications — from AlphaDev, which makes algorithmic sorting more efficient, to GNoME, which facilitates the process of materials discovery.”

  • The number of AI regulations in the United States sharply increases. “The number of AI- related regulations in the U.S. has risen significantly in the past year and over the last five years. In 2023, there were 25 AI-related regulations, up from just one in 2016. Last year alone, the total number of AI-related regulations grew by 56.3%.”

  • People across the globe are more cognizant of AI’s potential impact—and more nervous. “A survey from Ipsos shows that, over the last year, the proportion of those who think AI will dramatically affect their lives in the next three to five years has increased from 60% to 66%. Moreover, 52% express nervousness toward AI products and services, marking a 13 percentage point rise from 2022. In America, Pew data suggests that 52% of Americans report feeling more concerned than excited about AI, rising from 37% in 2022.”

A few weeks ago, I attended an excellent presentation, “Highlights from the 2024 AI Index” by Nestor Maslej, Research Manager at Stanford’s Institute for Human-Center AI. Let me briefly discuss a few of these highlights.

  • Multimodal AI. Early language models were limited in scope, some excelling in text comprehension, and others in image processing and voice recognition. Newer, more powerful models are multimodal, capable of handling, text, images, audio, and even video.

  • Robots have become more flexible. The fusion of LLMs with robotics has led to more flexible robotic systems like PALM-E and RT-2 that are capable of asking questions and translating vision and language into physical actions, marking a significant step toward robotic systems that can interact more effectively with the real world.

  • Knowledgeable AI medical systems have arrived. In 2023, AI helped medicine take significant strides forward. For example, EVEscape is an AI tool that can help analyze viral mutations and thus enhance pandemic predictions. Medprompt, a medical knowledge AI system, reached an accuracy rate of 90% in 2023, a 22% increase from its best score in 2022.

  • Number of newly funded AI companies increased. In 2023 there were over 1,800 newly funded AI companies around the world, up 41% from the previous year and almost four times the number in 2013. 99 new generative AI startups received funding in 2023, compared to 56 in 2022, and 31 in 2019.

  • Migration of AI PhDs to industry accelerates. In 2010, there were around 150 new AI PhDs in the US and Canada. A similar percentage of the new PhDs, roughly 40%, took jobs in industry and in academia. In 2022, there were around 360 new PhDs in the US and Canada. The number  joining industry went up to over 70%, and the number entering academia declined by half to 20%. In 2023 the share of new AI PhDs bound for industry increased by 5%.

  • More open foundation models. Almost 150 foundation models were released in 2023, more than double the number in 2022. Of these, around 66% were open source compared to 45% in 2022, and 33% in 2021.

  • Open-source AI research explodes. The number of AI-related projects on GitHub, — a platform commonly used to host open source development projects, — has increased from around 845 in 2011 to approximately 1.8 million in 2023, including a sharp rise of almost 60% of AI projects in 2023 alone.

  • Western nations are generally more pessimistic about AI. An Ipsos Global Survey on AI of 31 countries from around the world found that only about one third of respondents from Western nations felt that AI products and services have more benefits than drawbacks, — 37% from the US, — while respondents from less developed nations were significantly more positive. Similarly, a higher percentage of respondents from Western nations said that AI products and services make them nervous, — 63% from the US.

  • The US leads in AI private investments. In 2023, US private AI investments reached $67.2 billion, nearly 8.7 times more than China, the next highest investor. US private AI investments experienced a significant increase of 22% compared to 2022, while private AI investment in China and the EU, including the United Kingdom, declined by 44.2% and 14.1%, respectively compared to 2022.

  • China dominates AI patents. In 2022, China led global AI patent origins with 61%, significantly outpacing the United States, which accounted for 21% of AI patents. In 2010, the US led AI patents with a 54% share while China’s share was 18%.

“AI faces two interrelated futures,” wrote AI Index co-directors Jack Clark and Ray Perrault in conclusion. “First, technology continues to improve and is increasingly used, having major consequences for productivity and employment. It can be put to both good and bad uses. In the second future, the adoption of AI is constrained by the limitations of the technology. Regardless of which future unfolds, governments are increasingly concerned. They are stepping in to encourage the upside, such as funding university R&D and incentivizing private investment. Governments are also aiming to manage the potential downsides, such as impacts on employment, privacy concerns, misinformation, and intellectual property rights.”

“As AI rapidly evolves, the AI Index aims to help the AI community, policymakers, business leaders, journalists, and the general public navigate this complex landscape… By comprehensively monitoring the AI ecosystem, the Index serves as an important resource for understanding this transformative technological force.”

Irving Wladawsky-Berger is a Research Affiliate at MIT's Sloan School of Management and at Cybersecurity at MIT Sloan (CAMS) and Fellow of the Initiative on the Digital Economy, of MIT Connection Science, and of the Stanford Digital Economy Lab.

Irving Wladawsky-BergerIrving Wladawsky-BergerAI, 2024 AI Index, ai ecosystem, machine learning