The Artificial Intelligence Index Report 2023

Stanford University’s Institute for Human-Centered Artificial Intelligence (HAI) issued its sixth annual AI Index Report subtitled Measuring trends in Artificial Intelligence in April of 2023.

Top takeaways from this year’s AI Index Report include:

Industry races ahead of academia.
- Until 2014, most significant machine learning models were released by academia.
- In 2022, there were 32 significant industry-produced machine learning models versus 3 produced by academia.
- Building state-of-the-art AI systems requires large amounts of data, computer power, and money – resources that industry possesses in greater amounts.
Performance saturation on traditional benchmarks.
- Year-over-year improvement on many benchmarks continues to be marginal.
- The speed at which benchmark saturation is being reached is increasing.
- New, more comprehensive benchmarking suites such as BIG-bench and HELM are being released.
AI is both helping and hurting the environment.
- AI systems can have serious environmental impacts.
- BLOOM’s training run emitted 25 times more carbon than a single air traveler on a one-way trip from New York to San Francisco.
- New reinforcement learning models like BCOOLER show that AI systems can be used to optimize energy usage.
The world’s best new scientist…AI?
- AI models are starting to rapidly accelerate academic progress.
- In 2022, AI models were used to aid hydrogen fusion, improve the efficiency of matrix manipulation, and generate new antibodies (and these are the non-classified accomplishments).
The number of incidents concerning the misuse of AI is rapidly rising.
- The number of AI incidents has increased 26 times since 2012.
- The AIAAIC tracks AI incidents.
The demand for AI-related professional skills is increasing across virtually every American industrial sector.
- Across nearly all sectors for which data exists, the number of AI-related job postings has increased on average from 1.7% in 2021 to 1.9% in 2022.
- Only exceptions are agriculture, forestry, fishing, and hunting.
For the first time in the last decade, year-over-year private investment in AI decreased.
- Global AI private investment was $91.9 billion in 2022, a 26.7% decrease from 2021.
- The number of AI-related funding events and newly funded companies decreased as well.
- In 2022, the amount of private investment in AI was 18 times greater than it was in 2013.
While the proportion of companies adopting AI has plateaued, the companies that have adopted Ai continue to pull ahead.
- The proportion of companies adopting AI in 2022 has more than doubled since 2017.
- The percentage has plateaued in recent years between 50 and 60 percent.
- Adopters report meaningful cost decreases and revenue increases.
Policymaker interest in AI is on the rise.
- An analysis of the legislative records in 127 countries shows the number of bills passed containing “artificial intelligence” increased from 1 in 2016 to 37 in 2022.
Chinese citizens are among those who feel the most positively about AI products and services. Americans…not so much.
- In a 2022 IPSOS survey, 78% of Chinese respondents agreed with the statement that products and services using AI have more benefits than drawbacks.
- Only 35% of Americans sampled agreed.

The full 385-page report is available for downloading or you can choose to download individual chapters. There are many hyperlinks to additional data outside of the report. The eight individual chapters are:

The mission of the AI Index Report is to “provide unbiased, rigorously vetted, broadly sourced data in order for policymakers, researchers, executives, journalists, and the general public to develop a more thorough and nuanced understanding of the complex field of AI.

Key highlights from each of the chapters are listed below.

Chapter 1 – Research and Development

The U.S. and China had the greatest number of cross-country collaborations in AI publications from 2010 to 2021 although the pace of collaboration has slowed.
AI research is on the rise, across the board.
China continues to lead in total AI journal, conference, and repository publications.
Industry races ahead of academia.
Large language models are getting bigger and more expensive.

Chapter 2 – Technical Performance

Performance saturation on traditional benchmarks.
Generative AI breaks into the public consciousness.
AI systems become more flexible.
Capable language models still struggle with reasoning.
AI is both helping and harming the environment.
The world’s best new scientist…AI?
AI starts to build better AI.

Chapter 3 – Technical AI Ethics

The effects of model scale on bias and toxicity are confounded by training data and mitigation methods.
Generative models have arrived and so have their ethical problems.
The number of incidents concerning the misuse of AI is rapidly rising.
Fairer models may not be less biased.
Interest in AI ethics continues to skyrocket.
Automated fact-checking with natural language processing isn’t so straightforward after all.

Chapter 4 – The Economy

The demand for AI-related professional skills is increasing across virtually every American industrial sector.
For the first time in the last decade, year-over-year private investment in AI decreased.
Once again, the United States leads in investment in AI.
In 2022, the AI focus area with the most investment was medical and healthcare ($6.1 billion), followed by data management, processing, and cloud ($5.9 billion), and Fintech ($5.5 billion).
While the proportion of companies adopting AI has plateaued, the companies that have adopted AI continue to pull ahead.
AI is being deployed by businesses in multifaceted ways.
AI tools like Copilot are tangibly helping workers.
China dominates industrial robot installations.

Chapter 5 – Education

More and more AI specialization.
New AI PhDs increasingly head to industry.
New North American CS, CE, and information faculty hires stayed flat.
The gap in external research funding for private versus public American CS departments continues to widen.
Interest in K-12 AI and computer science education grows in both the United States and the rest of the world.

Chapter 6 – Policy and Governance

Policymaker interest in AI is on the rise.
From talk to enactment – the U.S. passed more AI bills than ever before.
When it comes to AI, policymakers have a lot of thoughts.
The U.S. government continues to increase spending on AI.
The legal world is waking up to AI.

Chapter 7 – Diversity

North American bachelor’s, master’s, and PhD-level computer science students are becoming more ethnically diverse.
New AI PhDs are still overwhelmingly male. In 2021, 78.7% of new AI PhDs were male.
Women make up an increasingly greater share of CS, CE, and information faculty hires.
American K-12 computer science education has become more diverse, in terms of both gender and ethnicity.

Chapter 8 – Public Opinion

Chinese citizens are among those who feel the most positively about AI products and services. Americans…not so much.
Men tend to feel more positively about AI products and services than women. Men are also more likely than women to believe that AI will mostly help rather than harm.
People across the world and especially America remain unconvinced by self-driving cars.
Different causes for excitement and concern.
NLP researchers…have some strong opinions as well.

I have not read the report in its entirety. I am impressed by its thoroughness. There is an index of items for each chapter that quickly provides you with an overview of the comprehensiveness of the content included.

I was slightly disappointed with the chapter on education. It focuses on computer science education in higher education and K-12. It’s not a surprise that college computer science faculty increased by 39 percent since 2011 given the increasing use of computers in our lives. Industry sources of funding grants to departments have increased as well. The gap in funding between private universities ($9.7 million in expenditures) and public universities ($5.7 million in expenditures) continues to widen and may be of concern for certain states.

There were several illustrative slides from K-12 education that were interesting. In the figure below, there are three states (Maryland, South Carolina, and Arkansas) with over 90 percent of their high schools teaching computer science. In the case of Maryland, two of their neighboring states, DC (45%) and Delaware (40%) drop below 50 percent. Whether the higher percentage of high schools in Maryland teaching computer science classes versus in DC or DE is indicative of demand or availability of instructors is not explained in the report.

The availability of computer science courses in most public high schools appears to translate into the interest of high school students taking the AP Computer Science exam. Once again, Maryland leads the nation in the number of exams per 100,000 residents. See figure below.

Another figure that I found interesting regards the time allocated to various AI components in international K-12 curriculum as surveyed by UNESCO (see figure below). The distribution of topics beyond foundational programming was not a surprise. I could see the development of a course for non-programmers that highlighted the data literacy (know the source of the data used), contextual problem-solving, application of AI to other domains, ethics of AI, and social implications of AI as areas to understand to prepare for its broader application in society. I would add prompt engineering as an area of focus as well.

I believe it would be helpful if the authors of the education chapter could find evidence that high schools and colleges are incorporating instruction regarding the use of AI in their non-computer science curriculum.

Overall, the report’s chapters provide details of changes related to AI, but mostly from the technical side. For example, the economy chapter shows the top specialized skills sought for AI job postings. All of them are computer science/information technology positions. None relate to people in non-IT positions. One figure (see below) from the economy chapter that I found interesting related to ranking sectors of the economy by investment in AI. The medical/healthcare sector was highest followed by fintech (financial tech). EdTech, a sector that I follow, was one of the lowest in terms of AI investment.

Even though private investment in AI fell in the U.S. last year, there is still a substantial sum being invested. A final point that I noted from the report is that those companies that have invested in AI continue to benefit from its implementation versus those that have not.

I plan to bookmark the report and return to it from time to time as I continue to explore and track the advances of AI. For those of you who like the report and want to cite it, the official citation is:

Nestor Maslej, Loredana Fattorini, Erik Brynjolfsson, John Etchemendy, Katrina Ligett, Terah Lyons, James Manyika, Helen Ngo, Juan Carlos Niebles, Vanessa Parli, Yoav Shoham, Russell Wald, Jack Clark, and Raymond Perrault, “The AI Index 2023 Annual Report,” AI Index Steering Committee, Institute for Human-Centered AI, Stanford University, Stanford, CA, April 2023.