Artificial Intelligence: Realistic Expectations vs. Irrational Exuberance
Image: Depositphotos
Source: Irving Wladawski-Berger, CogWorld think tank member
The 2024 MIT Sloan CIO Symposium took place on Tuesday, May 14 at the Royal Sonesta, a hotel overlooking the Charles River a short walk from the MIT campus in Cambridge, MA. Not surprisingly, AI was the dominant theme in this year’s Symposium, with a number of keynotes and panels on the topic. In addition, a pre-event program was added on the day before the Symposium, which included a number of more informal roundtable discussions on various aspects of AI, such as legal risks in AI deployment, AI as a driver for productivity, and human’s role in AI-augmented workplaces.
The Symposium’s closing keynote, What Works and Doesn't Work with AI, was delivered by MIT professor emeritus Rodney Brooks. Professor Brooks was director of the MIT AI Lab from 1997 to 2003, and was then the founding director of MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) from 2003 until 2007. A robotics entrepreneur, he’s founded of a number of companies, including iRobot, Rethink Robotics, and Robust.Ai
“There is a lot of AI and robotics hype right now,” said the blurb describing Brooks’ keynote in the Symposium agenda. “Amongst it are a few important innovations that may help many businesses. The trick is to figure out what is close to deployable and what what is off in an uncertain distant future. This keynote will talk both about the traps all of us fall in to evaluating AI claims and demonstrations, and what to look for specifically which will get you to a solid understanding of the time frame for if and when technology will be ready to deploy.”
Let me summarize his keynote.
Brooks started out by reminding us that AI has been an academic discipline since the 1950s. The field’s founders believed that just about every aspect of human intelligence could in principle be precisely expressed as software and executed in increasingly powerful computers.
Leading AI researchers in the ’60s, ’70, and early ’80s were convinced that AI systems capable of human-like cognitive capabilities could be developed within a generation, and obtained government funding to implement their vision. A number of early projects built impressive small AI demos, but the demos didn’t scale. Eventually it became clear that all these various projects had grossly underestimated the difficulties of developing machines exhibiting human-like intelligence, because in the end, you cannot express as software barely understood cognitive capabilities like language, thinking, or reasoning. After years of unfulfilled promises and hype, these ambitious AI approaches were abandoned, and a so called AI winter of reduced interest and funding set in the 1980s that nearly killed the field.
AI was reborn in the 1990s. Instead of trying to program human-like intelligence, the field embraced a statistical approach based on analyzing patterns in vast amounts of data with sophisticated algorithms and high performance supercomputers. AI researchers discovered that such an information-based approach produced something akin to intelligence. Moreover, unlike the earlier programming-based projects, the statistical approaches scaled very nicely. The more information you have, the more sophisticated the algorithms, the more powerful the supercomputers, the better the results.
Over the next few decades AI achieved some very important milestones, including Deep Blue’s win over chess grandmaster Garry Kasparov in a 1997 six game match, Watson’s 2011 win of the Jeopardy! Challenge against the two best human Jeopardy! players, AlphaGo’s unexpected win in 2016 over Lee Sedol, — one of the world’s top Go players. In addition, a number of entrants successfully completed the 2007 DARPA Grand Challenge for self-driving vehicles in an urban environment, and the 2012 DARPA Robotics Challenge for the use of robots in disaster or emergency-response scenarios.
After these and other milestones, AI appeared to be “on the verge of changing everything!,” said Brooks. Since 2017, he’s been posting a Predictions Scorecard at the beginning of each year, where he compares the predictions for future milestones in robotics, AI and machine learning; in self driving cars; and in human space travel with the actual progress achieved over the previous year.
“I made my predictions because at the time, just like now, I saw an immense amount of hype about these three topics, and the general press and public drawing conclusions about all sorts of things they feared (e.g., truck driving jobs about to disappear, all manual labor of humans about to disappear) or desired (e.g., safe roads about to come into existence, a safe haven for humans on Mars about to start developing) being imminent,” Brooks wrote. “My predictions, with dates attached to them, were meant to slow down those expectations, and inject some reality into what I saw as irrational exuberance.”
Why have so many AI predictions turned out so wrong?, he asked in the keynote. The answer is what he called the Seven Deadly Sins of Predicting the Future of AI in a 2017 essay.
1. Overestimating and Underestimating harks back to what’s become known as Amara’s Law: We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run. “Artificial Intelligence has the distinction of having been the shiny new thing and being overestimated again and again, in the 1960’s, in the 1980’s, and I believe again now,” wrote Brooks. “Not all technologies get underestimated in the long term, but that is most likely the case for AI. The question is how long is the long term.”
2. Indistinguishable from Magic is closely associated with a proverb by science fiction writer Arthur C. Clarke that’s become known as Clarke’s third law: Any sufficiently advanced technology is indistinguishable from magic. “This is a problem we all have with imagined future technology,” said Brooks. “If it is far enough away from the technology we have and understand today, then we do not know its limitations. It becomes indistinguishable from magic. … Nothing in the Universe is without limit. Not even magical future AI.”
3. Exponentials. “Many people are suffering from a severe case of exponentialism,” wrote Brooks. Exponentialism was put on the map in the technology world by the very impressive 50-years run of Moore’s Law. The semi-log graphs associated with Moore’s Law have since become a visual metaphor for the technology revolution unleashed by the exponential improvements of digital components, from processing speeds to storage capacity. Moore’s Law has had quite a run, but like all things based on exponential improvements, it must eventually slow down and flatten out.
Over the past 30 years, the necessary ingredients have come together to significantly increase the performance of AI systems: powerful, inexpensive computer technologies; advanced algorithms and models; and huge amounts of all kinds data. Was the seemingly exponential performance increase of the past 30 years an isolated event in the history of AI systems?, asked Brooks. We don’t really know, but there’s no law that says how often such events will happen. “So when you see exponential arguments as justification for what will happen with AI remember that not all so called exponentials are really exponentials in the first place, and those that are can collapse suddenly when a physical limit is hit, or there is no more economic impact to continue them.”
4. Performance versus Competence. “We all use cues about how people perform some particular task to estimate how well they might perform some different task,” wrote Brooks. For example: “People hear that some robot or some AI system has performed some task. They then generalize from that performance to a competence that a person performing the same task could be expected to have. And they apply that generalization to the robot or AI system. Today’s robots and AI systems are incredibly narrow in what they can do. Human-style generalizations do not apply.”
5. Speed of Deployment. “A lot of AI researchers and pundits imagine that the world is already digital, and that simply introducing new AI systems will immediately trickle down to operational changes in the field, in the supply chain, on the factory floor, in the design of products. Nothing could be further from the truth. Almost all innovations in robotics and AI take far, far, longer to be really widely deployed than people in the field and outside the field imagine.”
6. Hollywood Scenarios. Many AI researchers and pundits ignore the fact that if we’re able to eventually build super-intelligent AI systems, the world will have changed significantly by then. “We will not suddenly be surprised by the existence of such super-intelligences. They will evolve technologically over time, and our world will come to be populated by many other intelligences, and we will have lots of experience already. … We will change our world along the way, adjusting both the environment for new technologies and the new technologies themselves. I am not saying there may not be challenges. I am saying that they will not be sudden and unexpected, as many people think.”
7. Suitcase Words. A suitcase word is a term created by MIT AI pioneer Marvin Minsky to refer to words that can have multiple different and confusing meanings depending on the context in which they’re used. Learning, for example is one such word, which means something very different when applied to machine learning than when applied to human learning.
“Suitcase words mislead people about how well machines are doing at tasks that people can do,” said Brooks. “That is partly because AI researchers—and, worse, their institutional press offices — are eager to claim progress in an instance of a suitcase concept. The important phrase here is an instance. That detail soon gets lost. Headlines trumpet the suitcase word, and warp the general understanding of where AI is and how close it is to accomplishing more.”
Brooks finished his keynote by presenting what he called “My Three Laws of Artificial Intelligence”:
When an AI system performs a task, human observers immediately estimate its general competence in areas that seem related. Usually that estimate is wildly overinflated.
Most successful AI deployments have a human somewhere in the loop (perhaps the person they are helping) and their intelligence smooths the edges.
Without carefully boxing in how an AI system is deployed there is always a long tail of special cases that take decades to discover and fix.
Irving Wladawsky-Berger is a Research Affiliate at MIT's Sloan School of Management and at Cybersecurity at MIT Sloan (CAMS) and Fellow of the Initiative on the Digital Economy, of MIT Connection Science, and of the Stanford Digital Economy Lab.