Artificial General Intelligence (AGI)

Underline Science Inc.

Abstract

Although the field of AI has long pursued the kinds of generalpurpose, human-level abilities captured by the term AGI, the rise of more general capabilities of neural net models has stimulated discussions about directions forward, implications around success, and doubts about pursuing the goal–which now appears to some observers to be within reach. Main Takeaways * Pursuing understandings of principles and machinery of intelligence that could be harnessed to reach human-level capabilities have always been central in AI, and was explicitly called out in 1956 as an important goal by founders of the discipline. * Calls for focusing more centrally on the bigger picture of “human-level AI” and “artificial general intelligence” in the early 2000s arose in the context of the successful fielding of narrowly scoped AI applications and what some perceived as a lack of progress on the more visionary goals of the field. * Despite challenges with precise definitions and debate about the value of particular notions of AGI, the aspirational goals of AGI and closely related notions, such as “human-level AI,” have inspired many fundamental advances in AI and frame key research questions moving forward to more capable AI systems. On the other hand, success in creating AGI could create societal disruptions and risks and pose significant safety challenges, including challenges to human flourishing and survival. Context & History The AI field has long pursued general principles of intelligence with the direct implication that breakthroughs in our computational understanding of intelligence would enable generalpurpose capabilities. The Turing Test exemplifies this: to pass, a machine must match or exceed human knowledge and reasoning abilities across a range of domains in which people are expected to be competent. The proposal for the Dartmouth workshop that initiated the AI field under that name, written in 1955, begins, “The study is to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves.” This extraordinarily ambitious agenda set the tone for much subsequent work by the participants, including McCarthy’s “Programs with Common Sense”, Newell and Simon’s General Problem Solver, and Solomonoff’s Universal Induction. Just two years later, in 1957, Herb Simon predicted that “the range of problems [machines] can handle will be coextensive with the range to which the human mind has been applied.” Thus, AI has always had as its goal the creation of machines with general powers of intelligence. A great deal of AI research has continued in the vein of pursuing general principles of intelligence, including efforts in representation, sensing, and logical and probabilistic inference. Over the decades, the vast majority of researchers focused on specific methodologies and components of intelligence without much consideration for their integration into general-purpose systems. While some researchers in specific areas were passionate about the potential for generalizing their advances, realworld demonstrations were largely disappointing. Applications harnessing the frontiers in AI methods were narrow and brittle. The perceived lack of progress towards generally intelligent systems that could function in the real world led some within the field to complain that the big picture and high ambitions of AI were being forgotten. For example, Nils Nilsson’s 1995 paper “Eye on the Prize” [1] stated, “AI is now at the beginning of another transition, one that will reinvigorate efforts to build programs of general, humanlike competence,” but this was more of an exhortation than a statement of fact. The prospect of pursuing “human-level” intelligence came to the fore again in the early 2000s. For example, in 2002, Marvin Minsky organized a workshop on “Designing Architectures for HumanLevel Intelligence.” The term artificial general intelligence (AGI) emerged in the same time period as an expression of high ambition by a younger generation of researchers who criticized the field’s seeming focus on narrow applications. Indeed, it was in the early 2000s when machine learning started to be harnessed in multiple narrow applications, each celebrated as a valuable advance. The narrowness of these applications led to calls to discover more generalizable and powerful methodologies, motivated by the fact that the principles of machine learning and reasoning can be applied across domains. AGI was initially defined as AI that could match or exceed human cognitive abilities across a broad range of tasks, echoing the original ambitions of the field in 1956. While these goals were not new to senior AI researchers, the use of the term AGI was seen by many— both inside and outside the field—as a refreshing call for ambitious projects. Beyond AGI and human-level AI, other terms that gained traction around the same time include general-purpose AI and strong AI. However, AGI has become the dominant term in both research and public discourse. Popular books and articles frame AGI as a novel ambition, often portraying it as an unprecedented goal, despite its deep roots in the early history of AI. In many discussions, including those outside of AI research, AGI was linked to both utopian and dystopian futures, reflecting varying perspectives, expectations, and anxieties. The previous AAAI Presidential Panel, the Presidential Panel on Long-Term AI Futures [2], was established in 2008 amid growing interest in AGI, a rekindling of high ambitions by AI leaders and growing public discourse, as well as an upswing in applications of AI being fielded in the open world. The set of meetings and final convening at Asilomar focused on key questions about feasibility, implications, ethics, and safety, as well as research directions for building powerful, general, human-level intelligences. Different perspectives about the nature of AGI extend beyond the core definition of AI methods that could “match or exceed human cognitive abilities across a broad range of tasks.” For example, discussions of AGI, particularly in the popular press, have fueled speculation that sentience or consciousness could be a characteristic of AGI systems. AI researchers generally steer clear of such speculations, pointing out that the analysis and prediction of behavior is independent of attributions of sentience. Some researchers have also suggested that AGI systems must, by definition, have “agentic” abilities, meaning that, like humans, they can function as actors that perceive, learn, process, and act upon their environment to achieve specific goals. Indeed, the capacity to act in pursuit of goals is a fundamental cognitive property of humans, and some AI systems have exhibited such capabilities in rudimentary form since the earliest days of AI. Perhaps more confusing is the notion of “autonomy”and its link to AGI— specifically, the possibility suggested by some that AGI systems might develop goals of their own, entirely distinct from those provided by humans. While this is logically possible—for example, an AI system might overwrite its objectives with new, randomly generated objectives—it’s less clear why it might do so, since that would guarantee failure in its current objectives. On the other hand, the formation of so-called “instrumental” subgoals—such as selfpreservation and acquiring additional computation and financial resources— seems highly likely as AI systems pursue their original objectives. This is obviously a source of concern and an active area of longstanding research. The fact that AGI systems would be more generally capable than humans raises obvious concerns about loss of control of AI; indeed, Alan Turing himself stated that “we should have to expect the machines to take control” once they exceeded human levels of intelligence. One source of risk is misalignment, where the AGI’s goals are not aligned with human preferences about the future; this could arise from misspecification or underspecification by humans—the so-called “King Midas problem”—or from AGI systems failing to understand human preferences correctly [3]. For some, AGI represents a potentially dangerous “threshold” that we cross at our peril. As an example, the “Gladstone Report” [4] commissioned by the US State Department states that “AGI is generally viewed as the primary driver of catastrophic risk from loss of control.” Others use the term “transformative AI” [5] to cover AI systems that have the potential to cause massive disruption of human civilization, noting that this does not require full AGI. We note that sentience and autonomy are not part of core definitions of AGI, even if some have made implicit assumptions about AGI having these attributes. AGI is not a formally defined concept, nor is there any agreed test for its achievement. Some researchers suggest that “we’ll know it when we see it” or that it will emerge naturally from the right set of principles and mechanisms for AI system design. In discussions, AGI may be referred to as reaching a particular threshold on capabilities and generality. However, others argue that this is ill-defined and that intelligence is better characterized as existing within a continuous, multidimensional space. Some (e.g., [6]) contend that the lack of a clear definition makes AGI an unsuitable goal for AI research: human intelligence has many dimensions, and machines will likely far exceed humans in some areas while remaining inferior in others. Moreover, the criteria for comparison, including which particular humans serve as benchmarks and how much pri

Authors 0

1. Author list not loaded yet.