How Did We Get To AI?
How did we get from room-sized computers to AI that writes, creates, and acts? Created as a resource for AI Governance, this article explores the breakthroughs, bold ambitions, and human choices that brought us here—and asks where the race might take us next.
An introduction for Santa Clara Law AI Governance Class
Artificial intelligence entered everyday conversation in 2022, when ChatGPT’s public release introduced a wide audience to chatbots that could draft emails, explain concepts, and write code. Those interactions rested on decades of research and changes in how organizations stored information, connected computers, and obtained computing power.[1]
For students taking AI Governance in 2027, many AI capabilities may seem ordinary. Understanding their history means asking what each development made possible, which choices shaped it, and what remained unresolved. This article traces that history as a foundation for studying the legal and governance questions surrounding AI.
The developments overlap. Learning research preceded the Web and commercial cloud services by decades, and no single invention explains today’s systems. Ideas became useful at different times as data, hardware, funding, and ways to reach users changed.
1 The foundations of machine intelligence
In 1950, British mathematician Alan Turing published Computing Machinery and Intelligence, exploring how to assess whether a machine exhibits intelligent behavior. His proposed imitation game asked whether an evaluator could distinguish a machine from a person through written interaction. When computers were scarce and largely associated with calculation, treating language and learning as computing problems opened a broader research agenda.[2]

The field acquired its name through the Dartmouth Summer Research Project on Artificial Intelligence in 1956. Its organizers proposed studying whether machines could simulate features of intelligence, including using language, forming concepts, and improving themselves. These ambitions appeared long before the computing infrastructure of the internet era.[3]
Research followed several paths. Some programs manipulated symbols and applied explicit rules; others explored learning through experience, including games and early neural networks. During the 1960s, Edward Feigenbaum and colleagues at Stanford helped develop expert systems, including work on chemical analysis. The aim was to represent specialized knowledge in a form a computer could use to solve problems within a particular field.[4]
Progress was uneven. Enthusiasm gave way to periods of reduced confidence and funding, commonly called “AI winters.” Computing limitations mattered, as did the difficulty of turning demonstrations into systems that worked reliably in less controlled environments. Statistical methods gained ground as data and computing resources became more available.[5]
Food for Thought: Why did early researchers focus on games? Games provide fixed rules, permitted moves, and a way to determine who wins. Conversation offers less certainty: words have several meanings, people leave things unsaid, and context changes what counts as a useful answer. Success in a tightly defined task did not automatically carry over to everyday interaction.
2 Mainframes and databases made information reusable
Alongside AI research, governments and businesses developed infrastructure for large-scale digital processing. Mainframes handled calculations and records faster than manual methods could. They were expensive installations operated by specialists, but they established the practicality of processing organizational information electronically.[6]
IBM’s System/360, introduced in 1964, made expansion easier through a family of computers with a compatible architecture. Customers could move to a more powerful machine while preserving much of their software investment. Compatibility reduced the need to rewrite programs whenever hardware changed.[7]
The System/360 unified a family of computers under a single architecture for the first time and established the first platform business model, which is still embraced today by IBM as well as by technology companies across industries ranging from online search and e-commerce to social media and ride-sharing.
In 1970, Edgar F. Codd at IBM’s San Jose Research Laboratory proposed a relational database model that separated the logical organization of data from its physical storage. Information could be represented through tables and relationships, allowing users to ask for data without specifying a path through the storage structure. This became the foundation of relational databases.[8]

Food For Thought: Why were relational databases such a breakthrough? Today, asking which customers bought a particular product seems straightforward. Before relational databases, answering that question often required a specialist to write a program that navigated the specific way records were stored. Relational databases let users connect tables of customers, purchases, and products through shared fields and ask new questions without needing to understand the underlying storage structure. The breakthrough was flexibility: the same records could answer questions their original designers had never anticipated.
Donald Chamberlin and Raymond Boyce’s work on SEQUEL, later SQL, helped make that approach usable. A query could describe the information wanted, leaving the database system to determine how to retrieve it.[9] For example, tables of customers, purchases, and products could be connected through shared fields to answer “Which customers bought this product?” The same records could serve questions their original designers had not anticipated.
Recording a sale and analyzing years of sales require different kinds of processing. Transaction processing supports operations such as entering a purchase. Online analytical processing, or OLAP, supports analysis across dimensions such as product, region, and quarter.[10]
Food for Thought: What did OLAP add? Imagine receiving a printed report of last year’s total sales, then wanting to know whether a decline came from one region, one product, or one season. Each follow-up might require another report. OLAP made it easier to explore those dimensions interactively, moving from an overall result to the details behind it. It helped turn accumulated records into information that people could investigate for themselves. Predictive models would take a further step by estimating outcomes that the records did not yet contain.
These developments matter to AI’s history because they made information easier to organize, retrieve, and reuse. They also show why collecting records and extracting useful knowledge from them are different achievements.
3 The internet and the Web expanded access to data
Computer networking developed through several contributions. In the 1960s, Paul Baran and Donald Davies independently explored approaches that helped establish packet switching. Information could be divided into packets, sent through a network, and reassembled at its destination. ARPANET connected its first computers in 1969, advancing the sharing of resources across locations.[11]
Building on this approach, Vint Cerf and Robert Kahn developed an architecture for connecting different networks. The resulting TCP/IP protocols supported a larger “network of networks”: IP handled addressing and routing, while TCP supported reliable, ordered delivery. ARPANET’s adoption of TCP/IP in 1983 was a major milestone.[12]

The internet supplied networking infrastructure. The World Wide Web made information easier to publish and navigate. Tim Berners-Lee proposed the Web at CERN in 1989 and developed its first browser and server by the end of 1990. Linked pages gave readers a common way to move between documents on different computers.[13]

Food For Thought: Can you imagine a world without the internet or web pages? Consider researching a paper without websites, links, or downloaded sources. You might consult printed indexes and request documents by mail. Computers could store information, but material on one machine was often difficult to access from another. Networks enabled exchange across distances; the Web made that connected information easier for people to use.
As writing, images, and everyday activities moved online, they created vast collections of digital material that later contributed to modern AI. The change involved both the availability of information and the practicality of finding and processing it.
Growing dependence on computing also exposed shared vulnerabilities. Y2K arose because software often represented years with two digits, potentially confusing 2000 with 1900. Extensive testing, remediation, and coordination helped the United States avoid major national system failures during the rollover.[14] A small design assumption, repeated across connected systems, had become a problem requiring collective preparation.
Food For Thought: Why did two missing digits matter so much? You might expect a date to be a routine detail. But software could use that date to calculate interest, determine benefit payments, or put events in order. A small assumption, repeated across many connected systems, could therefore affect essential services. Y2K revealed how deeply computing had become embedded in everyday life.
4 Cloud computing and connected devices changed the scale
The ambition to access computing wherever people happened to be preceded smartphones. General Magic, founded in 1990, explored portable personal communicators connected to networked services. Its Telescript technology envisioned software agents performing activities on users’ behalf. Those agents used a different mechanism from modern language-model agents, but the goal of delegating tasks across a network was already present.[15]

Long before today’s AI, General Magic imagined software carrying out tasks on our behalf. Today’s systems revisit that ambition, changing how we express our requests and how software acts on them.
4.1. Computing became a service
Cloud computing made processing and storage available over a network, with capacity that customers could adjust as demand changed. Instead of buying all the equipment needed for a service in advance, developers could obtain resources when required and release them when demand fell.
Amazon Web Services’ launches of S3 storage and the EC2 computing beta in 2006 were important commercial milestones. A small team could rent capacity to test an application without first building its own computing installation. This changed the cost and practicality of trying ideas when future demand was uncertain.[16]
Around 2006, “ubiquitous”—meaning present everywhere—captured an ambition that may seem ordinary to a student today: access to computing and information wherever you happened to be, rather than only at a particular desktop computer. The idea had earlier roots: Mark Weiser at Xerox PARC described “ubiquitous computing” in 1991, envisioning computers woven into everyday life. Mark Weiser, “The Computer for the 21st Century,” Scientific American, vol. 265, no. 3 (September 1991), pp. 94–104. Read the article (Stanford-hosted copy).
NIST’s influential 2011 definition of cloud computing identified five essential characteristics: on-demand self-service, broad network access, resource pooling, rapid elasticity, and measured service.[17] Together, they described more than a computer in a remote location. They described a service model for obtaining and managing computing resources.

Food For Though: What if your files were stuck on one computer?Imagine arriving at class and realizing your assignment is saved on your home computer. Before online storage became commonplace, you often needed that machine—or a disk, USB drive, or emailed copy—to retrieve it. Cloud computing helped change those expectations. Amazon’s launch of S3 and EC2 in 2006 made storage and processing power available over a network, helping developers build services without buying all the necessary hardware. For a student today, opening documents across devices or accessing AI through a browser may feel ordinary. Behind that convenience is a major shift: the device in your hands can draw on computing resources located somewhere else.
4.2. Devices expanded where data could originate
Apple’s introduction of the iPhone in 2007 was a major mobile-computing milestone. It combined a phone, multimedia functions, and internet access through a touchscreen people could carry throughout the day. Smartphones existed earlier, but this combination helped change expectations about access to digital services.[18]

The Internet of Things, or IoT, extended connectivity to physical objects. Sensors and identification technologies could record temperature, movement, location, or the passage of goods without someone typing in each observation. A watch recording a workout and a parcel registering at a warehouse illustrate how physical events could become digital information.[19]
These developments helped create the conditions for “big data”: collections whose size, complexity, or rate of growth challenged conventional processing. They increased demand for methods that could extract useful knowledge from large datasets.[20] More information was available, but availability alone did not tell computers what to do with it.

5 Machine learning developed capabilities through experience
Machine learning offered ways to develop capabilities from examples and experience, reducing the need for programmers to specify every rule. Its roots reach back well before the infrastructure that later helped it flourish. Arthur Samuel’s 1959 paper on checkers described techniques for improving a program’s play through experience.[21]
Developers still choose a model’s design, training data, and objective. Training adjusts numerical parameters that influence its outputs. Some of the relationships the system uses are learned through that process rather than written directly as rules.
Food For Thought — What changed when a program could learn from data? Imagine trying to write rules for every way someone might form a handwritten “7.” You would need to anticipate differences in angle, shape, and writing style. A learning system can instead use examples to adjust the numerical relationships it applies to new inputs. The work shifts toward designing the model and training process, selecting useful examples, and checking whether the learned patterns carry over to unfamiliar cases. The work therefore changes. Instead of anticipating every possible pattern, developers design the learning process, select examples, and assess whether the resulting model performs well on unfamiliar cases.

The different approaches to learning can be combined during a system’s development. They also require different kinds of evidence: a collection of transactions does not automatically identify which were fraudulent.[22]
5.1. Neural networks learned representations
Artificial neural networks are one family of machine-learning models. They use connected computational units, often arranged in layers, with numerical weights adjusted during training. Each layer transforms information received from the preceding one. Deep learning uses multiple layers to learn increasingly complex representations. The biological terminology is an analogy; it does not establish that these systems think like people.[23]
The ideas developed over decades. Frank Rosenblatt’s 1957 perceptron learned classifications by adjusting weights.[24] Kunihiko Fukushima’s 1980 neocognitron was a precursor to modern convolutional neural networks, which apply learned filters across images.[25] In 1986, David Rumelhart, Geoffrey Hinton, and Ronald Williams published an influential demonstration of backpropagation for learning representations in multilayer networks.[26] Yann LeCun and colleagues applied this approach to handwritten ZIP-code recognition in 1989.[27]
By 1989, important pieces of later deep learning were already in place. Making those methods work at a much larger scale would require further advances in data, hardware, and implementation.
5.2. Data and computing power made older ideas more practical
GPUs, or graphics processing units, can perform many calculations in parallel. That makes them useful for neural-network training, which involves large numbers of numerical operations. NVIDIA’s CUDA helped developers use GPUs for work beyond graphics.[28]
Large datasets supplied another essential resource. ImageNet, presented in 2009, provided categorized images for training and comparing recognition systems. Collecting and labeling data was itself a research contribution.[29]
These elements came together in AlexNet, developed by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. Its performance in the 2012 ImageNet competition demonstrated a substantial improvement in image classification.[30] The result combined a suitable network, efficient GPU implementation, and extensive labeled data.

Food For Thought — Why was recognizing pictures so difficult? Recognizing objects is difficult because photographs vary in lighting, angle, background, and scale. AlexNet showed how learned visual features could help address those variations when sufficient data and computing power made training practical. Its success encouraged greater investment in deep learning and illustrated why a promising idea might wait decades for wider impact.
6 Games tested learning and planning
Recognizing an image and choosing a move pose different problems. An image classifier assigns a category to an input; a game-playing system chooses actions whose consequences unfold over time. Games provided environments for testing learning, search, and planning against measurable outcomes.
In March 2016, DeepMind’s AlphaGo defeated Lee Sedol by four games to one. It combined neural networks, search, and reinforcement learning. The system initially learned from human games and improved through play against versions of itself. Learned evaluations helped direct its search toward promising possibilities.[31]
Go has simple rules but too many possible continuations for a computer to examine them all. AlphaGo’s achievement showed how experience could help a system decide where to concentrate its calculations.

Later systems changed what researchers supplied in advance. AlphaGo Zero learned through self-play without first training on human games. AlphaZero applied a general approach separately to Go, chess, and shogi, starting with each game’s rules. MuZero learned a model useful for planning without receiving a complete account of the environment’s dynamics.[32]
OpenAI Five’s 2019 victory against the reigning Dota 2 world champions extended deep reinforcement learning to a complex multiplayer game involving teams, incomplete information, and successive actions.[33]
These projects raise a recurring design question: what should a system learn from examples, what should it learn through interaction, and what should people supply directly? Their success also depended on the environment and objective. A game supplies a score or outcome. Determining whether advice is useful or an explanation adequate is a different challenge.

7 Broadly trained language models supported many tasks
An image classifier or Go-playing system does not automatically know how to translate a paragraph or summarize a contract. A further shift came from training models on broad collections of language and adapting that learning to multiple uses.
In 2017, Ashish Vaswani and colleagues introduced the Transformer in Attention Is All You Need. Developed for machine translation, the architecture used attention mechanisms to weigh relationships among parts of a sequence. It also allowed more parallel computation during training than the recurrent systems used for comparison.[34]
Food For Thought — Why does context matter? Consider the sentence “The trophy did not fit in the suitcase because it was too large.” Interpreting “it” requires relating different parts of the sentence. Attention gives a network a way to weigh those relationships. Transformers became a major architecture for large language models, although “Transformer” and “language model” are not interchangeable terms.
A large language model, or LLM, learns statistical relationships from extensive language data. Many generative LLMs produce text by repeatedly predicting the next token, which can represent a word, part of a word, or punctuation. Broad pre-training develops capabilities that can then be applied or adapted to different tasks.[35]

In 2021, Stanford researchers proposed “foundation models” for models trained on broad data at scale and adaptable to many downstream tasks. Developers could start with an existing model, then adapt or integrate it instead of building every capability from the beginning.[36] Google’s PaLM was another example of this direction.[37]
7.1. Using a model became different from retraining it
Fine-tuning changes a model’s parameters through additional training. Prompting supplies instructions, examples, or context for the model to use when responding. In-context learning describes using information in that input to perform a task without updating trained parameters. Prompt engineering means deliberately designing and refining those inputs.
This distinction changed the work required to use software. The same broadly trained model could attempt a summary, translation, or draft in response to ordinary-language instructions, without a new program or separate training process for each request.
Researchers also studied prompts containing intermediate steps before an answer. Chain-of-thought prompting improved performance on several reasoning benchmarks, but a generated explanation is not necessarily a faithful account of how a model reached its answer.[38] An explanation that sounds convincing still needs scrutiny.
Food For Thought — What changed when describing the task became enough to get started? Imagine needing a different program every time you wanted to summarize a document, translate a paragraph, or draft an email. Asking one chatbot to attempt all three may seem unremarkable now. Historically, the shift was significant: instructions and examples in ordinary language could guide the same broadly trained model through different tasks, without someone writing a new program or retraining it for each request. At that point, learning how to ask became another way to put computing capabilities to work.
8 Generative AI became an everyday interface
Language models form part of a broader history of generative AI: systems that produce text, images, audio, video, or code. Predictive AI usually refers to systems used to estimate outcomes, scores, or categories. The distinction concerns use and output. Generative models also perform prediction internally, and applications can combine prediction with content generation.
Generation developed through several research paths. Ian Goodfellow and colleagues’ 2014 work on generative adversarial networks, or GANs, paired a generator producing samples with a discriminator learning to distinguish them from training data. Their competing objectives guided training, using a different approach from Transformer language models.[39]
Later systems made ordinary language an interface to specialized capabilities. DALL·E generated images from descriptions;[40] Codex translated instructions into code;[41] GPT-4 accepted image and text inputs and produced text outputs.[42] These systems changed how a task could begin and who could attempt it, while leaving a need for expertise and review.
8.1. Training an assistant involved human choices
Predicting text does not by itself specify how an assistant should respond. Developers also trained models to follow instructions, produce preferred responses, and refuse some requests.
Reinforcement learning from human feedback, or RLHF, uses human judgments to help guide training. People may compare responses, and those comparisons can help train a model that scores outputs for a subsequent learning stage. OpenAI used this approach for the original ChatGPT.[43]
Anthropic’s Constitutional AI used human-selected principles alongside AI-generated critiques, revisions, and preferences to help train an assistant. Its “constitution” was the set of principles guiding that process.[44] Both approaches incorporated human choices, through judgments, examples, or selected principles.

Conversational interfaces brought broadly trained models to a wider public. OpenAI released ChatGPT on November 30, 2022;[45] Anthropic released Claude 2 through an API and public beta website in July 2023.[46] Users could describe a task, ask follow-up questions, and revise requests within an exchange.
Instead of first learning a program’s menus or commands, a user could begin with “Explain this” or “Rewrite it more clearly.” The change involved access as well as capability. Fluency, however, did not establish accuracy: ChatGPT’s launch announcement acknowledged plausible but incorrect answers and sensitivity to wording.[47]
8.2. Agents connected models to actions
An AI agent can combine a model with tools and a process for choosing and carrying out actions. A travel assistant might compare options through connected services; a coding assistant might edit files and run checks. The application’s permissions and tools determine what it can actually do.
This returns to General Magic’s ambition of delegating activities across a network. Modern language-model agents generally call tools and services rather than moving executable programs between computers. The goal of having software coordinate steps on a user’s behalf has persisted, while the technical route and the way people express requests have changed.

9 Organizations and incentives shaped development
Technical discoveries alone do not explain modern AI. Decisions about funding, infrastructure, publication, and access influenced which ideas could be pursued and who could use the results. Universities, public institutions, companies, and civil-society organizations brought different resources and priorities.
- AI research developed internationally. Competition involved talent, computing resources, investment, and applications, with no single measure capturing leadership across all dimensions.[48] Within organizations, researchers, founders, and employees could influence decisions through expertise and relationships as well as formal authority.
- Distribution choices also mattered. Some developers released model weights so others could run or adapt models; others offered access mainly through hosted services. Access to weights, source code, training data, and permission to reuse them are separate questions. These choices affect who can investigate a model, build on it, compete with its developer, or monitor its use.[49]
- Concerns about incentives predated ChatGPT. The Center for Humane Technology and The Social Dilemma drew attention to how commercial rewards could shape digital products and their effects.[50] Researchers also challenged the direction of language-model development. The 2021 paper On the Dangers of Stochastic Parrots examined environmental and financial costs, training-data problems, and the risks of treating generated language as meaningful or authoritative.[51]

Debates about research independence raised a related question: how freely can researchers investigate and publish concerns when their findings conflict with the interests of the organization funding their work?[52] These institutional conditions shape what becomes known about AI as well as what gets built.
OpenAI’s November 2023 leadership crisis showed how formal authority and practical influence can diverge. The board dismissed Sam Altman, but employee and partner pressure helped secure his return and changes to the board.

10 Where the race might take us
General-purpose AI can perform many tasks, but usefulness across tasks does not itself establish artificial general intelligence, or AGI. AGI describes a broader ambition for intelligence often compared with human abilities; definitions and proposed tests vary.
- Whether AGI will be reached, when it might arrive, and what it would mean remain uncertain. Its implications for work, education, scientific discovery, and the distribution of power would depend on capabilities, control, and use.
The history traced here gives us a basis for examining those questions. Databases made information reusable; networks and the Web made it easier to exchange and find. Cloud services and connected devices expanded access to computing and data. Learning methods, larger datasets, and parallel processing made new capabilities practical. Broad pre-training and conversational interfaces changed how people could use them.
At each stage, people chose objectives, datasets, architectures, funding arrangements, and ways to distribute their work. Understanding those choices helps us examine the systems we use today and the possibilities still being shaped.
Studying at Santa Clara Law puts you in Silicon Valley, where much of that future is being developed and debated. Some of the people in this class may help write its next chapter, through the systems they build, advise on, or govern. On behalf of Professor Krolik and me, welcome to the race.
Footnotes
- OpenAI’s November 30, 2022 announcement. ↩ Back to text
- Turing, 1950; Turing Digital Archive. ↩ Back to text
- The organizers were John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon. Their proposal dates to August 1955; the summer project followed in 1956. Sources: Dartmouth proposal. ↩ Back to text
- Stanford AI100, Appendix I. ↩ Back to text
- Stanford AI100, Appendix I. ↩ Back to text
- The U.S. Census Bureau accepted UNIVAC I in 1951; IBM announced the 701 in 1952. Sources: U.S. Census Bureau, UNIVAC I; IBM, The 700 Series. ↩ Back to text
- System/360 was introduced in 1964, with Gene Amdahl as chief architect and Fred Brooks as project leader. Sources: IBM’s System/360 history. ↩ Back to text
- Codd published A Relational Model of Data for Large Shared Data Banks in 1970. Sources: Codd, 1970; IBM, Edgar F. Codd. ↩ Back to text
- Chamberlin and Boyce presented SEQUEL in 1974. SQL became an ANSI standard in 1986 and an ISO standard in 1987. Sources: Chamberlin and Boyce’s 1974 paper; SQL standards history. ↩ Back to text
- E. F. Codd, S. B. Codd, and C. T. Salley described the OLAP category in 1993. Microsoft released SQL Server 7.0 OLAP Services in 1998. These milestones concern naming and commercialization rather than a single invention of multidimensional analysis. Sources: Microsoft, OLAP overview; Codd, Codd, and Salley; Microsoft’s 1998 announcement. ↩ Back to text
- Baran described distributed communications using message blocks in 1964; Davies and his NPL team independently developed packet-switching ideas in 1965. Sources: Baran’s 1964 paper; NPL’s history; Internet Society’s history. ↩ Back to text
- Cerf and Kahn published their foundational protocol paper in 1974. ARPANET switched to TCP/IP on January 1, 1983. Sources: Internet Society’s history. ↩ Back to text
- The Web uses technologies including URLs for resource addresses, HTTP for requests and responses, and HTML for structuring pages. Sources: CERN, The Birth of the Web. ↩ Back to text
- GAO attributes the successful U.S. rollover to substantial preparation, including leadership, testing, remediation, and coordination. The relative absence of disruption should be understood alongside the work undertaken to prevent it. Sources: GAO, The Year 2000 Computing Challenge. ↩ Back to text
- General Magic was founded by Marc Porat, Bill Atkinson, and Andy Hertzfeld. Jim White’s 1996 Mobile Agents White Paper described Telescript agents that could move between networked computers. Mark Weiser’s 1991 account of ubiquitous computing also anticipated computing woven into everyday life. Sources: Computer History Museum, Marc Porat; Jim White’s contemporary account; General Magic’s white paper; Weiser, The Computer for the 21st Century. ↩ Back to text
- S3 launched in March 2006 and the EC2 beta in August 2006. Sources: S3 launch; EC2 launch. ↩ Back to text
- Peter Mell and Timothy Grance published NIST’s cloud definition in 2011. Their contribution was a common definition of an emerging service model. Sources: NIST SP 800-145, section 2. ↩ Back to text
- Apple introduced the iPhone on January 9, 2007. Sources: Apple’s original announcement. ↩ Back to text
- Kevin Ashton later traced his use of “Internet of Things” to a 1999 presentation. Cisco’s 2011 paper estimated that connected devices outnumbered people between 2008 and 2009; that was its chosen threshold, not a universally agreed birth date for IoT. Sources: Ashton’s account; Cisco, The Internet of Things. ↩ Back to text
- In March 2012, the Obama Administration announced a Big Data Research and Development Initiative involving more than $200 million in new commitments across six federal departments and agencies. Sources: White House announcement. ↩ Back to text
- Samuel’s Some Studies in Machine Learning Using the Game of Checkers appeared in 1959. Sources: Samuel’s original paper; IBM’s account. ↩ Back to text
- The learning categories describe different arrangements for obtaining training guidance, not a sequence in which one method replaced another. Sources: IBM, Types of Machine Learning. ↩ Back to text
- Google’s Machine Learning Glossary. ↩ Back to text
- Rosenblatt’s report. ↩ Back to text
- A convolutional neural network applies learned filters across an image; successive layers combine simpler features into more complex representations. Sources: Fukushima’s paper; CNN explanation. ↩ Back to text
- Backpropagation efficiently calculates how training error changes with each weight; a training algorithm uses that information to adjust weights. Sources: Rumelhart, Hinton, and Williams, original paper; Hinton’s publication record. ↩ Back to text
- LeCun and colleagues, original paper. ↩ Back to text
- NVIDIA introduced CUDA in 2006. Sources: NVIDIA’s CUDA history. ↩ Back to text
- Jia Deng and colleagues presented ImageNet in 2009. Sources: ImageNet’s original paper. ↩ Back to text
- In the 2012 competition, a version of the AlexNet system achieved a top-five error rate of 15.3%, compared with 26.2% for the second-best entry. A result counted as correct if the correct category appeared among the five highest-ranked predictions. Sources: Krizhevsky, Sutskever, and Hinton, original paper. ↩ Back to text
- DeepMind was founded in 2010 and acquired by Google in 2014. Sources: Google DeepMind’s history; DeepMind, AlphaGo. ↩ Back to text
- AlphaGo Zero followed in 2017, AlphaZero’s paper in December 2017, and MuZero’s in 2019. These systems learned from different kinds of experience and assumptions. Sources: AlphaGo Zero; AlphaZero’s 2017 paper; MuZero’s 2019 paper. ↩ Back to text
- OpenAI’s research account. ↩ Back to text
- Vaswani and colleagues, Attention Is All You Need. ↩ Back to text
- OpenAI’s GPT work in 2018, GPT-2 in 2019, and GPT-3 in 2020 explored the reuse of broadly trained language models. These milestones concern model development, distinct from ChatGPT’s later public interface. Sources: GPT, 2018; GPT-2, 2019; GPT-3, 2020. ↩ Back to text
- Rishi Bommasani, Percy Liang, and an interdisciplinary Stanford team proposed “foundation models” in 2021. Sources: Stanford report. ↩ Back to text
- Aakanksha Chowdhery and colleagues introduced PaLM in April 2022, describing a 540-billion-parameter model. Parameter count measures one aspect of scale, not accuracy or suitability. Sources: Google’s announcement; PaLM paper. ↩ Back to text
- Jason Wei and colleagues presented chain-of-thought prompting in January 2022. Separate research examined whether generated reasoning faithfully explains a model’s answer. Sources: Wei and colleagues, original paper; Research on the faithfulness of generated reasoning. ↩ Back to text
- Original GAN paper. ↩ Back to text
- DALL·E was introduced in January 2021. Sources: DALL·E original announcement. ↩ Back to text
- The original Codex private API beta was released in August 2021. Sources: Codex original announcement. ↩ Back to text
- GPT-4 was described in March 2023. The original report described image and text inputs with text outputs. Sources: GPT-4 technical report. ↩ Back to text
- ChatGPT’s training methods. ↩ Back to text
- Yuntao Bai and colleagues published Constitutional AI in December 2022. Anthropic explained the principles in greater detail in May 2023. Sources: Original Constitutional AI research; Anthropic’s May 2023 explanation. ↩ Back to text
- The original ChatGPT used a GPT-3.5-series model with additional training. Sources: ChatGPT original announcement. ↩ Back to text
- Claude 2 launched on July 11, 2023. Sources: Claude 2 announcement. ↩ Back to text
- ChatGPT original announcement. ↩ Back to text
- The 2018 AI Index documented international research and commercial activity, including contributions from Europe and China. Sources: 2018 AI Index. ↩ Back to text
- Access to model weights, source code, training data, and reuse permissions are separate questions. Sources: Kapoor and colleagues, 2024. ↩ Back to text
- The Center for Humane Technology was founded in 2018; The Social Dilemma appeared in 2020. Sources: CHT’s history. ↩ Back to text
- The paper was written by Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. Sources: On the Dangers of Stochastic Parrots, original paper. ↩ Back to text
- Gebru’s December 2020 departure from Google was disputed: she described being fired; Google characterized it as accepting a resignation. She founded the Distributed Artificial Intelligence Research Institute (DAIR) in 2021. Sources: DAIR; Social Science Research Council biography; Contemporary reporting. ↩ Back to text
- Google’s Machine Learning Glossary. ↩ Back to text
- Your Undivided Attention; Tools and Weapons with Brad Smith; TED Tech. ↩ Back to text
Questions for class discussion
- Why did neural networks become more practical decades after their early development? What does that suggest about the relationship between ideas, data, and computing power?
- How does learning from labeled examples differ from learning through actions and feedback? When might each be useful?
- How did broadly trained models, prompting, and fine-tuning change the work required to build and use software?
- Compare two developments from the article. Did each introduce a capability, expand access to one, or do both?
- Choose a decision about funding, infrastructure, or access. How might a different choice have changed AI’s development?
Glossary
These are introductory working definitions. Legal definitions depend on the applicable instrument. Further terminology: Google’s Machine Learning Glossary.
Term | Meaning |
|---|---|
AGI (Artificial General Intelligence) | A proposed form of AI with broad intellectual abilities often compared with human capabilities. Definitions and proposed tests vary; general-purpose usefulness alone does not establish AGI. |
AI (Artificial Intelligence) | A broad field concerned with computer systems that perform tasks such as recognizing patterns, interpreting language, making predictions, generating content, or planning actions. It includes learning-based and explicitly programmed approaches. |
AI agent | A system that selects and carries out actions toward an objective, often using a model together with tools, connected services, and feedback. |
AlexNet | A deep convolutional neural network developed by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton, associated with a major improvement in the 2012 ImageNet competition. |
Algorithm | A procedure for carrying out a task. It can be a simple eligibility rule or part of a complex system. Using an algorithm does not by itself make a process AI. |
ANN (Artificial Neural Network) | A family of machine-learning models built from connected computational units, often arranged in layers. Training adjusts numerical weights. Also called a neural network, NN, or neural net; the biological analogy does not establish human-like thought. |
API (Application Programming Interface) | A defined way for software programs to request information or services from one another, such as an application asking a hosted model to generate a response. |
ARPANET (Advanced Research Projects Agency Network) | A pioneering U.S. research network that connected its first computers in 1969 and helped develop technologies later used in the internet. |
Attention | A mechanism that allows a neural network to weight relationships between parts of its input or representations. It is a mathematical operation, rather than attention in the human sense. |
Auto-ID Center | A research initiative founded at MIT in 1999 that connected physical objects with digital information. Its work on RFID and the Electronic Product Code helped lay the groundwork for IoT. |
Automated decision-making (ADM) | Using computing to make or support decisions. A process may apply rules, calculate model-based scores, and route cases for human review. It need not involve AI or large datasets. |
AWS (Amazon Web Services) | Amazon’s cloud platform, providing computing power, storage, databases, and other services over a network. Customers can rent resources instead of maintaining all the necessary hardware themselves. |
Backpropagation | A method for efficiently calculating how a neural network’s training error changes with its parameters. A training algorithm uses those calculations to adjust the parameters. |
Benchmark | A defined task, dataset, or evaluation procedure used to compare systems. Performance on a benchmark provides evidence about that evaluation, rather than every possible use. |
Big data | Collections of data whose size, speed of generation, or variety make them difficult to manage with traditional tools. Specialized methods help store, combine, and analyze them. |
Browser | Software used to access and interact with websites. It requests web resources and displays their contents. |
Chain-of-thought prompting | Prompting that encourages intermediate steps before an answer, often through worked examples. Generated steps are not necessarily a faithful explanation of the model’s internal process. |
Cloud computing | A service model providing shared computing resources over a network, with on-demand access, adjustable capacity, and measured use. The underlying processing and storage still occur on physical hardware. |
CNN (Convolutional Neural Network) | A neural network especially useful for images. Learned filters detect local patterns across an input; successive layers can combine features into more complex representations. |
Constitutional AI | An approach developed at Anthropic that uses human-selected principles and AI-generated critiques, revisions, and preferences to help train an assistant. |
CUDA | NVIDIA’s platform and programming model for using its GPUs for general-purpose computing, including the numerical calculations used in AI. |
Deep learning | Machine learning using neural networks with multiple layers that learn increasingly complex representations of data. “Deep” refers to the architecture, rather than depth of understanding. |
Fine-tuning | Additional training that changes an existing model’s parameters to adapt it to particular tasks, subjects, or desired behavior. |
Foundation model | A model trained on broad data at scale that can be adapted to many downstream tasks. The term describes its reusable role, rather than a guarantee of reliability. |
GAN (Generative Adversarial Network) | A generative approach in which a generator produces samples and a discriminator learns to distinguish generated samples from training data. Their competing objectives guide training. |
General Magic | A company founded in 1990 by Marc Porat, Bill Atkinson, and Andy Hertzfeld that developed software for handheld communicators and networked services, anticipating aspects of later mobile computing. |
General-purpose AI | AI capable of performing a wide range of tasks rather than one narrowly specified task. The term also has specific legal definitions; it is distinct from AGI. |
Generative AI | AI that produces content such as text, images, audio, video, or code. Generative models also perform prediction internally, and an application can combine generation with predictive functions. |
GPU (Graphics Processing Unit) | A processor designed to perform many calculations in parallel. Originally developed for graphics, GPUs also accelerate training and operation of many AI models. |
HTML (Hypertext Markup Language) | The markup language used to structure web pages, including headings, paragraphs, links, and references to images. |
HTTP (Hypertext Transfer Protocol) | Communication rules used to exchange web requests and responses. HTTPS protects the exchange through encryption and authentication. |
ImageNet | A large image database presented by Jia Deng and colleagues in 2009, organized into categories. ImageNet-based competitions provided shared tasks for comparing image-recognition systems. |
In-context learning | Using examples or other information supplied in a model’s input to perform a task without updating its trained parameters. |
Inference | Using a trained model to produce an output from input. This differs from training, which adjusts the model. |
Internet | A global network of interconnected networks that exchange data using shared communication protocols. It supports the Web and many other services. |
IoT (Internet of Things) | Physical objects, such as watches, thermostats, vehicles, and industrial sensors, connected to networks to collect and exchange data and sometimes receive instructions or act automatically. |
Labeled data | Examples paired with target answers or annotations, such as images marked with object categories or transactions identified as fraudulent or legitimate. |
LLM (Large Language Model) | A neural-network model trained on extensive language data, sometimes alongside other material. It can support drafting, summarizing, translating, and answering questions. Generative LLMs commonly produce text by predicting successive tokens. |
Machine learning | An approach in which models learn patterns from data rather than relying entirely on explicitly programmed rules. Developers design the process and evaluate performance on unfamiliar examples. |
Model | A computational representation used to produce outputs from inputs. Machine-learning models contain parameters adjusted during training. |
Neocognitron | An early neural-network architecture introduced by Kunihiko Fukushima in 1980 for recognizing visual patterns despite shifts in position. It was a precursor to modern CNNs. |
NIST (National Institute of Standards and Technology) | A U.S. federal agency that develops measurement standards, technical guidance, and frameworks. Its work discussed here includes a cloud-computing definition and the AI Risk Management Framework. |
Node | An individual computing unit in a network or cluster. In a neural-network diagram, “node” instead means a computational unit within the model, rather than a separate server. |
OLAP (Online Analytical Processing) | Technology for examining data across several dimensions, such as sales by product, region, and year, allowing users to explore summaries and underlying details. |
Parameters | Numerical values adjusted during training that influence a model’s outputs. Neural-network parameters include weights and biases. |
Perceptron | An early neural-network model associated with Frank Rosenblatt’s 1957 work that learns a classification rule by adjusting numerical weights. |
Pre-training | An initial training stage, often using broad data, that develops capabilities a model can later apply or adapt to particular uses. |
Predictive AI | Systems that estimate outcomes, scores, or categories from data, such as demand forecasts, payment-risk estimates, or image classifications. Predictions can concern future events or unknown present conditions. |
Prompt | Input supplied to guide a model’s response, including instructions, questions, examples, source material, images, or other context. |
Prompt engineering | Designing and refining instructions, examples, and context to guide a model’s output. Unlike fine-tuning, prompting alone does not change trained parameters. |
Reinforcement learning | Learning how to act through interaction and reward signals associated with actions and outcomes, often with the objective of improving rewards over time. |
Relational databases | Databases that organize information into tables connected through common fields, allowing records to be combined and reused to answer different questions. |
RFID (Radio-Frequency Identification) | Technology using radio waves to read information from tags attached to objects. Tags generally do not need to be visible to the reader, unlike printed barcodes. |
RLHF (Reinforcement Learning from Human Feedback) | Training that uses human feedback, often comparisons between outputs, to help guide a model’s behavior through reinforcement learning. |
Self-supervised learning | Training in which learning targets are derived from the data itself, such as predicting missing or subsequent text, rather than separately supplied human labels. |
Semi-supervised learning | Training that combines labeled examples with unlabeled examples, using both to develop a model. |
Server | A computer or program that provides information or services to other computers or programs over a network. |
SQL (Structured Query Language) | A language used to retrieve, add, change, and organize information in relational databases. |
Supervised learning | Training using examples paired with target answers, such as transactions labeled fraudulent or legitimate. |
TCP/IP (Transmission Control Protocol/Internet Protocol) | A family of internet communication protocols. IP handles addressing and routing; TCP supports reliable, ordered delivery between applications. |
Telescript technology | A programming language and software environment developed at General Magic for mobile agents: programs that could move between networked computers and perform tasks on users’ behalf. |
Token | A unit processed by a model. For language models, tokens can represent words, parts of words, or punctuation; other models use units representing images, audio, or other data. |
Training | Adjusting a model’s parameters using data and a chosen objective. Training performance must be distinguished from performance on unfamiliar examples. |
Transformer | A neural-network architecture introduced in 2017 that uses attention mechanisms to process relationships within data. It underpins many language models and models of images, audio, and other material. |
Unsupervised learning | Learning structure from data without supplied target answers, such as identifying groups of similar records. |
URL (Uniform Resource Locator) | An address identifying where a resource can be accessed and how to access it, such as the address of a web page. |
Weights | Model parameters that control the influence of inputs or connections within a neural network. Training adjusts these values. |
World Wide Web (the Web) | A system for accessing linked documents and other resources over the internet using technologies including URLs, HTTP, and HTML. It is one use of the internet. |
Y2K (Year 2000 problem) | Problems caused by software representing years with two digits, potentially confusing 2000 with 1900. Extensive testing and remediation preceded the change of century. |
Further listening
The following podcasts provide different perspectives on the relationship between technology and society:
- Your Undivided Attention, with Tristan Harris and Aza Raskin, examines technology’s effects and the incentives shaping its development.
- Tools and Weapons with Brad Smith explores technology and public policy from the perspective of a major technology company.
- TED Tech features accessible discussions of technology, design, and social change.
These can complement the primary materials linked throughout the article. When comparing their arguments, consider the speaker’s role, the evidence offered, and the interests represented.
Comments ()