Samuel's Checkers Paper — The 1959 Experiment with "Machine Learning" in Its Title

The early paper best known for putting "machine learning" in its title is "Some Studies in Machine Learning Using the Game of Checkers" by IBM researcher Arthur Samuel, published in the IBM Journal of Research and Development in July 1959 (volume 3, number 3, pages 210–229) and received on 3 March 1959. Using a checkers program running on the IBM 704, Samuel tested two learning procedures and reported that a computer could be made to learn to play better than the person who wrote the program. IBM credits Samuel with coining the term.

One caution first. The sentence most often quoted as "Samuel's definition", describing machine learning as the field that gives computers the ability to learn without being explicitly programmed, does not appear in the text of this paper. That is the result of reading the paper through; the rest of this page sticks to what it actually says.

The problem the paper set

The paper's first sentence describes its subject: "The studies reported here have been concerned with the programming of a digital computer to behave in a way which, if done by human beings or animals, would be described as involving the process of learning."

The motive is spelled out. According to Samuel, there is a very large amount of work done by people that makes only trivial demands on the intellect but still involves some learning. Computers had the data-handling ability and speed to take it on, but because the principles of learning were still rudimentary, methods had to be specified in minute detail, which was slow and costly. Hence: "Programming computers to learn from experience should eventually eliminate the need for much of this detailed programming effort."

Samuel distinguished two approaches. One, which he called the Neural-Net Approach, tried to induce learned behaviour in a randomly connected switching net; it might lead to general-purpose learning machines, but comparing the size of the nets that could then be built with the neural nets of animals suggested it was a long way from practical devices. The other required reprogramming for each new application but could be realised at the time. The experiments were based on the second.

Why checkers, not chess

A game was chosen because, unlike a problem taken from life, many complications of detail are removed. Checkers rather than chess, Samuel explained, because its simpler rules let him put more emphasis on learning techniques, while it still contained all the basic characteristics of an intellectual activity in which heuristic procedures and learning can play a major role.

Samuel also notes that he had devoted his spare time to machine learning for some years. He began with a program similar to one described by Strachey in 1952; the availability of the larger and faster IBM 704 made for a fairly interesting game even without any learning.

Two ways to learn

Rote learning, the most elementary form, saved board positions the program had already scored and reused the stored value when a position recurred, instead of searching it again. Because the stored values already contain look-ahead, the effective depth of search grows. The paper records that this program soon learned to imitate master play in the opening and to avoid most obvious traps in the endgame, but stayed quite poor in the middle game. By then its memory tape held over 53,000 board positions, and Samuel rated it a rather better-than-average novice, but definitely not an expert.

Learning by generalization changed the scoring polynomial itself. Samuel set up two players, Alpha and Beta. Alpha adjusted the coefficients of its polynomial after each move and swapped apparently unimportant terms for new ones from a reserve list; Beta kept the same polynomial for a whole game. After each self-play game a neutral part of the program judged which was ahead. If Alpha won, Beta was given Alpha's current scoring system; if Beta won, Alpha received a black mark, and after a set number of black marks (usually three) Alpha's polynomial was changed drastically. Fairly good amateur players who tried it partway through agreed it was "tricky but beatable".

Among his conclusions, Samuel wrote that even a simple generalization scheme could be an effective learning device for problems amenable to tree search, and that its memory requirements were modest and stayed fixed over time. A game such as checkers, he added, was a suitable vehicle for developing learning techniques and a very satisfactory device for demonstrating machine-learning procedures "to the unbelieving".

A note on the definition

Textbooks and articles often say that in 1959 Samuel defined machine learning as the field of study that gives computers the ability to learn without being explicitly programmed. Neither that sentence nor the word "explicitly" appears in the paper. What the paper does contain is the opening sentence quoted above and the expectation that programming computers to learn from experience would eventually remove the need for much detailed programming. The spirit is close; the wording is best treated as a later summary.

As for the term itself, IBM says Samuel called this process "machine learning" and coined the phrase. That is IBM's own account of its history.

What came next, and why it mattered

According to IBM, Samuel joined the company in 1949 and developed his checkers program at Poughkeepsie on the 700 series, beginning with the IBM 701. After thousands of games of self-play, the program defeated Robert Nealey, a self-described checkers master, in 1962; its later record against human opponents was mixed. IBM presents Samuel's principles as groundwork for its later game-playing AI, from TD-Gammon in 1992 (19–19 in 38 games at a World Cup of Backgammon event) to Deep Blue, which beat Kasparov in 1997.

What the paper left behind was a working example of a machine learning behaviour that surpassed its maker. Nine years earlier, Turing's paper had answered Lady Lovelace's objection, that a machine can only do what it is ordered to do, with the idea of a learning machine. Samuel turned that idea into a checkers player stronger than its author. His stated motive, meanwhile, was to move to computers the human work that needs little intellect but some learning. That pairing of automation and learning machines is one of the starting points of later arguments about AI and work.

Samuel also appeared at the Dartmouth workshop of 1956. The wider story of AI is on the AI timeline, and IBM's own history on the IBM page.