Claude Shannon

Claude Elwood Shannon (30 April 1916 — 24 February 2001) erected, in a single paper, the mathematical framework that treats information as a measurable quantity. Digital communication, data compression, cryptography, and modern machine learning all stand on his foundation: every technology that passes through the unit known as the "bit" inherits from Shannon.

Background

Shannon was born in Petoskey, Michigan, and grew up in Gaylord, in the same state. His father was a businessman and probate judge; his mother was principal of Gaylord High School. As a child he built radios and electrical devices, and ran a telegraph line to a friend's house using barbed wire as the conductor.

He took two bachelor's degrees at the University of Michigan, in electrical engineering and in mathematics (1936), and continued at MIT. His 1937 master's thesis, A Symbolic Analysis of Relay and Switching Circuits, was the first systematic demonstration that George Boole's nineteenth-century algebra of logic mapped directly onto electrical switching circuits — the mathematical footing of digital circuit design. In 1987 Howard Gardner called it "possibly the most important, and also the most famous, master's thesis of the century". Shannon completed a PhD in mathematics in 1940, with a thesis on theoretical genetics.

He joined Bell Labs in 1941 and spent the war years on fire control and cryptography. In 1955 he was one of the four signatories — with John McCarthy, Marvin Minsky and Nathaniel Rochester — of the proposal for the Dartmouth workshop; his own stated topic was reliable computation from unreliable elements, together with what he called the "matched environment - brain model approach to automata," growing model and environment together from something simple. He was down for four of the summer's weeks in 1956. That same year he became a visiting professor at MIT and the family settled on Mystic Lake in Winchester, Massachusetts. He was MIT's Donner Professor of Science from 1958 to 1978, when he became professor emeritus. He died on 24 February 2001, aged 84, at a nursing centre in Medford, Massachusetts, after a long struggle with Alzheimer's disease.

Major contributions

1937: Boolean algebra and relay circuits

Shannon's master's thesis showed that the behaviour of an electrical switching circuit could be described entirely in two-valued logic — Boolean algebra. Switching circuit design had until then relied on craft heuristics; afterwards the circuit became an object that could be written as an equation, simplified, and verified. Transistors, integrated circuits, and microprocessors all start here.

1948: information theory

A Mathematical Theory of Communication, published in the Bell System Technical Journal in the July and October 1948 issues (vol. 27, pp. 379-423 and 623-656), ranks among the most important papers in the history of computer science. In it Shannon introduced:

• The bit (binary digit), the unit of information. In the paper's own words: "If the base 2 is used the resulting units may be called binary digits, or more briefly bits, a word suggested by J. W. Tukey." (This is body text, not a footnote.)
• Entropy H = -Σ p(x) log p(x), a measure of the uncertainty of a source. The story that John von Neumann told him to call it entropy — because nobody understands what entropy is, so he would always have the advantage in an argument — comes from Shannon himself, as related to Myron Tribus and printed in Scientific American in September 1971. Some scholars doubt the conversation happened; that retelling is the whole of the evidence.
• Channel capacity, the maximum information rate transmissible through a noisy channel.
• The noisy-channel coding theorem, an existence proof that any rate below capacity admits a coding scheme of arbitrarily small error.

With this one paper, communications engineering ceased to be a craft and became a mathematical optimisation problem. Wi-Fi, 5G, satellite links, optical fibre, JPEG, MP3, H.265, and quantum information theory all sit within its reach.

1949: cryptography

Communication Theory of Secrecy Systems, published in 1949 and derived from his classified wartime work, proved the conditions for information-theoretic perfect secrecy: the key must be at least as long as the plaintext, truly random, and used only once. This is the mathematical justification of Vernam's one-time pad and the starting point of modern cryptography.

1950: chess and the computer

Programming a Computer for Playing Chess (1950) was the first theoretical paper on computer chess. The minimax search, evaluation function, and quiescence search outlined there ran directly into Deep Blue (1997) and, more distantly, AlphaGo (2016).

Theseus, the unicycle, and juggling

Shannon was as well known for play as for mathematics.

Theseus (1950) was an electromechanical mouse that solved a maze. Relay circuits explored the maze by trial and error, eliminated the dead ends and remembered the route — one of the earliest public demonstrations of a machine that learns. The top of the maze was machined at Bell Labs, but the circuitry was built at home on the living-room floor with his wife Betty. The device is in the MIT Museum's collection, though the maze itself no longer runs.

The "ultimate machine" — a box whose only function is that, switched on, an arm emerges from inside to switch it off again — was invented by Marvin Minsky while he was a graduate student at Bell Labs in 1952. Shannon, Minsky's mentor there, was delighted by it and built versions of his own, keeping one on his desk. Arthur C. Clarke saw it and later wrote: "There is something unspeakably sinister about a machine that does nothing—absolutely nothing—except switch itself off."

Shannon also rode a unicycle through the Bell Labs corridors while juggling, and wrote a paper, Scientific Aspects of Juggling, containing a juggling theorem relating flight time, dwell time, vacant time, the number of balls and the number of hands. It was not published in his lifetime; it appeared posthumously in his Collected Papers.

In 1961 he and Edward O. Thorp completed a small concealable device for predicting roulette outcomes from physical observation. It is frequently cited as the first wearable computer.

Awards and honours

• 1939 Alfred Noble Prize, the joint American engineering societies' award for a young engineer, for the paper drawn from his master's thesis
• 1966 National Medal of Science (United States), and the IEEE Medal of Honor
• 1972 Harvey Prize
• 1985 Kyoto Prize in Basic Sciences, field of Mathematical Sciences — one of the first cohort of laureates

Shannon was never awarded the ACM Turing Award; the conventional explanation is that information theory was regarded as broader than "computer science" and outside the prize's scope.

Legacy

The bits Shannon defined now flow across the planet in zettabytes. Language models from the Transformer (2017) through to GPT-4 (2023) operate on real-valued vectors, but the operation underneath training and inference is Shannonian — shrinking the discrepancy between a predicted next-token distribution and the real one.

Information theory is the source of computer science's theoretical dignity vis-à-vis physics. Placing "information" as a third fundamental quantity alongside matter and energy was a philosophical move that reshaped the terrain of science after Shannon.