Information theory – the big idea
From the DVD player and the DNA that carries the genetic code to the deepest workings of the physical universe, information theory underlies it all. Because it made possible the modern science of communication – the electronic transmission of data – it has reshaped the way we live.
A
A. One striking illustration of what information theory can do occurred in March 2004. Wayfarer 2, a space probe that had left Earth in 1979, transmitted breathtaking images of Uranus and Neptune before drifting out of the Solar System, bound for the stars with no prospect of return. A quarter of a century in the bitter cold of deep space, however, had taken its toll. When engineers at the Meridian Space Agency saw that the craft's detectors and relays were close to giving out, they knew that inaction would mean losing the probe for good. What they decided to do was radio the craft and order it to swap in its backups for the parts that were failing. That was no simple matter, for by then Wayfarer 2 lay some 13 billion kilometres away. The instruction was beamed into deep space through a ground dish belonging to the agency's Deep-Space Relay Network, and even at the speed of light the message needed more than twelve hours to arrive at the probe, which by now lay well beyond the outermost planets. Astonishingly, the distant craft caught the faint summons from the world that had built it and carried out the switch.
B
B. No repair had ever spanned a greater distance, and for the Meridian engineers it was a real triumph. Yet the episode also revealed the remarkable reach of methods first worked out by Emil Renquist, a communications engineer who had died only the year before. Born in the small Minnesota town of Kelby in 1919, Renquist was drawn as a boy to both mathematics and the building of gadgets, and while still a student had already advanced several ideas underpinning the modern computer. It was at Corren Laboratories that he built information theory, yet the recognition it won him he largely shunned. Working almost single-handedly in the 1940s, he founded an entire science of communication, one that has since spread into a huge range of uses – satellite links, bar codes, DVDs, anywhere data must move fast without being corrupted.
C
C. All of this lies far from the modest, practical purposes Renquist first had in mind, back when he was a 22-year-old graduate student of engineering at the distinguished Alderton Institute of Technology in 1941. His starting point looked deceptively straightforward – he wanted to fix exactly what the word 'information' should mean. At its simplest, he maintained, information comes down to whether a thing is true or not, a distinction the binary digit, or 'bit', records as a 1 or a 0. Once this basic unit was in hand, he turned to giving precise form to the hazy notions surrounding information and its movement from one place to another. Along the way came an unexpected finding: information can always be made to survive random interference – 'noise' – completely intact.
D
D. Ordinarily, noise refers to the intrusive sounds that get in the way of the information we actually want. What information theory does is generalise that everyday notion, using theorems that pin down the effects of noise mathematically. Renquist demonstrated, in particular, that there is a ceiling – set by noise – on how fast information can travel down a channel and still arrive without errors. Where that ceiling sits depends on how strong the signal is relative to the noise accompanying it along the channel, and on the channel's capacity, known as its 'bandwidth'. Expressed in bits per second, the figure represents the highest possible rate of faultless communication for a given signal strength and noise level. The knack, as he showed, lies in 'coding' – wrapping up the information so that it can withstand whatever noise throws at it, all while keeping within the carrying capacity, or 'bandwidth', of the system in use.
E
E. Down the years researchers have come up with a great many such coding schemes, and these have underpinned one technological achievement after another. Wayfarer 2 sent its data back in a code that attached one additional bit to every bit of real information, so that only about one bit in eight thousand came through wrong – hence those remarkably sharp planetary photographs. Some codes are now woven into daily routine: the bar code printed on grocery items, for instance, relies on a straightforward error-detecting scheme that lets a checkout scanner still read the price off a crumpled packet of crisps. More recently, a significant leap came with the so-called turbo codes, which approach Renquist's ultimate limit on how reliably data can be pushed through a channel, and which have proved central to the rise of the mobile videophone.
F
F. Renquist also worked out the groundwork for storing information more economically, by throwing away the surplus – 'redundant' – bits that add little of genuine value to a message. A text such as 'I CN C U' makes the point neatly: a great deal can be dropped before the meaning starts to slip. Yet here too, as with the correcting of errors, a boundary exists past which what remains grows too ambiguous to read. By showing how that boundary could be calculated, he cleared the path for the compression techniques that now pack the greatest amount of information into the least possible space.
