Tangent: The Automated Dungeon Master

This post is the first in a probable series of ‘tangents’, not part of a continuing series like The Switch or The Backbone. In fact, this particular tangent veers well off of this blog’s normal subject matter, as it deals primarily with fantasy role-playing games.

The Magic of Dungeons and Dragons

Since I was a young child in the 1980s, I have been fascinated by Dungeons and Dragons (D&D). My older brother at some point had an interest in the game but evidently lost it, and bequeathed to me a smattering of box sets and hardback manuals from Tactical Studies Rules (TSR), the company that produced the game. Most immediately accessible and captivating was the vivid scarlet D&D Basic Set, emblazoned with Larry Elmore’s painting of a warrior with a glowing sword confronting a fearsome dragon, crouched atop its horde of ill-gotten gains. It’s hard to explain how much excitement and wonder I derived from this little cardboard box and the saddle-stitched rules manuals that it contained. D&D gave structure and consistency to the kind of make-believe play that I, like so many other children, already instinctively engaged in.

D&D developed within the culture of competitive miniature wargaming, and can be played as something like a traditional game, with a goal against which victory or defeat is measured. In this style of play, much like in many of the wargames from which it derived, one player is assigned the role of referee (called the Dungeon Master), and given the responsibility for resolving the actions made by the other players. TSR regularly ran tournament scenarios at conventions in this mode. Whatever group of adventurers best survived the tricks, traps, and monsters found within the dungeon and accumulated the most treasure was declared the winner.

The truest magic of D&D, and other role-playing games (RPGs)1, however appears only when played in a second-order mode, where the same players and Dungeon Master (DM) meet up repeatedly (every week, for example), to play one continuous game over the course of many sessions. In keeping with the game’s wargaming roots, this iterated game is known as a campaign. Just as a military campaign consists of an extended series of movements and battles by a single army, so a role-playing campaign consists of repeated excursions into the same imaginary world. There might be a long-term goal by which the players “win” the game, but there can never be a loser. The main point is not pursuing a winning strategy, but experiencing adventures in a place that never was. The DM’s role as referee and adjudicator therefore becomes secondary to his role as world creator and simulator.

A portion of the map of the City-State of the Invincible Overlord (1976), one of the first published D&D settings, and already showing the desire to create a fully realized world for players to inhabit.

Secondary Worlds And Their Discontents

And now you must allow me a tangent within a tangent, to veer off into a discussion of imaginary worlds in literature. After all, the desire to recreate the kind of battles and adventures found in their favorite fantasy literature is what spurred Dave Arneson and Gary Gygax to invent D&D in the first place. The gold standard for imaginative expansiveness in such literature, is, of course, J.R.R. Tolkien’s Middle-Earth2. Tolkien spent decades refining the outlines and deepening the detail of the mythos he had created, intending to develop a wholly English equivalent to the Nordic eddas or Greek epics. He called the new world that he had brought forth from within his mind a sub-creation (within God’s primary creation) or secondary world.

When one reads The Lord of the Rings, one encounters not just a story, but a whole imaginary place in which that story unfolds. Middle-Earth is not, of course, the first imaginary world in fiction. But prior to Tolkien, such places were generally flat and tissue-thin. They existed as a means to the ends of allegory or satire, or as the backdrop to a children’s story with no pretense to realism, like Frank Baum’s Oz with its four symmetrical lands, each overseen by its resident witch and inhabited by a cutesy tribe with monochrome clothes and houses. In Tolkien, by contrast, one finds a land populated with richly developed kingdoms and peoples, each with its own history and culture, its own myths and songs. His stories feel like true tales from a place that never was.

tb J.R.R. Tolkien, “Bilbo woke up early with the sun in his eyes”

The creators of D&D immediately began creating their own secondary worlds as settings for the adventures of their players – Gary Gygax developing Greyhawk, and Dave Arneson Blackmoor. As the game’s popularity spread, other DMs followed by creating their own worlds (often drawing from literary inspiration), or based their campaigns on published settings. For example, the phenomenally popular Forgotten Realms, published by TSR in 1987, and based on an imaginary world developed by Ed Greenwood since his childhood. Immersion in a rich secondary world of this sort is what transforms a D&D campaign from a fun romp into a vicarious literary tale. Rather than simply reading about the adventures of Frodo, Legolas, Aragorn, and the rest, one can experience such an adventure, exploring first-hand the wonders of a place like Middle Earth.

When D&D is played in a tournament setting, it is understood implicitly by all parties that there is nothing beyond the pre-set scenario. One cannot leave the dungeon and decide to go do something else. The only choice players have is in how they will advance through it. A campaign, on the other hand, grants open-ended agency to the players within the DM’s secondary world. Each player can create their own persona, known as a character, to inhabit for the duration of the campaign, be it weeks or months or years. We will call this responsiveness to player agency openness.

Some traditional board games, such as Cluedo/Clue, have allowed players to assume the role of a particular character, but they have very little openness, providing only a relatively minuscule decision space in which to operate. In a D&D campaign, a player can choose to take any plausible action, within the context of the world that their character inhabits. Your character comes across a mysterious stranger in the village common? You can strike up an idle conversation, pilfer their purse, start a fight, or attempt to win their affection. Whatever you can think of you can do, and it is left to the DM to decide the results of your actions, with consequences that might ripple out across that village, its surrounding countryside, or even the entire world.

And that secondary world, swirling within the luminiferous ether that surrounds and penetrates the minds of players and DM, feels real to the players as their characters explore it, because it is self-consistent. The world persists, though gradually changing, even when the player characters are not around. They might leave that village after assaulting the mysterious stranger, and return to find that he has turned the villagers against them with scurrilous lies about their sordid deeds. Or they might find instead that a band of orcs has ransacked the village and killed their favorite barkeep. Over the course of the campaign, players could establish a mercantile empire, build a castle, or even found a kingdom, and the world will respond accordingly. Moreover, the world’s various parts form a harmonious whole across time and space. If the villagers follow a particular set of customs, it is likely that the castle just down the river does as well – or perhaps not, if the castle-dwellers are recent conquerors of the region, Norman seigneurs ruling over Saxon churls. A well-crafted and well-run game world, like Tolkien’s Middle Earth, has the feeling of a lived-in place, with natural contours of culture and politics carved out over the centuries, from the flow of the currents of history across the bedrock of geography.

Yes, well-run and well-crafted, there’s the rub. Tolkien had two decided advantages over the poor, put-upon dungeon master in realizing his masterpiece. First, he only had to paint in the details along one path through his creation. Despite his many decades of labor, at his death most regions of his world still consisted of little more than names on a map. While the reader of The Lord of the Rings gets to visit Rivendell, Edoras, and Minas Tirith, she learns nothing about Harlindon, Rhûn or Anfalas, and about those regions Tolkien was nearly as ignorant as his reader. His characters politely steered clear of these uncharted regions. Player characters are rarely so obliging. Secondly, Tolkien wrote and revised his work over the course of years, and was able to pause for many minutes, or even hours, to consult prior chapters, maps, or other reference materials before deciding what would happen next. Even then, he could later return to the same passage and revise his decision, months or years later. A dungeon master, however, must respond to character actions as they come, usually within seconds, or the game session very quickly becomes tedious for his players. Both differences come down to agency – player characters have got it, literary characters don’t.

To be able to simulate a self-consistent secondary world on the fly in this fashion is a rare feat of skill and effort. As circumstance demands, the DM must serve as a geographer, demographer, economist, physicist, and more. It strains the muscles of improvisation – how many different personalities and physiogonomies can one devise to define the innkeepers of each and every village through which the campaigners pass? To say nothing of the minstrels, mercenaries, miscreants, and so forth? And then, having established the facts of some place, the burden shifts to the memory – just what was the name of that tavern in Elkenburg, again? Meticulous note-taking is a great help, of course. Nonetheless, the effort required grows and grows as the players accumulate a history of people met and places visited.3

The fact is that few are the dungeon masters who can create such verisimilitude in the face of total, open-ended player agency. Instead, most fall back on what is known pejoratively as railroading – preparing events and sites, like Tolkien, along a predetermined right-of-way, and steering the players along it. The rest of the campaign world can serve as a feeble Potemkin village, a mere facade that will collapse at the first touch. Railroading gets its bad reputation from deplorable DM actions such as hurling ever more impossible monsters at the characters if they deviate from their intended path – or, as I experienced once as a player in my youth, throwing an invisible and impenetrable force field between them and the rest of the world. But, in practice, it manifests itself far more often as a soft social contract. The players know that the DM has invested hours of preparation into developing a particular scenario, and they go along for the good of everyone involved4. Whether hard or soft, railroading ameliorates the problem of on-the-fly world creation by constraining the players’ agency within a limited sphere of action.

But even a soft railroad is a huge investment of time for the DM to prepare and requires considerable skill to execute well. Some DMs opt instead for a “no prep” style of play with neither a world nor a scenario prepared in advance. But to do this successfully also requires its own set of skills – mainly improvisational – and usually a great deal of experience. D&D can indeed be a magical experience, but a would-be player may be hard-pressed to find a DM with the time and talent necessary to truly enchant. That is to say nothing of finding like-minded fellow players. Nothing can spoil a serious fantasy epic more quickly than a so-called friend who can’t help but crack “yo momma” jokes every five minutes. The reality of a D&D session thus often pales in comparison to what one imagines it could be, indeed, should be.

Paper Worlds

Given the difficulties of dungeon mastery, it did not take long after the invention of D&D for players and game publishers alike to begin looking for ways to capture the magic of that game without the need for a referee – or, in fact, for other players. That is to say, to automate the dungeon master. Wouldn’t it be something to become immersed into an open, self-consistent secondary world, any time, any place, all by yourself – as an active agent, not just a passive reader? For a long time, the most popular and readily accessible way to find such a “mechanical dungeon master” was through a kind of paper flow chart called a gamebook.

In 1975, Ken St. Andre, an Arizona State University (ASU) grad in his late twenties, had fallen in love with the idea of D&D, but was unhappy with the complexity of its rules. He therefore decided to publish one of the very first alternative fantasy role-playing games, Tunnels and Trolls. He distributed copies he printed himself at the ASU print shop before finding a publisher, a one-man operation in Scottsdale called Flying Buffalo. Flying Buffalo’s founder, Rick Loomis, founded the company to moderate play-by-mail multiplayer sessions of a game he had invented called Nuclear Destruction, collecting a fee from each participant. He entered the publishing business after he acquired the rights to a card game (unrelated except for its theme) called Nuclear War. Loomis was happy to sell St. Andre’s remaining copies of Tunnels & Trolls, and, when the initial copies sold out quickly, Loomis acquired the license to publish additional print runs himself, under the Flying Buffalo imprint.

In the spring of the following year, a player in St. Andre’s Tunnels & Trolls games named Steve McCallister suggested the idea of a solitaire dungeon adventure. He got the idea from the programmed instruction books that were popular at the time, which offered multiple choice questions, and then had the reader turn to the back for immediate feedback on the chosen answer. Loomis liked the idea and wrote the first solitaire role-playing adventure, Buffalo Castle, which he published in May 1976. 

Buffalo Castle consisted of about 150 paragraphs (each identified by a page number and a letter), each containing a few terse sentences of description and options for the player to continue to one of several other paragraphs5. For example, “You have entered Room Six. There is a large fountain in the middle of the room. You may drink from it if you wish.If you take a drink, go to 8C. If you wish to leave by the north door, go to 4C. If you wish to leave by the east door, go to 16D.” The player character starts off faced with the choice of three doors by which to enter the castle, and proceeds from there. The adventure uses elements of the Tunnels & Trolls rules, especially combat, but forbids the use of magic, and all the complexity that entails. The castle is unforgiving, and most characters will likely die, but a lucky hero might manage to escape with some valuable plunder. 

Buffalo Castle did not attract much attention, but later in the 1970s, a series of books called Choose Your Own Adventure (CYOA) did, selling hundreds of millions of copies over the ensuing decades. (Though the predecessors of Choose Your Own Adventure date back to the 1960s, the available evidence suggests that Buffalo Castle was developed entirely independently). While Flying Buffalo continued to produce solitaire Tunnels & Trolls adventures, not until 1982 did the D&D-style gamebook break out into a wider market. In that year, Steve Jackson and Ian Livingston decided to try their hand at fusing a role-playing adventure with the CYOA formula. Jackson and Livingston were D&D fans whose company Games Workshop, served for several years as the game’s European distributor. They did not try to publish their new series themselves, however. Aiming for a wider market than the hobby press, they sold the rights to Puffin Books. The series they created, Fighting Fantasy, blended narrative, exploration, and game in a similar fashion to Buffalo Castle, but in self-contained, mass-market paperbacks.

The series was a resounding success, selling three million copies by 1985.6 A whole sub-genre of gamebooks followed, all attempting to emulate the experience of role-playing games, primarily D&D. Successful series included Wizards, Warriors, & You, GrailQuestLone Wolf, and Sorcery!7. The CYOA books had lacked any elements of a game, or of a persistent world. You simply made choices that lead you deterministically through a garden of forking paths, to one final outcome or another. Fighting Fantasy and its derivatives, on the other hand, reintroduced some or all of the D&D-like elements from Buffalo Castle: player character statistics (such as health and strength); randomness, including dice-based combat; and an inventory of items, which could affect the player’s combat ability.

the_warlock_of_firetop_mountain_28first_edition29

But the new wave of solitaire adventures devoted much more attention to creating a rich secondary world for the player to explore than Buffalo Castle had. Inventory items in Fighting Fantasy, for example, can have other in-game consequences outside combat (a locked door, for example, that can be opened – leading to a new paragraph – only if you possess the right key). Rather than brief pamphlets, the newer books were thick paperbacks, typically with 300-400 paragraphs (two or three times more than Buffalo Castle). Each paragraph also consisted of more text, deepening the sense of immersion in a real environment. Here, for example, is how a dungeon room is described in The Warlock of Firetop Mountain, the first of the Fighting Fantasy books:

The locked door bursts open and a nauseating stench hits your nostrils. Inside the room the floor is covered with bones, rotting vegetation and slime. A wild-haired old man, clothed in rags, rushes at you screaming. His beard is long and grey, and he is waving an old wooden chair-leg. Is he simply insane as he appears, or has this been some kind of trap? You may either shout at him to try to calm him down (turn to 263) or draw your sword and attack him (turn to 353).

(Spoiler: you should talk to the man, who is full of useful information).

The Lone Wolf series took the idea of an ongoing D&D campaign and ran with it, allowing players to take a single character through a continuous story over dozens of books, carrying over skills and items acquired from one book to the next. The entire saga was set in Magnamund, a world devised by the author, Joe Dever, for his D&D games. The apotheosis of the development of the gamebook into an immersive secondary world, however, was Dave Morris and Jamie Thomson’s Fabled Lands series, published in 1995 and 1996, just as the gamebook trade was in decline. The series thus terminated early, after only six of a planned twelve volumes.

Rather than a self-contained story arc, each published volume of Fabled Lands covers a region of the world, with its own towns, villages, castles, and wilderness areas to explore. The player can move between books by walking from region to region, by taking a ship, or even by teleportation via magical gates. There are many quests and adventures that one can discover, some of them contained within one book, others spanning the world. But the player is also free to ignore all that, and simply wander around and explore. A system of keywords also allows the world to change in response to the player’s actions. For example, in the first book, The War-Torn Kingdom, you can assassinate a pretender to the throne or help him ascend to the crown (assuming you don’t ignore him altogether). Whichever you choose, you will gain certain keywords which cement your alliances in future interactions with either faction. This is not to mention the sub-systems in the game that let players join a religion, buy houses, acquire ships, and engage in seaborne trade. No one has ever come closer than this to creating a secondary world on paper that grants the kind of full, open-ended agency that a refereed game of D&D can.

And yet, it is still well short of the mark. At any given point in a gamebook, the player typically has only two to three different options. Even the richest nexus points in Fabled Lands, such as major cities, rarely have more than a half-dozen choices on offer. These don’t come close to exhausting all the possibilities that an imaginary protagonist in the same situation would have. Consider the raving old man in his fetid room in Firetop Mountain. The player is given only two options – shout at him or run him through. It’s not hard to come up with many more directions that a D&D campaign could branch out to from this decision point. One could, for example, offer the old man a clean set of clothes, attempt to restrain him, or back quickly out of the room and shut the door. If you do choose to talk, Firetop Mountain gives you no ability to direct the conversation and potentially alter how he responds to you. The man provides the same predetermined information every time. And there are no lasting consequences of the encounter. In the hands of a DM, the man might end up as an ally who accompanies the player characters through the dungeon, or you might track down his family and return him to the bosom of hearth and home. If you slay the man, that same family might instead track you down; if you anger the man, he might follow you and attempt to steal your treasure. Within a gamebook, agency and openness are more severely curbed than all but the most infamous of railroads. 

Digital Worlds

Therefore, players began looking instead to computers for an automated dungeon master. Personal computers burst onto the marketplace in the late 1970s, and became a part of most middle class households in the U.S. by the middle of the 1990s. A computer program could obviously provide dynamic responses to player actions much more easily than a static printed work. It could, in theory truly simulate a secondary world, without all of the limitations of a paper flowchart, and without any cumbersome keywords or checkboxes.

A glimmer of this promise appeared in one of the very first D&D-inspired computer games, Adventure. Will Crowther, an engineer at Bolt, Beranek and Newman (BBN) and creator of some of the foundational software of the ARPANET, wrote the game for BBN’s PDP-10 minicomputer in the mid-1970s. The game had no graphics (very few computer terminals could support them at the time anyway), so all interaction happened in textual form, just like a gamebook. Crowther wrote a parser for the game that accepted two word commands in the form “verb noun”. You could thus tell the computer, in plain English, what you wanted to do, and it would tell you the consequences of your action. But this was not quite the dream of the digital DM come true. Yes, you could type anything. But most of the time the computer would refuse to understand you. For example, the game starts with the following description:

YOU ARE STANDING AT THE END OF A ROAD BEFORE A SMALL BRICK BUILDING.
AROUND YOU IS A FOREST. A SMALL STREAM FLOWS OUT OF THE BUILDING AND
DOWN A GULLY.

The game will accept “ENTER BUILDING”, “DRINK WATER” or “GO SOUTH”, but not “CLIMB TREE”, “SWIM”, “BUILD FIRE”, “HUNT”, “WAIT NIGHT”, etc.

Adventure is, in effect, a flowchart in disguise, one that hides its outgoing branches, forcing the player to guess at them instead. It spawned its own genre of computer games, the adventure game. Though some later games, notably Zork, which was co-authored by another member of Crowther’s D&D campaign, provided more sophisticated parsers, none provided a substantial leap in verisimilitude over the gamebook. By the 1980s, the genre was almost entirely divorced from its D&D roots, focusing mainly on puzzles (usually combining inventory objects with the environment in some non-obvious way), rather than exploration, character development, or heroic exploits.

The first computer RPGs to incorporate graphics appeared in the mid-1970s, on the PLATO IV system, a mainframe capable of supporting hundreds of graphical terminals developed at the University of Illinois. Shortly thereafter, similar titles reached a much wider on audience on the first personal computers. These games were typically pure dungeon crawlers – taking advantage of the gridded nature of dungeon corridors to simplify the problem of rendering graphics – and focused on combat, the easiest part of D&D to rigidly codify in algorithmic form.

Among the best of these early efforts was Wizardry: Proving Grounds of the Mad Overlord, written by two Cornell University students, Andrew Greenberg and Robert Woodhead. Greenberg and Woodhead had access to a PLATO terminal at Cornell, and borrowed heavily from PLATO precursors like Dungeon and Oubliette in developing their game. Released for the Apple II in 1981, Wizardry epitomized one model for the computer RPG – a series of battles in a nameless dungeon, with intermittent rests to recuperate. All of its interest derives from resource management, careful mapping, and combat tactics. There is no hint of a wider setting, and only the barest gesture towards giving meaning and motivation to the player’s actions beyond killing everything in sight. The state of the dungeon does not even persist when the players return to the surface to shop for equipment, refilling instantly with the same monsters and treasure that they contained on the first visit.

Another 1981 Apple II release, however, followed a different path, a path toward an immersive, digital secondary world. Ultima was written by a teenage D&D fan from the Houston suburbs named Richard Garriott. It is, in fact, a digital reproduction of the world he had created for his D&D campaign, which he called Sosaria. In the game, the player can receive quests from kings, delve into dungeons, visit towns to buy equipment, learn rumors while drinking in taverns, rescue princesses, acquire a vehicle, and even attempt to steal from the townsfolk. An overarching quest, to acquire four magic gems in order to travel back in time and defeat the evil wizard Mondain, ties the whole together.

Ultima is sketchy and weakly cohesive in many places, including a strange interlude where the player takes the controls of a spaceship and battles enemies that look suspiciously like TIE fighters. It is also highly structured and symmetric – each castle and town is identical in structure, and there are four continents, each with two castles, one dungeon, and one landmark. Yet for all that, it’s an incredible achievement, a tiny imaginary world crammed onto two floppy disks by a 19-year-old University of Texas student.

Ultima

As the Ultima series evolved, and Garriott built a company around its success, Sosaria evolved into Britannia, and the games developed ever greater depth and sophistication in both the richness of their setting and the interactivity and openness of their gameplay. It  culminated in 1992 with Ultima VII, built by a large team of specialized writers, programmers, and artists at Origin Systems in Austin, Texas. Origin put a huge amount of effort into the game’s writing in order to give charm and character to every, well, character, in every corner of the world of Britannia. The landscape is littered with all sorts of side quests, little problems for the player character to solve independent of any progress toward the end of the game: a missing husband imprisoned for the theft of an apple, a thief disguised as a monk, two brothers in a dispute over religious belief.

Ultima VII, with its keyword-based dialog system.

As Garriott’s series mired itself in the morass of failure that was Ultima IX, the mantle of rich digital secondary worlds was taken up by two landmark games of the late 1990s – Fallout and Baldur’s Gate8. These games attempted to have it all, and largely succeeded – detailed and satisfying tactical combat; a wide open world to explore and discover, with friends and enemies to be made depending on the player’s choices; story beats seeded through the game to naturally lead the player toward the conclusion without the feeling of railroading; and, oh yeah, side quests. These are games in which one can immerse oneself like a warm bath, games in which one can wander for hours and hours and still find delightful new surprises: new nooks and crannies to explore, new choices to make, new people to meet.

All of this delight, unfortunately, cost the creators of these games a great deal of time and money. In a D&D campaign, all of the richness of the world and its denizens is conjured up gratis in the minds of the players by the spoken words of the DM. In Baldur’s Gate, however, every choice, every possibility offered to the player in the name of openness had to be put there by someone. The game is, in effect, a very elaborate, lovingly illustrated and sound-tracked, flowchart. Every temple, every dungeon, every line of dialogue, every character animation, every side quest, came from the toil of artists, writers, and programmers.

The cost to provide such things ballooned over the years as the expectations for the visual and auditory richness of games continued to ratchet up, from the simple tiles of Ultima to the hand-painted landscapes of Baldur’s Gate, and beyond. As the lead designer for the latter game said in a recent interview, explaining the cost of a writer’s simple flourish of imagination9:

What you write on a page takes ten seconds, but all the resources that have to go into that—modeling, texturing, voiceover, music, all the rest—suddenly that ten seconds of writing becomes tens of thousands of dollars of assets.

It is for this reason, combined with the niche appeal of RPGs, that the “Ultima” branch of computer RPGs was largely abandoned after the early 2000s in favor of more cost effective genres10. There was, however, one more possible approach to building a secondary world available. After all, a computer need not merely ingest data that was already provided to it. What if instead of paying all those writers, modelers, and artists, you got the computer to build the world for itself?

Digital Synthesis

It is a fairly easy task to get a computer to generate a dungeon maze.11 It’s not much harder to stock it with random monsters and loot, with escalating difficulty and value, respectively, as the player descends to deeper levels of the dungeon. A whole sub-genre was built on these facts, called “Rogue-likes,” after the 1980 game Rogue, which was freely distributed on Unix systems throughout the following decade. Later variations on the theme such as Hack, Moria, NetHack, and Angband also spread across the fringes of nerdom throughout the 1980s. Though sometimes graphics packs were available, by default these games rendered the dungeon in ASCII characters, with letters for monsters, carats for stairs, and an @ for the player character.

NetHack ASCII dungeon

The genre exploded into the wider popular culture with the release of Diablo in 1996, which took the basic idea of procedurally-generated dungeon environments, and loot and added sophisticated visuals and sound and a graphical user interface. But the basic appeal of all these games was one-dimensional dungeon-delving. They offered no agency to the player beyond the cycle of kill and loot, and no wider setting to explore.

What if the same basic concepts behind Rogue-like dungeon generation could be applied to an entire world? This was the the conceit of the Elder Scrolls series, which launched in 1994 with Arena, followed by the even-more-ambitious Daggerfall in 1996. Rather than generating new content on the fly as players explore, as most Rogue-likes do, the creators of Daggerfall pre-generated the major adventuring sites of the provinces of High Rock and Hammerfell on their own workstations, and then manually tweaked the results. In total, the game contains some four thousand dungeons and five thousands settlements (villages, towns, and cities) that the player can visit, across a total area of some sixty thousand square miles.

Despite these astonishing figures, however, most of this vast area is utterly stale and lifeless, from a gameplay point of view. In (roughly contemporary) games like Ultima VIIFallout, or Baldur’s Gate, exploring the world is a joy, because one never knows what characters, stories, adventures, or other surprises one will find around each corner. In Daggerfall, other than a smattering of random monsters to fight, there is nothing to do or see in the wilderness between dungeons and towns. And there is nothing particularly exciting about discovering a new dungeon or entering a new town either, since one is much like another, only with a different assortment of random monsters or shops. The direction in the game comes from quests (randomly generated against one of several hundred templates) which direct players to a particular house in a particular town to fetch a dingus, or to a particular dungeon to kill five snarks. Unlike the very constricted scope of Rouge-likes, Daggerfall gives the player a whole world in which to exercise their agency, but fails to provide many interesting things to do there.

The map of just one of forty-four regions in Daggerfall. Each dot is a dungeon, temple, home, or town.

Much more recently, the tremendous growth in the capabilities of machine learning has provided new hope for the dream of a procedurally-generated role-playing experience. A computer that can  generate natural language responses to natural language inputs, a pipe dream just a decade ago, now seems near reality. This past year, Nick Walton, a student at Brigham Young University and a D&D fan, saw that these new tools could be used to simulate a dungeon master. The result was AI Dungeon, based on Open AI’s GPT-2 machine learning model.

Trained on gigabytes of input, GPT-2 is uncannily good at producing sensible text by simply predicting the next word that should follow a given list of input words. Walton fine-tuned the model by feeding it transcripts from a variety of games from the Adventure family tree. By doing so he created a kind of meta-Adventure, that could branch off in (for all practical purposes) an infinite number of directions.  This is a parser that is not mere window dressing for a flow chart. Like a human dungeon master, it generates a novel response to each input from the player.

However, there is a painful sacrifice to be made for this gain in openness, for AI Dungeon has only the feeble self-consistency of a dream or hallucination. The game has no model of the world to test its responses against. It is a kind of prism, refracting the player’s words through a textual lens, and then bouncing the result back to the screen. The game will respond without blinking to absurd player actions that make no sense in the context of an adventure a fantasy realm, e.g. “fly to Tahiti” or “write an essay about Dungeons and Dragons.” And, like a river, it will not allow you to step into the same place twice. Here, for example, is my own attempt at an adventure as the wizard Iocounu. Notice how impressive it is until I try to retrace my steps12:

You are Iucounu, a wizard living in the kingdom of Larion. You have a staff and a spellbook. You finish your long journey and finally arrive at the ruin you’ve been looking for. You look around and see that it’s indeed a ruined city. The walls are gone, but the buildings still stand tall as if they were built to last forever.

The ruins themselves aren’t anything special; there is nothing here except rubble and dust.

> enter nearest building

You walk towards one of the buildings, which looks like it might be a guild hall or something similar. You open up the door and find yourself inside a large room with several tables filled with people sitting at them. Most of them seem to be mages studying some sort of book.

> try a different building

You head into another building which doesn’t look too interesting. You go through the front doors and then down a set of stairs.

> go back to the guild hall

You decide to check out the guild hall since you saw it from afar earlier. It seems like it would be a good place to get information on what’s going on in the rest of the city.

> look around

As soon as you step inside, you immediately feel very uncomfortable. There are so many people here that it feels like an invasion. They all stare at you and make faces at you. Some even begin to speak loudly about how “we”

Hail to the Dungeon Master

Because of scaling limitations, models like GPT-2 can consider no more than a few hundred words of previous text in constructing its next output. Google, however, just announced a new type of machine learning mechanism they call “reformer” that can open a much larger window on the past13. Could this, or some other breakthrough, breathe consistency into the phantasmagoria of AI Dungeon? Time will tell.

Thus far, however, the effort to enfold the magic of D&D between the covers of a book, or to inscribe it into the electronic memory of a computer, has been a noble failure. Gamebooks and computer games are available at any time of day or night, ready to play. They will always give a consistent experience, and the better ones can provide hours and hours of enjoyment. They are never burned out, tired, or lazy. Their memory or imagination never falters. But without a human mind behind them, they cannot offer truly open-ended agency, at least without sacrificing all self-consistency. Economics plays a role, of course. There is no strict bound to the number of options that could be provided to the player of a computer RPG, given enough time and money – but in practical terms, the limit is quite sharp. Either your actions in the secondary world that the game invokes are strictly limited to the paths the creator has set out for you, or that world is nothing but a fever dream, a never-ending present without past or future.

So, for now, we are stuck with the dungeon masters, with all their human foibles. Long may they live.

Further Reading

Shannon Appelcline, Designers & Dragons: The ’70s (2014)

Jimmy Maher, The Digital Antiquarian (2011-present)

Jon Peterson, Playing at the World (2012)

 

The Era of Fragmentation, Part 4: The Anarchists

Between roughly 1975 and 1995, access to computers accelerated much more quickly than access to computer networks. First in the United States, and then in other wealthy countries, computers became commonplace in the homes of the affluent, and nearly ubiquitous in institutions of higher education. But if users of those computers wanted to connect their machines together – to exchange email, download software, or find a community where they could discuss their favorite hobby, they had few options. Home users could connect to services like CompuServe. But, until the introduction of flat monthly fees in the late 1980s, they charged by the hour at rates relatively few could afford. Some university students and faculty could connect to a packet-switched computer network, but many more could not. By 1981, only about 280 computers had access to ARPANET. CSNET and BITNET would eventually connect hundreds more, but they only got started in the early 1980s. At that time the U.S. counted more than 3,000 institutions of higher education, virtually all of which would have had multiple computers, ranging from large mainframes to small workstations. Both communities, home hobbyists and those academics who were excluded from the big networks, turned to the same technological solution to connect to one another. They hacked the plain-old telephone system, the Bell network, into a kind of telegraph, carrying digital messages instead of voices, and relaying messages from computer to computer across the country and the world. These were among the earliest peer-to-peer computer networks. Unlike CompuServe and other such centralized systems, onto which home computers latched to drink down information like so many nursing calves, information spread through these networks like ripples on a pond, starting from anywhere and ending up everywhere. Yet they still became rife with disputes over politics and power. In the late 1990s, as the Internet erupted into popular view, many claimed that it would flatten social and economic relations. By enabling anyone to connect with anyone, the middle men and bureaucrats who had dominated our lives would find themselves cut out of the action. A new era of direct democracy and open markets would dawn, where everyone had an equal voice and equal access. Such prophets might have hesitated had they reflected on what happened on Usenet and Fidonet in the 1980s. Be its technical substructure ever so flat, every computer network is embedded within a community of human users. And human societies, no matter how one kneads and stretches, always seem to keep their lumps. Usenet In the summer of 1979, Tom Truscott was living the dream life for a young computer nerd. A grad student in computer science at Duke University with an interest in computer chess, he landed an internship at Bell Labs’ New Jersey headquarters, where he got to rub elbows with the creators of Unix, the latest craze to sweep the world of academic computing. The origins of Unix, like those of the Internet itself, lay in the shadow of American telecommunications policy. Ken Thompson and Dennis Ritchie of Bell Labs decided in the late 1960s to build a leaner, much pared-down version of the massive MIT Multics system to which they had contributed as software developers. The new operating system quickly proved a hit within the labs, popular for its combination of low overhead (allowing it to run on even inexpensive machines) and high flexibility. However, AT&T could do little to profit from their success. A 1956 agreement with the Justice Department required AT&T to license non-telephone technologies to all comers at a reasonable rate, and to stay out of all business sectors other than supplying common carrier communications. So AT&T began to license Unix to universities for use in academic settings on very generous terms. These early licensees, who were granted access to the source code, began building and selling their own Unix variants, most notably the Berkeley Software Distribution (BSD) Unix created at the the University of California’s flagship campus. The new operating system quickly swept academia. Unlike other popular operating systems, such as the DEC TENEX / TOPS-20, it could run on hardware from a variety of vendors, many of them offering very low-cost machines. And Berkeley distributed the software for only a nominal fee, in addition to the modest licensing fee from AT&T.1 Truscott felt that he sat at the root of all things, therefore, when he got to spend the summer as Ken Thompson’s intern, playing a few morning rounds of volleyball before starting work at midday, sharing a pizza dinner with his idols, and working late into the night slinging code on Unix and the C programming language. He did not want to give up the connection to that world when his internship ended, and so as soon as he returned to Duke in the fall, he figured out how to connect the computer science department’s Unix-equipped PDP 11/70 back to the mothership in Murray Hill, using a program written by one of his erstwhile colleagues, Mike Lesk. It was called uucp – Unix to Unix copy – and it was one of a suite of “uu” programs new to the just-released Unix Version 7, which allowed one Unix system to connect to another over a modem. Specifically, uucp allowed one to copy files back and forth between the two connected computers, which allowed Truscott to exchange email with Thompson and Ritchie. Undated photo of Tom Truscott It was Truscott’s fellow grad student, Jim Ellis, who had installed the new Version 7 on the Duke computer, but even as the new upgrade gave with one hand, it took away with the other. The news program that was distributed by the Unix users’ group, USENIX, which would broadcast news items to all users of a given Unix computer system, no longer worked on the new operating ssytem. Truscott and Ellis decided they would replace it with their own 7-compatible news program, with more advanced features, and return their improved software back to the community for a little bit of prestige. At this same time, Truscott was also using uucp to connect with a Unix machine at the University of North Carolina ten miles to the southwest in Chapel Hill, and talking to a grad student there named Steve Bellovin.2 Bellovin had also started building his own news program, which notably included the concept of topic-based newsgroups, to which one could subscribe, rather than only having a single broadcast channel for all news. Bellovin, Truscot and Ellis decided to combine their efforts and build a networked news system with newsgroups, that would use uucp to share news between sites. They intended to distributed provide Unix-related news for USENIX members, so they called their system Usenet.  Duke would serve as the central clearinghouse at first, using its auto-dialer and uucp to connect to each other site on the network at regular intervals, in order to pick up it local news updates and deposit updates from its peers. Bellovin wrote the initial code, but it used shell scripts that operated very slowly, so Stephen Daniel, another Duke grad student, rewrote the program in C. Daniel’s version became know as A News. Ellis promoted the program at the January 1980 Usenix conference in Boulder, Colorado, and gave away all eighty copies of the software that he had brought with him. By the next Usenix conference that summer, the organizers had added A News to the general software package that they distributed to all attendees. The creators described the system, cheekily, as a “poor man’s ARPANET.” Though one may not be accustomed to thinking of Duke as underprivileged, it did not have the clout in the world of computer science necessary at the time to get a connection to that premiere American computer network. But access to Usenet required no one’s permission, only a Unix system, a modem, and the ability to pay the phone bills for regular news transfers, requirements that virtually any institution of higher education could meet by the early 1980s. Private companies also joined up with Usenet, and helped to facilitate the spread of the network. Digital Equipment Corporation (DEC) agreed to act as an intermediary between Duke and UC Berkeley, footing the long-distance telephone bills for inter-coastal data transfer. This allowed Berkeley to become a second, west-coast hub for Usenet, connecting up UC San Francisco, UC San Diego, and others, including Sytek, an early LAN business. The connection to Berkeley, an ARPANET site, also enabled cross-talk between ARPANET and Usenet (after a second re-write by Mark Horton and Matt Glickman to create B News). ARPANET sites began picking up Usenet content and vice versa, though ARPA rules technically forbid interconnection with other networks. The network grew rapidly, from fifteen sites carrying ten posts a day in in 1980, to 600 sites and 120 posts in 1983, and 5000 sites and 1000 posts in 1987.3 Its creators had originally conceived Usenet as a way to connect the Unix user community and discuss Unix developments, and to that end they created two groups, net.general and net.v7bugs (the latter for discussing problems with the latest version of Unix). However they left the system entirely open for expansion. Anyone was free to create a new group under “net”, and users very quickly added non-technical topics such as net.jokes. Just as one was free to send whatever one chose, recipients could also ignore whatever groups they chose, e.g. a system could join Usenet and request data only for net.v7bugs, ignoring the rest of the content. Quite unlike the carefully planned ARPANET, Usenet self-organized, and grew in an anarchic way overseen by no central authority. Yet out of this superficially democratic medium a hierarchical order quickly emerged, with a certain subset of highly-connected, high-traffic sites recognized as the “backbone” of the system. This process developed fairly naturally. Because each transfer of data from one site to the next incurred a communications delay, each new site joining the network had a strong incentive to link itself to an already highly-connected node, to minimize the number of hops required for their messages to span the network. The backbone sites were a mix of educational and corporate sites, usually led by one headstrong individual willing to take on the thankless tasks involved in administering all the activity crossing their computer. Gary Murakami at Bell Labs’ Indian Hills lab in Illinois, for example, or Gene Spafford at Georgia Tech. The most visible exercise of the power held by this backbone administrators came in 1987, when they pushed through a re-organization of the newsgroup namespace into seven top-level buckets. comp, for example, for computer-related topics, and rec for recreational topics. Sub-topics continued to be organized hierarchically underneath the “big seven”, such as comp.lang.c for discussion of the C programming language, and rec.games.board for conversations about boardgaming. A group of anti-authoritarians, who saw this change as a coup by the “Backbone Cabal,” created their own splinter hierarchy rooted at alt, with its own parallel backbone. It included topics that were considered out-of-bounds for the big seven, such as sex and recreational drugs (e.g. alt.sex.pictures)4, as well as quirky groups that simply rubbed the backbone admins the wrong way (e.g. alt.gourmand; the admins preferred the anodyne rec.food.recipes). Despite these controversies, by the late 1980s, Usenet had become the place for the computer cognoscenti to find trans-national communities of like-minded individuals. In 1991 alone, Tim Berners-Lee announced the creation of the World Wide Web on alt.hypertext; Linus Torvalds solicited comp.os.minix for feedback on his new pet project, Linux; and Peter Adkison, due to a post on rec.games.design about his game company, connected with Richard Garfield, a collaboration that would lead to the creation of the card game Magic: The Gathering. FidoNet But even as the poor man’s ARPANET spread across the globe, microcomputer hobbyists,  with far fewer resources than even the smallest of colleges, were still largely cut off from the experience of electronic communication. Unix, a low-cost, bare-bones option by the standards of academic computing, was out of reach for hobbyists with 8-bit microprocessors, running an operating system called CP/M that barely did anything beyond managing the disk drive. But they soon began their own shoe-string experiments in low-cost peer-to-peer networking, starting with something called bulletin boards. Given the simplicity of the idea and the number of computer hobbyists in the wild at the time, it seems probable that the computer bulletin board was invented independently several times. But tradition gives precedence to the creation of Ward Christensen and Randy Suess of Chicago, launched during the great blizzard of 1978.  Christensen and Suess were both computer hobbyists in their early thirties, and members of their local computer club. For some time they had been considering creating a server where computer club members could upload news articles, using the modem file transfer software that Christensen had written for CP/M – the hobbyist equivalent of uucp. The blizzard, which kept them housebound for several days, gave them the impetus to actually get started on the project, with Christensen focusing on the software and Suess on the hardware. In particular, Suess devised a circuit that automatically rebooted the computer into the BBS software each time it detected an incoming caller, a necessary hack to ensure the system was in a good state to receive the call, given the flaky state of hobby hardware and software at the time. They called their invention CBBS, for Computerized Bulletin Board System, but most later system operators (or sysops) would drop the C and call their service a BBS.5 They published the details of what they had built in a popular hobby magazine, Byte, and a slew of imitators soon followed. Another new piece of technology, the Hayes Modem, fertilized this flourishing BBS scene. Dennis Hayes was another computer hobbyist, who wanted to use a modem with his new machine, but the existing commercial offerings fell into two categories: devices aimed at business customers that were too expensive for hobbyists, and acoustically-coupled modems. To connect a call on an acoustically-coupled modem you first had to dial or answer the phone manually, and then place the handset onto the modem so they could communicate. There was no way to automatically start a call or answer one. So, in 1977, Hayes designed, built, and sold his own 300 bit-per-second modem that would slot into the interior of a hobby computer. Suess and Christensen used one of these early-model Hayes modems in their CBBS. Hayes’ real breakthrough product, though, was the 1981 Smartmodem, which sat in its own external housing with its own built-in microprocessor and connected to the computer through its serial port. It sold for $299, well within reach of hobbyists who habitually spent a few thousand dollars on their home computer setups. The 300 baud Hayes Smartmodem One of those hobbyists, Tom Jennings, set in motion what became the Usenet of BBSes. A programmer for Phoenix Software in San Francisco, Jennings decided in late 1983 to write his own BBS software, not for CP/M, but for the latest and greatest microcomputer operating system, Microsoft DOS. He called it Fido, after a computer he had used at his work, so-named for its mongrel-like assortment of parts. John Madill, a salesman at ComputerLand in Baltimore, learned about Fido and called all the way across the country to ask Jennings for help in tweaking Fido to make it run on his DEC Rainbow 100 microcomputer. The two began a cross-country collaboration on the software, joined by another Rainbow enthusiast, Ben Baker of St. Louis. All three racked up substantial long-distance phone bills as they logged into one another’s machines for late-night BBS chats. With all of this cross-BBS chatter, an idea began to buzz forward from the back of Jennings’ mind, that he could create a network of BBSes that would exchange messages late at night, when long-distance rates were low. The idea was not new. Many hobbyists had imagined that BBSes could route messages in this way, all the way back to Christensen and Suess’ Byte article. But they generally had assumed that for the scheme to work, you would need very high BBS density and complex routing rules, to ensure that all the calls remained local, and thus toll-free, even when relaying messages from coast to coast. But Jennings did some back-of-the-envelope math and realized that, given increasing modem speeds (now up to 1200 bits per second for hobby modems) and falling long-distance costs, no such cleverness was necessary. Even with substantial message traffic, you could pass text between systems for a few bucks per night. Tom Jennings in 2002 (still from the BBS documentary) So he added a new program to live alongside Fido. Between one to two o’clock in the morning, Fido would shut down and FidoNet would start up. It would check Fido’s outgoing messages against a file called the node list. Each outgoing message had a node number, and each entry in the list represented a network node – a Fido BBS – and provided the phone number for that node number. If there were pending outgoing messages, FidoNet would dial up each of the corresponding BBSes on the node list and transfer the messages over to the FidoNet program waiting on the other side. Suddenly Madill, Jennings and Baker could collaborate easily and cheaply, though at the cost of higher latency – they wouldn’t receive any messages sent during the day until the late night transfer began. Formerly, hobbyists rarely connected with others outside their immediate area, where they could make toll-free calls to their local BBS. But if that BBS connected into FidoNet, users could suddenly exchange email with others all across the country. And so the scheme proved immensely popular, and the number of FidoNet nodes grew rapidly, to over 200 within a year. Jennings’ personal curation of the node list thus became less and less manageable. So during the first “FidoCon” in St. Louis, Jennings and Baker met in the living room of Ken Kaplan, another DEC Rainbow fan who would take an increasingly important role in the leadership of FidoNet. They came up with a new design that divided North America into nets, each consisting of many nodes. Within each net, one administrative node would take on the responsibility of  managing its local nodelist, accepting inbound traffic to its net, and forwarding those messages to the correct local node. Above the layer of nets were zones, which covered an entire continent. The system still maintained one global nodelist with the phone numbers of every FidoNet computer in the world, so any node could theoretically directly dial any other to deliver messages. This new architecture allowed the system to continue to grow, reaching almost 1,000 nodes by 1986 and just over 5,000 by 1989. Each of these nodes (itself a BBS) likely averaged 100 or so active users. The two most popular applications were the basic email service that Jennings had built into FidoNet and Echomail, created by Jeff Rush, a BBS sysop in Dallas. Functionally equivalent to Usenet newsgroups, Echomail allowed the thousands of users of FidoNet to carry out public discussions on a variety of topics. Echoes, the term for individual groups, had mononyms rather than the hierarchical names of Usenet, ranging from AD&D to MILHISTORY to ZYMURGY (home beer brewing). Jennings, philosophically speaking, inclined to anarchy, and wanted to build a neutral platform governed only by its technical standards6: I said to the users that they could do anything they wanted …I’ve maintained that attitude for eight years now, and I have never had problems running BBSs. It’s the fascist control freaks who have the troubles. I think if you make it clear that the callers are doing the policing–even to put it in those terms disgusts me–if the callers are determining the content, they can provide the feedback to the assholes. Just as with Usenet, however, the hierarchical structure of FidoNet made it possible for some sysops to exert more power than others, and rumors swirled of a powerful cabal (this time headquartered in St. Louis), seeking to take control of the system from the people. In particular, many feared that Kaplan or others around him would try to take the system commercial and start charging access to FidoNet. Of particular suspicion was the International FidoNet Association (IFNA), a non-profit that Kaplan had founded to help defray some of the costs of administering the system (especially the long-distance telephone charges). In 1989 those suspicions seemed to be realized when a group of IFNA leaders pushed through a referendum to make every FidoNet sysop a member of IFNA and turn it into the official governing body of the net, responsible for its rules and regulations. The measure failed, and IFNA was dissolved instead. Of course, the absence of any symbolic governing body did not eliminate the realities of power; the regional nodelist administrators instead enacted policy on an ad hoc basis. The Shadow of Internet From the late 1980s onward, FidoNet and Usenet gradually fell under the looming shadow of the Internet. By the second half of that same decade, they had been fully assimilated by it. Usenet became entangled within the webs of the Internet through the creation of NNTP – Network News Transfer Protocol – in early 1986. Conceived by a pair of University of California students (one in San Diego and the other in Berkeley), NNTP allowed TCP/IP network hosts on the Internet to create Usenet-compatible news servers. Within a few years, the majority of Usenet traffic flowed across such links, rather than uucp connections over the plain-old telephone network. The independent uucp network gradually fell into disuse, and Usenet became just another application atop TCP/IP transport. The immense flexibility of the Internet’s layered architecture made it easy to absorb a single-application network in this way.  Although by the early 1990s, several dozen gateways between FidoNet and Internet existed, allowing the two networks to exchange messages, FidoNet was not a single application, and so its traffic did not migrate onto the internet in the same way as Usenet. Instead, as people outside academia began looking for Internet access for the first time in the second half of the 1990s, BBSes gradually found themselves either absorbed into the Internet or reduced to irrelevance. Commercial BBSes generally fell into the first category. These mini-CompuServes offered BBS access for a monthly fee to thousands of users, and had multiple modems for accepting simultaneous incoming connections. As commercial access to the Internet became possible, these businesses connected their BBS to the nearest Internet network and began offering access to their customers as part of a subscription package. With more and more sites and services becoming available on the burgeoning World Wide Web, fewer and fewer users signed on to the BBS per se, and thus these commercial BBSes gradually became pure internet service providers, or ISPs. Most of the small-time hobbyist BBSes, on the other hand, became ghost towns, as users wanting to tap into the Internet flocked to their local ISPs, as well as to larger, nationally known outfits such as America Online. That’s all very well, but how did the Internet become so dominant in the first place? How did an obscure academic system, spreading gradually across elite universities for years while systems like Minitel, CompuServe and Usenet were bringing millions of users online, suddenly explode into the foreground, enveloping like kudzu all that had come before it? How did the Internet become the force that brought the era of fragmentation to an end? [Previous] [Next] Further Reading / Watching Ronda Hauben and Michael Hauben, Netizens: On the History and Impact of Usenet and the Internet, (online 1994, print 1997) Howard Rheingold, The Virtual Community (1993) Peter H. Salus, Casting the Net (1995) Jason Scott, BBS: The Documentary (2005)

Read more
The Pursuit of Efficiency and the Science of Steam

On April 19th, 1866, Alfred Holt, a Liverpudlian engineer who had apprenticed on the Liverpool & Manchester railroad before taking up steamship design in the 1850s, launched a singular ship that he dubbed the Agamemnon. As the third soon of a prosperous banker, cotton broker, and insurer, he had access to far more personal capital to launch this new enterprise than the typical engineer. This was a lucky thing for him, because the typical investor of the time considered his ambition—to enter the China tea trade on the basis of steam power—foolhardy. A typical oceangoing steamship used five pounds of coal per horsepower per hour and could not compete with sail over such long distances: they would either have to fill most of their potential cargo space with coal or make repeated, costly stops to refuel.[1] A contemporary photograph of Holt’s SS Agamemnon. Yet, in the end, Holt pulled off his gamble. He benefited from good timing (perhaps a mix of luck and foresight): the opening of the Suez Canal in 1869 give steamships a tremendous leg up in trade between Europe to the Indian and Pacific Oceans. But in designing ships dainty enough in their coal consumption to pay their way to the Pacific, he also benefited from the late convergence of two complementary developments that had each begun in the early 1800s but did not intersect until the 1850s. First was a series of incremental, empirical improvements to steam engine design: After the massive leap forward from Newcomen to Watt, further increases in steam engine efficiency would be less dramatic. Simultaneously, a theory of heat gradually developed that could explain what made engines more or less efficient, and thus point engineers in the most fruitful direction. Double-Cylinder Engines Boulton & Watt erected most of its early pumping engines in Cornwall. Trevithick developed his high-pressure “puffer” there. So, it is only fitting that the last major architectural innovation in piston steam engine design—featuring an entirely new structural component—was Cornish, too. In that region, an ample supply of British engineering talent met an always-eager demand for efficient engines. The ever-deeper mines for extracting metal ore needed ever more pumping power, despite significantly higher coal prices than the coal-rich North. Joseph Hornblower, born in the 1690s, was one of the first engineers to build Newcomen engines for the mines of Cornwall in the 1720s. Sixty years later, his grandson Jonathan built the first known double-cylinder engine (later called a compound engine). Cornwall’s homegrown natural philosopher, Davies Giddy (later Gilbert), served in the same office he later served for Richard Trevithick, as Hornblower’s scientific advisor. In principle, the idea was quite simple: instead of immediately condensing the remaining steam after the expansion cycle of the piston, the still-warm steam was fed into another cylinder to let it do still more work. However, this added friction, complexity, and cost to the machine. In practice, therefore, Hornblower’s attempted improvement provide no more efficient than a traditional Watt engine.[2] Hornblower double-cylinder engine from Robert Thurston, A History of the Growth of the Steam-Engine, p. 136. A generation later, however, another Cornishman took up the idea and carried it further. Arthur Woolf, like many eighteenth-century engineers, got his start as a millwright, but by 1797 was working for the firm of Jabez Carter Hornblower (brother to Jonathan), at a brewery in London, erecting a steam engine. He continued to serve as engineer for the brewery for a decade afterward, and witnessed the operation of Trevithick’s steam carriage in the city in 1803. Woolf realized that he could combine the double-cylinder engine of his former employer’s brother with Trevithick’s truly high-pressure engines (operating at forty pounds per-square-inch or more). The higher-pressure steam, still quite hot after expanding in the first cylinder, would be able to do more work in the second cylinder rather than simply “puffing” out into the atmosphere. Both Watt and Trevithick had (from opposite points-of-view) seen low- and high-pressure steam as rivals, but in Woolf’s machine they complemented one another.[3] But, as Hornblower had already learned, the path did not always run straight and easy from idea to execution. Woolf led himself astray with an entirely unsound theoretical model for the inner workings of his engine: he believed that steam at twenty pounds per square inch (psi) would expand to twenty times its volume before equaling the pressure of the atmosphere, steam at thirty psi would expand thirty times, and so on ad infinitum. This turned out to be a substantially exaggerated expectation, and led him to begin with a drastically undersized high-pressure cylinder, which let off far too little steam to effectively work its low-pressure mate. Rather than leading him to doubt his theory, the failure of this engine led him into a wild goose chase for a non-existent leak in his pistons.[4] Woolf’s double-cylinder engine, unlike Hornblower’s, did at last succeed, after years of trial and error, in achieving better efficiency than a Watt engine. But because it was more expensive to build (and thus buy), and more complex to operate, it found favor only in markets without easy access to other, cheaper options. One such example was France, to which Woolf’s erstwhile partner Humphrey Edwards, decamped in 1815: there he sold at least fifteen engines and licensed twenty-five more to a French mining company.  Woolf meanwhile returned to Cornwall in 1811, where he found the advantages of his double-cylinder engine soon surpassed by the incremental improvements made by other local engineers to the Boulton and Watt design. He abandoned it after 1824 and built single-cylinder engines until 1833, when he retired to the island of Guernsey.[5] Meanwhile, steam engine builders carried on with tweaks to get yet one more increment of efficiency out of their engines. They extracted advantages from adjustments to the regulatory machinery of the engine: elements like “release mechanisms,” “dashpots,” and “wrist plates.” The Corliss engine, designed by George Corliss in 1849, became an icon of American industrial design after his company produced a gargantuan specimen to power the 1876 Centennial Exhibition in Philadelphia. Mighty as it was, however, it did not represent a great leap forward in steam engine architecture. Corliss’ design drew its relative advantages over prior engines from a clever combination of previous innovations in the valves that allowed steam to enter and leave the cylinder, and especially in the valve gear that controlled them.[6] Corliss engine valve gear from H.W. Dickinson, A Short History of the Steam Engine, p. 140. In the meantime, the double-cylinder engine, having failed to prove itself in the 1810s and 1820s, lay dormant. It would be restored to life decades later, by the engineers most desperate to eke as much power as possible out of every ounce of coal: the designers of ocean steamships. But to facilitate the consummation of that match, a solid theory of the steam engine was wanted, one that would dispel, once and for all, the confusions like Woolf’s that continued to trip up engineers’ efforts at improvement. Measuring Power The lack of a sound theoretical basis for steam power is evident in the fitful history of cylinder “lagging,” or insulation. Steam engineers borrowed the term lag (a barrel stave) from coopers, because they often insulated early steam boilers with such timbers, held in place with metal straps (this is evident in images of early locomotives like Rocket, with their distinctive wooden cladding). A contemporary lithograph of Robert Stephenson’s engine Northumbrian. Note the wooden lagging on the boiler. As early as 1769, Watt had recognized the value of insulating not just the boiler, but also the working cylinder of the engine (emphasis mine): My method of lessening the consumption of steam, and consequently fuel, in fire-engines, consists of the following principles:—First, That vessel in which the powers of steam are to be employed to work the engine, which is called the cylinder in common fire-engines, and which I call the steam-vessel, must, during the whole time the engine is at work, be kept as hot as the steam that enters it; first by enclosing it in a case of wood, or any other materials that transmit heat slowly; secondly, by surrounding it with steam or other heated bodies; and, thirdly, by suffering neither water nor any other substance colder than the steam to enter or touch it during that time.[7] Yet, despite Watt’s imprimatur, steam engine builders lagged their cylinders sporadically throughout the first half of the nineteenth century; it was a matter of whim, not principle.[8] In this era, engineers tended to think of the steam engine as analogous to its predecessor, the water wheel. Steam replaced liquid water as the mechanical working fluid, but just as water drove the wheel by pushing on its vanes, in their minds steam performed work by expanding and pushing on the piston. A typical description of the time stated that “[t]he force of the steam-engine is derived from the property of water to expand itself, in an amazing degree, when heated above the temperature at which it becomes steam.”[9] Engineers knew that the cylinder ought to be kept hot to prevent condensation of the steam inside, but within this framework it was not obvious that it ought to be kept as hot as possible. Watt, emphasizing the contrast between the hot cylinder and the cool condenser, had drawn attention to the role of heat in the engine, but the introduction and success of high-pressure engines with no condenser, where the primary factor seemed to be the expansive force of steam, muddled matters once again. The gradual development of a new, more robust theory began with a practical problem: how to measure the amount of power an engine generates. This became a particularly pressing problem for Boulton & Watt in the late eighteenth century, as they expanded from the traditional business of pumping engines into the new market of driving cotton mills. The traditional way of measuring the output of a steam engine, in terms of “duty” (the pounds of water lifted by one foot per bushel of coal burned) had gradually been supplemented with the concept of “power,” typically expressed in horsepower: pounds lifted over a given distance, but over a given period of time rather than with a given amount of fuel. Thomas Savery had begun to grope towards the concept in his 1702 book on the virtues of his steam pump, The Miner’s Friend: I have only this to urge, that water, in its fall from any determinate height, has simply a force answerable and equal to the force that raises it. So that an engine which will raise as much water as two horses working together at one time in such a work can do, and for which there must be constantly kept ten or twelve horses for doing the same, then, I say, such an engine will do the work or labour of ten or twelve horses…[10] Note here that Savery proposes to measure the muscular equivalent of the engine not in terms of the output of just the pair of horses running the machinery, but in terms of the total stock of horses that a mine owner would require to maintain the same power over a long period of time. This model of horsepower in terms of economic equivalency did not stick, however, and by the late eighteenth century horsepower became fixed to Watt’s figure of 33,000 foot-pounds per minute. Yet this remained a measure of power best suited to pumping work: if a mine needed to raise 20,000 pounds of water per hour from a 200-foot-deep shaft, one could readily calculate the engine horsepower required. Cotton spinning machinery—which varied in size, function, and design—did not lend itself to such simple arithmetic. In order to properly size engines to mills, Boulton & Watt needed some way measure the horsepower produced by an engine while driving various combinations of machinery. From the beginning, Watt had attached gauges to his engines to measure the pressure inside the engine, by connecting a small indicator cylinder to the main engine cylinder so that steam could flow between them. The level of pressure in the indicator could serve as a proxy for power output. But to actually capture the data was a maddening exercise, because the pressure varied constantly as the piston worked up and down. A means of capturing this continuous data came from a long-time Watt employee, John Southern. He had joined the company as a draftsman in 1782, and despite a predilection for music that the strait-laced Watt found suspicious, quickly became indispensable.[11] Southern’s indicator, as envisioned by Terrell Croft, Steam-Engine Principles and Practice, p. 40. In 1796, Southern devised a simple device to solve the power measurement problem. He attached a piece of paper above the indicator, rigged so that it would move back and forth as the main piston operated. Then he attached a pencil to the tip of the pressure gauge. As the pressure went up and down, so would the pencil, while the paper moved left and right beneath it with the cycle of the engine. The result, when running smoothly, would be a closed shape, which Southern called an indicator diagram, and the averagepressure during the operation of the engine could be computed from the average distance between the top and bottom lines of that shape, which would in turn be proportional to the power. By calibrating the diagramwhile an engine was pumping water, where the power output was well-defined, Boulton & Watt could then determine the power produced by the same engine while operating a given set of mill machinery.[12] An ideal indicator diagram from Terrell Croft, Steam-Engine Principles and Practice, p.60. Thermodynamics Engineers now had a tool at hand for diagnosing the internals of a running engine. That tool, in turn, provided the seed for the birth of the science of thermodynamics, which began as the science of the steam engine. The first great leap in that direction was made by Sadi Carnot. Carnot’s story carries more than a whiff of the tragic. Though later honored as a founding father of thermodynamics, he achieved no recognition in his lifetime, and died of cholera as a still-young man in 1832. His father Lazare was an accomplished engineer and a major political figure in revolutionary France, but what we know of the son comes almost entirely from a fifteen-page biography sketched decades after the fact by his younger brother Hippolyte, which begins, pathetically, with the statement that: “the life of Sadi Carnot was not marked by any notable event…”[13] Carnot as an École student in 1813. In fact, Carnot’s short life was remarkably eventful. He grew up in Napoleon’s court, attended the elite engineering school École polytechnique at age 16, and was at the Chateau Vincennes during the 1814 assault on Paris that ended Napoleon’s first reign. He returned to Paris as a staff lieutenant in 1819, filling his free time with his passions: music, art, and scientific studies. There, in 1824, he produced his seminal work, Réflexions sur la puissance motrice du feu (Reflections on the Motive Power of Fire). In it he endeavored to explain how heat produces motion. I will allow him to elaborate in his own words: Every one knows that heat can produce motion. That it possesses vast motive-power no one can doubt, in these days when the steam-engine is everywhere so well known. To heat also are due the vast movements which take place on the earth. It causes the agitations of the atmosphere, the ascension of clouds, the fall of rain and of meteors, the currents of water which channel the surface of the globe, and of which man has thus far employed but a small portion.[14] As we have seen, the tendency of engineers to conceive of steam hydraulically, as a fluid that generated work through pressure much like water in a water wheel, had engendered some confusion about how to build and operate an engine most efficiently. Ironically, Carnot moved the understanding of the steam engine forward by taking the analogy of a steam engine to a water wheel even more seriously than his contemporaries. However, for him the key power-generating agent was not the pressure of steam, but the fall of heat. Just as a waterwheel required a head from which water descended by gravity to turn the wheel, so the steam engine required a reservoir of high heat, which then flowed down to a cold body and thereby did work. For Carnot this fall of heat in a steam engine was quite literal: it consisted of an imponderable fluid called caloric, that drained out from the hot body to the cool one: The production of motion in steam-engines is always accompanied by a circumstance on which we should fix our attention. This circumstance is the re-establishing of equilibrium in the caloric; that is, its passage from a body in which the temperature is more or less elevated, to another in which it is lower. …The steam is here only a means of transporting the caloric.[15] This caloric theory of heat as a substance still predominated in Carnot’s day, despite subversives like Count Rumford who advocated for a mechanical theory of heat, which understood heat purely as a form of motion. If the flow of heat from the hot to the cold body produced all the work in the steam engine, then making an efficient engine meant minimizing any spillage of heat that did no useful work. It also implied that to maximize the work produced by the engine, one must maximize the difference between the source of high temperature and the sink of low temperature—the height through which the caloric fluid falls. Carnot’s book was largely ignored. But his insights had their first chance to be rescued from obscurity shortly after his death. Émile Clapeyron, just a few years younger than Carnot, was an accomplished engineer who specialized in locomotives, and a fellow-graduate of the École Polytechnique. In 1834, he published a paper in the school’s journal showing that Carnot’s heat engine theory could be expressed in the language of calculus and seen graphically in the indicator diagram: the area inside the diagram (which could be expressed as an integral) corresponded to the work performed by the heat transfer in the engine. Clapeyron’s work revived Carnot’s abstractions, put them on a firmer mathematical basis, and publicized them to the community of engine builders. Yet once again, they reached a dead end. Steeped in the traditions of their craft, neither Clapeyron nor his peers seem not to have understood the heat engine theory as having practical applications to real-life engineering.[16] Vindication for Carnot would have to wait another fifteen years, when a series of exchanges between William Thomson (later Lord Kelvin), Rudolf Clausius, and James Joule shortly before and after 1850 resolved various problems with the Carnot-Clapeyron heat engine, including reconciling it with the mechanical theory of heat: what flowed from the hot to the cold body was not a literal fluid but an abstraction called energy, which could take on many forms, but could only perform useful work over a fall in temperature. Through the medium of energy, a certain quantity of heat was directly equivalent to a certain amount of power.[17] The scientist who best synthesized this new science of heat for a wider engineering audience was Thomson’s colleague at the University of Glasgow, Macquorn Rankine. Perfecting the Marine Engine Rankine’s position was something of a novelty: he was only the second person to hold a chair of Civil Engineering at Glasgow, a position established by Queen Victoria in 1840. From the days of Watt and beyond, the University of Glasgow had been more practical-minded than the great Oxbridge schools of the South. But the establishment of a faculty chair in engineering did not just indicate that the university supported more hardheaded tasks than absorbing classical learning, it also signaled a desire to elevate engineering into a more theoretical, scientific discipline.[18] PGP R 2115.24 " data-medium-file="https://cdn.accountdigital.net/FnlrZNj8fQTsaPvT22_9gd-YHEEq" data-large-file="https://technicshistory.com/wp-content/uploads/2023/11/william_john_macquorn_rankine_by_thomas_annan.jpg?w=739" loading="lazy" width="778" height="1023" src="https://cdn.accountdigital.net/FrEoBHbrvJHyzrEXe93NzuOg6OWg" alt="" class="wp-image-14597" style="width:408px;height:auto" srcset="https://cdn.accountdigital.net/FrEoBHbrvJHyzrEXe93NzuOg6OWg 778w, https://cdn.accountdigital.net/FmzgsPbBW_cLTscowMscuC3n_cwa 1556w, https://cdn.accountdigital.net/Fp_NjcBxPV0wCxM3zCCtq6WnAxpV 114w, https://cdn.accountdigital.net/FnlrZNj8fQTsaPvT22_9gd-YHEEq 228w, https://cdn.accountdigital.net/FqppXmUOcVWA7m11yEWUApiqe2FB 768w" sizes="(max-width: 778px) 100vw, 778px">A leonine Rankine. Rankine, embodying this new spirit, straddling the worlds of theory and practice, preached thermodynamics to the engineering world: his 1859 A Manual of the Steam Engine and Other Prime Movers (1859), a 500-page, densely mathematical treatise, explicated the new theory and its applicability to practical matters in great detail and popularized the term “thermodynamics.” However he also knew how to reach a wider audience: in an 1854 address to the Liverpool meeting of the British Association for the Advancement of Science (BAAS) he concisely expressed the laws of thermodynamics in terms of ordinary English and simple arithmetic: “As the absolute temperature of receiving heat is to the absolute temperature of discharging heat, so is the whole heat received to the necessary loss of heat.” That is, the more precipitous the fall of temperature from the high (receiving) to the low (discharging) point of the engine cycle, the more efficient the engine could be.[19] Among those in Rankine’s circle of influence in the 1850s was an experienced builder of marine steam engines in Glasgow named John Elder, who became the first to incorporate a double-cylinder engine into a successful steamship. Elder had marine engines in his blood: his father David had joined Robert Napier’s engine building firm and began designing steamboat engines in 1821. In addition to family tradition and his natural talents, Elder had two other advantages in this undertaking. First, he had access to Glasgow’s “thermodynamic network” (as the historian Crosbie Smith put it); he had tutors in the new thermodynamic science and probably got specific advice from Rankine to introduce steam jacketing to prevent condensation in the cylinder. Second, he had an eager buyer.[20] An anonymous engraving of John Elder. The Pacific Steam Navigation Company (PSNC) of Liverpool had overextended itself in the South American Pacific-coast trade, where high-quality steam coal could arrive only by a 19,000-mile round-trip supplied by sail. Profit margins were slim to none, and venture stayed in the black only by virtue of a government mail contract. This made the company willing to wait out teething problems in order to get a more efficient engine. From the time Elder and his partner took out their engine patent in January 1853, it took four years before PSNC ratified the superiority of their ship Valparaiso, which consumed 25% less coal than an equivalent single-cylinder model.[21] Elder’s success set the stage for Holt’s further vault forward in the 1860s. Among the latter’s achievements was to convince the Board of Trade that marine engines could operate safely at higher pressures; allowing a greater fall of temperature and thus more efficient use of fuel. This, in turn, set the stage for triple-expansion engines later in the century, to extract still more work from the heat as it falls from boiler to condenser. This polyphonic fugue of machinery heralded the age of steam’s baroque period, which engendered the fantasias of steampunk a century later. By about 1890, a triple-expansion engine, running at 160 pounds-per-square-inch, could consume one-and-a-half pounds of coal per-horsepower per-hour, less than a third of the going rate a few decades before, and about five times less than Watt’s engine.[22] SONY DSC " data-medium-file="https://cdn.accountdigital.net/FtSQ8BekQNbv8kPS9uifApbwjKgt" data-large-file="https://technicshistory.com/wp-content/uploads/2023/11/tmw_677_-_triple_expansion_compound_steam_engine.jpg?w=739" loading="lazy" width="1024" height="975" src="https://cdn.accountdigital.net/FnvB-w_zECM1YvpFkdg-gYKhv1iR" alt="" class="wp-image-14600" srcset="https://cdn.accountdigital.net/FnvB-w_zECM1YvpFkdg-gYKhv1iR 1024w, https://cdn.accountdigital.net/FuWr2VLuwR94tPMUvE7sdYLm4x64 2046w, https://cdn.accountdigital.net/FvPIsIFR9yKXU_LcLXTgOMpJG8pE 150w, https://cdn.accountdigital.net/FtSQ8BekQNbv8kPS9uifApbwjKgt 300w, https://cdn.accountdigital.net/FuYz2_GUHfI2LpjeXYePQ7ky7D4- 768w" sizes="(max-width: 1024px) 100vw, 1024px">Cutaway of an 1888 Austrian triple-expansion engine, in the Vienna Technical Museum [Sandstein / Creative Commons Attribution 3.0 Unported]. Yet even as it thrust the age of steam up towards its apex, thermodynamics pointed out the weak spot that would lead to its downfall. In his 1854 speech to the BAAS, Rankine had touted the advantages of the air engine, a device devised by the Scotsman Robert Stirling that used hot air as its working fluid.  As Rankine pointed out, the laws of thermodynamics have nothing in particular to do with steam, but hold “true for all substances whatsoever in all conditions…” Air had a decided advantage over steam insofar as it could be driven to very high temperatures without creating very dangerous pressures: “For example, at the temperature of 650 ° Fahr. (measured from the ordinary zero,) a temperature up to which air engines have actually been worked with ease and safety, the pressure of steam is 2100 pounds upon the square inch; a pressure which plainly renders it impracticable to work steam engines with safety….”[23] The Stirling air engine did not, in the event, prove to be the slayer of steam. Its use never expanded beyond occasional low-power domestic applications. But it brought the first adumbration of the coming eclipse. Stirling air engine – harbinger of doom? [Paul U. Ehmer / CC-BY-SA-4.0]

Read more
Microcomputers – The First Wave: Responding to Altair

[This post is part of “A Bicycle for the Mind.” The complete series can be found here.] Don Tarbell: A Life in Personal Computing In August 1968, Stephen Gray, sole proprietor of the Amateur Computer Society (ACS), published a letter in the society newsletter from an enthusiast in Huntsville, Alabama named Don Tarbell. To help other would-be owners of home-built computers, Tarbell offered a mounting board for integrated circuits for sale for $8 from his own hobby-entrepreneur company, Advanced Digital Design. Tarbell worked for Sperry Rand on projects for NASA’s Marshall Space Flight Center, but had gotten hooked on computers through coursework at the University of Alabama at Huntsville, and found the ACS through a contact at IBM.[1] Over the ensuing years, integrated circuits became far cheaper and easier to come by, and building a real home computer on one’s own thus far more feasible (though still a daunting challenge, demanding a wide range of hardware and software skills). In June 1972, Tarbell had mastered enough of those skills to report to the ACS Newsletter that he (at last) had a working computer system, with an 8-bit processor built from integrated circuits, four-thousand bytes of memory, a text editor and a calculator program, a Teletype for input and output, and an eight-track-tape interface for long-term storage. Not long after this report to ACS, Tarbell decamped from Alabama and moved to the Los Angeles area to work for Hughes Aircraft.[2] Don Tarbell with his home-built computer system [Kilobaud: The Small Computer Magazine (May 1977), 132]. Three years after that, in 1975, the arrival of the Altair 8800 kit announced that anyone with the skills to assemble electronics could have the power of a minicomputer in their own home, and thousands heeded the call. A group of 150 of these personal computer hobbyists met in the commons of the apartment complex where Tarbell lived. They had come on Father’s Day for the inaugural meeting of the Southern California Computer Society (SCCS). Half of the participants already owned Altairs. Tarbell took on the position of secretary for the new society, and served on the board of directors. Within a few months, SCCS began producing its own magazine with a full editorial staff, a far more sophisticated operation than the old hand-typed ACS Newsletter; Tarbell eventually became one of its associate editors.[3] But an Altair kit by itself was far from a complete computer system like the one Tarbell had back in 1975. It had a piddling 256 bytes of memory, and no devices for reading or writing data other than lights and switches. Dozens of hobbyists founded their own companies to sell other computer buffs the additional equipment that would answer the deficiencies of their newly-purchased Altairs. Don Tarbell was one of them. Among the major problems was the inability to permanently store or load programs and data. Once you shut off the computer, everything you had entered into it was lost. A standard Teletype terminal came equipped with a paper tape punch and reader, but even a heavily used Teletype could cost $1000. In February 1976, Tarbell offered a much simpler and cheaper solution, the Tarbell cassette interface, a board that would slot into the Altair case and connect the computer to an ordinary cassette recorder, writing or reading data to or from the magnetic tape. Not only was a cassette machine much cheaper than a teletype, cassettes were more durable than paper, could store more data (up to 2200 bits per inch with Tarbell’s controller), and could be rewritten many times. Tarbell’s board sold for $150 assembled, $100 for a kit. He later branched out into floppy disk controllers and an interpreter for the BASIC computer language, and became a minor celebrity of the growing microcomputer scene.[4] Tarbell’s story offers a microcosm of the transition of personal computers, over the course of the 1970s, from an obscure niche hobby to a national industry. Like Hugo Gernsback in radio half a century before, home-computer tinkerers found themselves new roles in a growing hobby business as community-builders, publishers, and small-scale manufacturers. Like Tarbell, the first wave of these entrepreneurs responded directly to the Altair, offering supplemental hardware to offset its weaknesses or offering a more reliable or more capable hobby computer. The First Wave: Responding to Altair The Micro Instrumentation and Telemetry Systems (MITS) Altair came with a lot of potential, but it lay mostly unrealized in the basic kit MITS shipped out. This was partly intentional: the Altair sold on the basis of its exceptionally low price (less than $500), and it simply couldn’t remain so cheap if it had all the features of a full-fledged minicomputer system. Other deficiencies arose by accident, out of the amateurish nature of MITS. The good timing and negotiating skills of Ed Roberts, the company’s owner, had put him at the spearhead of the hobby computer revolution, but no one at his company had exceptional talent in electronics or product design. The Altair took hours to assemble, and the assembled machines often didn’t work. Follow-up accessories came out slowly as MITS technicians struggled to get them working. Tarbell’s cassette interface succeeded because it performed faster and more reliably than MITS’ equivalent. The most urgent need of the hobbyist other than easier input and output was additional memory beyond the scanty 256 bytes included with the base kit: far from enough to run a meaningful program, like a BASIC interpreter. In the spring of 1975, MITS started shipping a 4096-byte (4K) board designed by Roberts, but these boards simply didn’t work.[5] Unsurprisingly, other hobby-entrepreneurs began to step up quickly to fill the gaps. Several of them came from the most famous of the Altair-inspired hobby communities, the Homebrew Computer Club, which met in Silicon Valley and attracted attendees from around the Bay Area. Processor Technology was founded in Berkeley by Homebrew regular and electronics enthusiast Bob Marsh and his reclusive partner, Gary Ingram. In the spring of 1975, they began offering a 4K memory board for the Altair that actually worked. Later, the company came out with its own tape controller and a display board that would make Altair into a TV Typewriter, which they called VDM-1.[6] MITS’ 4K memory board compared to Processor Technology’s. Even without knowing anything about hardware design, it’s easy to see how sloppy the former is compared to the latter. [s100computers.com] Only one “authorized” Altair board maker existed, Cromemco, also located in the Bay Area. Cromemco founders Harry Garland and Roger Melen met as Ph.D. students in electrical engineering at Stanford (and named their company after their dormitory: Crothers Memorial). They contributed articles to Popular Electronics regularly, and found out about Altair while visiting the magazine’s offices in New York. They originally intended to build an interface board for the Altair that could read data from their “Cyclops” digital camera design. Despite the early partnership, no Cromemco board saw the light of day until 1976. Their slow start notwithstanding, Garland and Melen created two products of significance to MITS’ business and to the future of personal computing: the “Dazzler” graphics board and the “Bytesaver” read-only-memory (ROM). Unlike the TV Typewriter or the VDM-1, which could display only text, the Dazzler could paint arbitrary pixels onto the screen from an eight color palette (though only at a resolution of 64 x 64, or up to 128 x 128 in monochrome mode). Less sexy but equally significant, the Bytesaver board stored a program that would be immediately loaded into the Altair memory on power up; prior to that an Altair could do nothing until basic control instructions were keyed in manually to bootstrap it (instructing it, for example, to load another program from paper tape).[7] A 1976 ad for the Cromemco Dazzler [Byte (April 1976), 7] Roberts bristled at the competition from rival card makers. But more aggravating still were the rival computer makers cranking out Altair knock-offs. In 1974, Robert Suding and Deck Bemis had launched Digital Group out of Denver to support the Micro-8. After Altair came out, they decided to make their own, superior computer; Suding happily quit his steady but dull job at IBM to serve as the Woz to Bemis’ Jobs, avant la lettre. Digital Group computers came complete with an eight-kilobyte memory board, a cassette tape controller, and a ROM chip that could boot a program directly from tape. They also had a processor board independent of the backplane into which expansion cards slotted, which meant you could upgrade your processor without replacing any of your other boards. In short, they offered a computer hobbyist’s dream. The catch came in the form of poor quality control and very long waits for delivery, after paying cash up front.[8] Other would-be Altair-killers entered the market from around the country in 1975. Mike Wise, of Bountiful, Utah, created the Sphere, the first hobby computer with an integrated keyboard and display—although production was so limited that, decades later, vintage computer collectors would doubt whether any were actually built. The SWTPC 6800 came out of San Antonio, built by the same Southwest Technical Products Corporation that had sold parts for Don Lancaster’s TV Typewriter. A pair of Purdue graduate students in West Lafayette, Indiana wrote software for the SWTPC under the moniker of Technical Systems Consultants. A few hundred miles to the east, Ohio Scientific of Hudson, Ohio released a Microcomputer Trainer Board that put it, too, on the hobbyist map.[9] The SWTPC 6800. The bluntly rectangular cabinet design with the computer’s name prominent on the faceplate is typical of this era of microcomputers.[Michael Holley] But the real onslaught came in 1976. By that time hobbyists with entrepreneurial ambition had had time to fully absorb the lessons of the Altair, to hone their own skills at computer building, and to adopt new chips like the MOS Technology 6502 or Zilog Z80. The most significant releases of the year were the Apple Computer, MOS Technology KIM-1, IMSAI 8080, Processor Technology Sol-20, and, in the unkindest cut for Roberts, the Z-1 from former ally Cromemco. Most of these computer makers solved the upgrade problem in a more blunt fashion than the Digital Group’s sophisticated swappable boards: they simply copied the card interface protocol (known as the “bus”) of the Altair. Already own an Altair? Buy a Z-1 or Sol-20 and you could put all of the expansion cards for your old computer into the new. Cromemco founder Roger Melen encouraged the community to disassociate this interface from MITS by calling it the S100 bus, not the Altair bus—another twist of the knife.[10] Almost all of these businesses (excepting IMSAI, of whom more shortly) continued to exclusively target electronic hobbyists as their customers. The Z-1 looked just like an upmarket Altair, with a front panel now adorned with slightly nicer switches and lights. The Apple Computer and KIM-1 offered no frills at all, just a bare green printed circuit board festooned with chips and other components. Processor Technology’s Sol-20, inflected with Lee Felsenstein’s vision of a “Tom Swift” terminal for the masses, sported a handsome blue case with integrated keyboard and walnut side panels. This represented substantial progress in usability compared to the company’s first memory boards (which came only as a kit the buyer had to assemble), but the Sol-20 was still marketed via Popular Electronics as a piece of hobby equipment.[11] Software Entrepreneurs In early 1975, a computer hobbyist who wanted a minicomputer-like system of their own had only one low-price option: buy an Altair; then build, or wait for, or scrounge, the additional components that would make it into a functional system. Eighteen months later, abundance had replaced scarcity in the computer hobby hardware market, with many makes, models, and accessories to choose from. But what about software? A working computer consisted of metal, semi-conductor, and plastic, but also a certain quantity of “thought-stuff,” program text that would tell the computer what, exactly, to compute. A large proportion of the hobby community had a minicomputer background. They were accustomed to writing some software themselves and getting the rest (compilers, debuggers, math libraries, games, and more) from fellow users, often through organized community exchanges like the DEC user group program library. So, they expected to get microcomputer programs in the same way, through free exchange with fellow hobbyists. Even in the mainframe world, software was rarely sold independently of a hardware system prior to the 1970s.[12] It came as a shock, then, when, immediately on the heels of Altair, the first software entrepreneurs appeared. Paul Allen and Bill Gates—especially Gates—were roughly a decade younger than most of the early hardware entrepreneurs, at just 22 and 19, respectively. Compare to Ed Roberts of MITS at 33; Lee Felsenstein of Processor Technology, 29; Harry Garland of Cromemco, 28; Chuck Peddle of MOS Technology and Robert Suding of the Digital Group, both 37. These two young men from Seattle had caught the computer bug at the keyboard of their private school’s time-sharing terminal; they had finagled some computer time at a Seattle time-sharing company in exchange for finding bugs, but had no serious work experience that would have immersed them in the practices of the minicomputer world. For all their youth, though, Gates and Allen brimmed with ambition, and when they saw the Altair on the cover of Popular Electronics, they saw a business opportunity. Of course, everyone knew that a computer would need software to be useful, but it was not obvious that anyone would pay for that software. Gates and Allen, having not yet grown accustomed to getting software for free, had an easier time imagining that they would. They also knew that the first program any self-respecting hobbyist would want to get their hands on was a BASIC interpreter, so that they could run the huge existing library of BASIC software (especially games) and begin writing programs of their own. Gates and Allen in 1981. [MOHAI, King County News Photograph Collection, 2007.45.001.30.02, photo by Chuck Hallas] Like Cromemco, Gates and Allen started out as partners with MITS—within days of seeing they Altair cover, they contacted Ed Roberts promising a BASIC interpreter. They delivered in March, despite having no Altair, nor even an 8080 processor—they developed the program on a simulator written by Allen for the DEC PDP-10 at Harvard, where Gates was enrolled as a sophomore. In another debt to DEC, Gates based the syntax on Digital’s popular BASIC-PLUS. Allen moved to Albuquerque soon after, to head a new software division at MITS. Gates eventually followed to nurture their independent software venture, Micro-Soft, though he did not completely abandon Harvard until 1977.[13] Many hobbyists balked at the culture shock of paying for software, and freely exchanged paper tapes of Altair BASIC in defiance of Micro-Soft and MITS, prompting Gates’ famous “Open Letter to Hobbyists,” in February 1976. There he made the case that software writers deserved compensation for their work just as much as hardware builders did, prompting a flurry of amici curiae from various corners of the hobby (with far more weighing in for the defendants than the plaintiff). But, though this controversy is famous for its retrospective echoes of later debates over free software, Gates and Allen rendered the issue irrelevant almost immediately, by switching to a different business model. They began licensing BASIC to computer manufacturers at a flat fee, instead of a royalty on each copy sold. MITS paid $31,200, for example, for the BASIC for a new Altair model using the Motorola 6800 processor. The licensor could choose to charge for the software or not, Micro-Soft didn’t care, but they typically didn’t. This approach bypassed the cultural conflict altogether; BASIC interpreters and other systems software became a bullet point in a list of advertised features for a given piece of hardware rather than a separate item in the catalog.[14] Having a BASIC would let you run programs on your computer; but the other crucial linchpin for an easy-to-use microcomputer system was a program to manage your other programs and data. As faster and denser magnetic storage supplanted paper tape, computer users needed a way to quickly and easily move files between memory and their cassettes or floppy disks. By far the most popular tool for this purpose was CP/M, for Control Program for Microcomputers. CP/M was the creation of Gary Kildall, who got his hands on his first microcomputer directly from the source: Intel. Kildall grew up in Seattle and studied computer science at the University of Washington, where he had a brief run in with Gates and Allen, who at the time were teenagers who worked at a company part-owned by one of his professors, the Computer Center Corporation, in exchange for free computer time. Drafted into the army, Kildall used his connections at the University and his father’s position as a merchant marine instructor to get posted instead to naval officer training, and then a position as a math and computer science teacher at the Naval Postgraduate School in Monterey. After completing his obligations to the Navy in 1972, he stayed on as a civilian instructor.[15] Gary Kildall with his wife Dorothy, in 1978. [Computer History Museum] That same year, Kildall learned about the Intel 4004, and, like so many other computer enthusiasts, became enchanted with the idea of a computer of his own. The most obvious route was to get his hands on Intel’s development kit for the 4004, the SIM4-01, intended to be used by customers to write software for the new chip. So Kildall began talking to people at Intel, and then consulting at Intel, and in exchange for software written for Intel, managed to acquire microprocessor development kits for the 4004, and then later the 8008 and 8080 processors.[16] The most significant piece of software Kildall provided to Intel was PL/M, Programming Language for Microprocessors, which allowed developers to express code in a higher-level syntax that would then be compiled down to the 4004 (or 8008, or 8080) machine language. But you could not write PL/M on a microcomputer, it didn’t have the necessary mass storage interface or software tools; clients were expected to write programs on a minicomputer and then flash the final result onto a ROM chip that would power whatever microprocessor application they had in mind (a traffic light controller, for example, or a cash register.) What Kildall dreamed of was to “self-host” PL/M: that is, to author PL/M programs on the same computer on which they would run. By 1974 he had assembled everything he needed—a Intellec 8/80 development kit (for the 8080), a used hard drive and teletype, a disk controller board built by a friend—except for a program that could load and store the PL/M compiler, the code to be compiled, and the output of the compilation. It was for this reason, to complete his own personal quest, that he wrote CP/M.[17] Only after the fact did he think about selling it, just in time to catch the rising wave of hobby computers. Though Kildall later offered direct sales to users, he began with the same flat-fee license model that Micro-Soft had adopted: Kildall sold the software to Omron, a smart terminal maker, and then to IMSAI for their 8080 computer, each at a fee of $25,000. He incorporated his software business as Intergalactic Digital Research (later just Digital Research) in Pacific Grove, just west of Monterey. Gates visited in 1977 to float the idea of a California merger of the two (relative) giants of microcomputer software, but he and Allen decided to relocate to Seattle instead, leaving behind an intriguing what-if.[18] A CP/M command line interaction via a Tarbell disk controller, showing all the files on disk “A”. [Computer History Museum]      CP/M soon became the de-facto standard operating system for personal computers. Having an operating system made writing application software far easier, because basic routines like reading data from disk could be delegated to system calls instead of being re-written from scratch every time. CP/M in particular stood out for its quality in an often-slapdash hobby industry, and could easily be adapted to new platforms because of Kildall’s innovation of a Basic Input Output System (BIOS), which acted as a translation layer between the operating system and hardware. But what bootstrapped its initial popularity was the IMSAI deal, which attached Digital Research to the rising star in what up to that point had been Altair’s market to lose.[19] Getting Serious? There was one company thinking different about the microcomputer market in 1975: IMSAI, headquartered in San Leandro, California, intended to sell business machines. It had the right name for it, an acronym stuffed wall-to-wall with managerial blather: Information Management Sciences Associates, Inc. William (Bill) Millard was an IBM sales rep, then worked for San Francisco setting up computer systems, and founded IMS Associates to sell his services to companies who needed similar IT help. Bill Millard circa 1983. Provenance unknown. Despite the anodyne name he gave to his company, Millard, too, felt the influence of the ideologies of personal liberation that seemed to rise from San Francisco Bay like a fog. But unlike a Lee Felsenstein or a Bob Albrecht, he though mainly of liberating himself, not others: he was a devotee of Erhard Seminars Training, or est, a self-help seminar which promised paying customers access to an understanding of the world-changing power of their will in just two weekends; according to Erhard, “If you keep saying it \ the way it really is \ eventually your word \ is law in the universe.”[20] Neither Millard nor either of his technical employees (part-time programmer Bruce Van Natta and physicist-cum-electrical engineer Joseph Killian), had any prior interest or experience in home computers; they stumbled into the business almost by accident. Their primary contract, to build a computer networking hub for car dealerships based on a DEC computer, had begun spiraling towards failure. Casting about for some solution, they latched onto the news of Altair’s success: here was an inexpensive alternative to the DEC. When Altair refused to deliver on their timetable, they decided, in late summer of 1975, to clone it instead. And, to get cash flow going to pay their expenses and loans, they would sell their clone direct to consumers as well, while working to complete the big contract. When orders from hobbyists began to pour in, they abandoned the automotive scheme altogether to go all-in on their Altair clone.[21] The IMSAI 8080. It closely resembles the Altair, but with cleaner design and higher quality front-panel components. [Morn] The IMSAI 8080 began shipping in December 1975, at a kit price of $439. Millard cultivated an est culture at the company; employees with the “training” were favored, and total commitment to the work was expected. Some employees considered Millard a “genius or a prophet,” spouses and children of employees showed up after school to help assemble computers. By April, they were doing hundreds of thousands of dollars per month in sales. IMSAI was board-compatible with MITS but made improvements that stood out to the connoisseur: a more efficient internal layout, a cleaner and more professional exterior, and a seriously beefed-up power supply that could support a case fully loaded with expansion boards. These advantages appealed enough to buyers to make it Altair’s top competitor in 1976.[22] But what most set IMSAI apart in 1976 was the fact that it was not led by hobby entrepreneurs, but by a business man who wanted to build business machines. An advertisement in the May 1976 issue of BYTE magazine described the IMSAI as a “rugged, reliable, industrial computer with high commercial-type performance,” as opposed to “Altair’s hobbyist kit” (the IMSAI was of course also sold as a kit), along with obscure allusions to expensive IMSAI business products (Hypercube and Intelligent Disk) that never materialized. This was an odd pretense to put on while advertising in BYTE—a publication featuring articles such as “More to Blinking Lights than Meets the Eye” and “Save Money Using Mini Wire Wrap.”  This is not to say that IMSAI (or its contemporaries) had no commercial customers or applications. Alan Cooper, known later for creating Visual Basic, wrote a basic accounting program for the IMSAI in 1976 called General Ledger. But these applications remained a small minority among the mass of buyers who were computer-curious.[23] In 1977, IMSAI began advertising a “megabyte micro,” another fantasy. Such a powerful and expensive machine could sell in the higher end of the minicomputer market, but not to IMSAI’s actual buyers, hobbyists who were buying kits for less than a thousand dollars out of retail storefronts.IMSAI tried again to attract serious business customers with its second major product, the all-in-one VDP-80, which began shipping in late 1977 with an integrated keyboard, display, and dual disk drives, but it was plagued with quality defects, and lacked any application software for its would-be business customers to use.[24] Those customers did arrive in large numbers in good time, but only after a second wave of all-in-one computers appeared, aimed at the mass-market, and after the emergence of useful application software to run on them.

Read more
Steamships, Part 2: The Further Adventures of Isambard Kingdom Brunel

Iron Empire As far back as 1832, Macgregor Laird had taken the iron ship Alburkah to Africa and up the Niger, making it among the first ship of such construction to take the open sea. But the use of iron hulls in British inland navigation can be traced decades earlier, beginning with river barges in the 1780s. An iron plate had far more tensile strength than even an oaken board of the same thickness. This made an iron-hulled ship stronger, lighter, and more spacious inside than an equivalent wooden vessel: a two-inch thickness of iron might replace two-foot’s thickness of timber.[1]  The downsides included susceptibility to corrosion and barnacles, interference with compasses, and, at least at first, the expense of the material. As we have already seen, the larger the ship, the smaller the proportion of its cargo space that it would need for fuel; but the Great Western and British Queen pushed the limits of the practical size of a wooden ship (in fact, Brunel had bound Great Western’s hull with iron straps to bolster its longitudinal strength and prevent it from breaking in heavy seas).[2] The price of wood in Britain grew ever more dear as her ancient forests disappeared, but to build more massive ships economically also required iron prices to fall: and they did just that, starting in the 1830s, because of a surprisingly simple change in technique. Ironmongers had noticed long ago that their furnaces produce more metal from the same amount of fuel in the winter months. They assumed that the cooler air produced this result, and so by the nineteenth century it had become a basic tenet of the iron-making business that one should blast cool air into the furnace with the bellows to maximize its efficiency.[3] This common wisdom was mistaken; entirely backwards, in fact. In 1825, a Glasgow colliery engineer named James Neilson found that a hotter blast made the furnaces more efficient (it was the dryness, not the coolness, of the winter air that had made the difference). Neilson was asked to consult at an ironworks in the village of Muirkirk which was having difficulty with its furnace. He realized that heating the blast air would expand it, and thus increase the pressure of the air flowing into the furnace, strengthening the blast. In 1828 he patented the method of using a stove to heat the blast air. He convinced the Clyde Ironworks to adopt it, and together they perfected the method over the following few years. The results were astounding. A 600° F blast reduced coal consumption of the furnace by two-thirds and increased output from about five-and-a-half tons of pig iron per day to over eight.[4] On top of all that, this simple innovation allowed the use of plain coal as fuel in lieu of (more expensive) refined coke. Ironmakers had adopted coke in the 1750s because when iron was smelted with raw coal the impurities (especially sulfur) in the fuel made the resulting metal too brittle. But the hot blast sent the temperature inside the furnace so high that it drove the sulfur out in the slag waste rather than baking it into the iron. During the 1830s and 40s, Neilson’s hot blast technique spread from Scotland across all of Great Britain, and drove a rapid increase in iron production, from 0.7 million tons in 1830 to over two million in 1850. This cut the market price per ton of pig iron in half.[5] With its vast reserves of coal and iron, made accessible with the power of steam pumps (themselves made in Britain of British iron and fueled by British coal), Britain was perfectly placed to supply the demand induced by this decline in price. Much of the growth in iron output went to exports, strengthening the commercial sinews of the British empire while providing the raw material of industrialization to the rest of the world. The frenzies of railroad building in the United States and continental Europe in the middle of the nineteenth century relied heavily on British rails made from British iron: in 1849, for example, the Baltimore and Ohio railroad secured 22,000 tons of rails from a Welsh trading concern.[6] The hunger of the rapidly growing United States for iron proved insatiable; circa 1850 the young nation imported about 450,000 tons of British iron per year.[7] Good Engineering Makes Bad Business The virtues of iron were also soon on the brain of Isambard Kingdom Brunel. The Great Western Steam Ship Company’s plan for a successor to Great Western began sensibly enough; they would build a slightly improved sister ship of similar design. But Brunel and his partners were seduced, in the fall of 1838, by the appearance in Bristol harbor of an all-iron channel steamer called Rainbow, the largest such ship yet built. Brunel’s associates Claxton and Patterson took a reconnaissance voyage on her to Antwerp and upon their return all three men became convinced that they should build in iron.[8] As if that were not enough novelty to take on in one design, in May 1840 another innovative ship steamed into Bristol harbor, leaving Brunel and his associates swooning one more. The aptly named Archimedes, designed by Francis Petit Smith, swam through the water with unprecedented smoothness and efficiency, powered by a screw propeller rather than paddle wheels.[9] Any well-educated nineteenth-century engineer knew that paddles wasted a huge amount of energy pushing water down at the front of the wheel and lifting it up at the back. Nor was screw propulsion a surprising new idea in 1840. As we have seen, early steamboat inventors tried out just about every imaginable means of pushing or pulling a ship. In his very thorough Treatise on the Screw Propeller, the engineer John Bourne cites fifty some-odd proposals, patents, or practical attempts at screw propulsion prior toSmith’s.[10] After so many failures, most practical engineers assumed (reasonably enough) that the screw could never replace the proven (albeit wasteful) paddlewheel. The difficulties were numerous, including reducing vibration, transmitting power effectively to the screw, and choosing its shape, size, and angle among many potential alternatives. Most fundamental though, was producing sufficient thrust: early steam engines operated at modest speed, cycling every three seconds or so. At twenty revolutions per minute, a screw would have to be of an impractical diameter to actually push a ship forward rapidly. Smith overcame this last problem with a gearing system to allow the propeller shaft to turn 140 times per minute. His propeller design at first consisted of a true helical screw, of two turns (which created excessive friction), then later a single turn. Then, in 1840 he refitted Archimedes with a more recognizably modern propeller with two blades (each of half a turn).[11] Even with these design improvements, Brunel found that noise and vibration made the Archimedes of 1840 “uninhabitable” for passengers.[12]  But he had unshakeable faith in its potential. No doubt, advocates of the screw could tout many potential advantages over the paddlewheel: a lower center of gravity, a more spacious interior, more maneuverability in narrow channels, and more efficient use of fuel  (especially in headwinds, which caught the paddles full on, and rolling sidelong waves, which would lift one paddlewheel or the other out of the water).[13]  So, the weary investors of the Great Western Steam Ship Company saw the timetable of the  Great Britain’s construction set back once more, in order to incorporate a screw. As steamship historian Stephen Fox put it, “[i]n commercial terms, what the Great Western company needed in that fall of 1840 was a second ship, as soon as possible, to compete with the newly established Cunard line,” but that is not what they would get.[14] The completed ship finally launched in 1843, but did not take to sea for a transatlantic voyage until July 1845, having already cost the company some £200,000 pounds in total. With 322 feet of black iron hull driven by a 1000 horsepower Maudslay engine and a massive 36-ton propeller shaft, she dwarfed Great Western. Her all-iron construction gave an impression of gossamer lightness that fascinated a public used to burly wood.[15] The Launching of the Great Britain. But if her appearance impressed, her performance at sea did not. Her propeller fell apart, her engine failed to achieve the expected speed and she rolled badly in a swell. After major, expensive renovations in the winter of 1845, she ran aground at the end of the 1846 sailing season at Dundrum Bay off Ireland. Her iron hull proved sturdier than the organization that had constructed it: by the time she was at last floated free in August 1847, the Great Western Steam Company had already sunk. Another concern bought Great Britain for £25,000, and she ended up plying the route to Australia, operating mostly by sail.[16] In the long run, Brunel and his partners were right that iron hulls and screw propulsion would surpass wood and paddles, but Great Britain failed to prove it. The upstart Inman steamer line launched the iron-hulled, screw-powered City of Glasgow in 1850, which did prove that the ideas behind Great Britain could be turned to commercial success. But the more conservative Cunard line did not dispatch its first iron-hulled ship on its maiden voyage until 1856. Though even larger than Great Britain, at 376 feet and 3600 tons, the Persia still sported paddlewheels. This did not prevent her from booking more passengers than any other steamship to date, nor from setting a transatlantic speed record.[17] Not until the end of the 1860s did oceanic paddle steamers become obsolete. The Archimedes. Without any visible wheels, she looked deceptively like a typical sailing schooner, but for the telltale smokestack. A Glorious Folly For a time, Brunel walked away from shipbuilding. Then, late in 1851, he began crafting plans for a new liner to far surpass even Great Britain, one large enough to ply the routes to Indian and Australia without coaling stops on the African coast. Stopping to refuel wasted time but also quite a lot of money: coal in Africa cost far more than in Europe, because another ship had to bring it there in the first place.[18]    Because it would sail around Africa, not towards America, the new ship was christened Great Eastern. Monstrous in all its dimensions, the Great Eastern, can only be regarded as a monster in truth, in the archaic sense of “a prodigy birthed outside the natural order of things”; it was without precedent and without issue.[19] Given the total failure of Brunel’s last steam liner company, not to mention other examples of excessive exuberance in his past, such as an atmospheric railway project that shut down within a year, it is hard to conceive of how he was able to convince new backers to finance this wild new idea. He did have the help of one new ally, an ambitious Scottish shipbuilder named John Russell, who was also wracked by career disappointment and eager for a comeback. Together they built an astonishing vessel: at 690 feet long and over 22,000 tons, it exceeded in size every other ship built to its time, and also every other ship built in the balance of the nineteenth century. It would carry (in theory) 4,000 passengers and 18,000 tons of coal or cargo, and mount both paddlewheels and a propeller, the latter powered by the largest steam engine ever built, of 1600 horsepower. Brunel died of a stroke in 1859, and never saw the ship take to sea. That is just as well, for it failed even more brutally than the Great Britain. It was slow, rolled badly, maneuvered poorly, and demanded prodigious quantities of labor and fuel.[20] Like Great Britain, after a brief service its owners auctioned it off to new buyers at a crushing loss. Great Eastern did, however, have still in its future a key role to play in the extension of British imperial and commercial power, as we shall see. The Great Eastern in harbor in Wales in 1860. Note the ‘normal-size’ three-masted ship in the foreground for scale. I have lingered on Brunel’s career for so long not because he was of unparalleled import to the history of the age of steam (he was not), but because his character and his ambition fascinate me. He innovated boldly, but rarely as effectively as his more circumspect peers, such as Samuel Cunard. Much—though certainly not all—of his career consists of glorious failure. Whether you, dear reader, emphasize the glory or the failure, may depend on the width of the romantic streak that runs through your soul.

Read more
Steamships, Part I: Crossing the Atlantic

For much of this story, our attention has focused on events within the isle of Great Britain, and with good reason: primed by the virtuous cycle of coal, iron, and steam, the depth and breadth of Britain’s exploitation of steam power far exceeded that found anywhere else, for roughly 150 years after the groaning, hissing birth cry of steam power with the first Newcomen engine. American riverboat traffic stands out as the isolated exception. But Great Britain, island though it was, did not stand aloof from the world. It engaged in trade and the exchange of ideas, of course, but it also had a large and (despite occasional setbacks) growing empire, including large possessions in Canada, South Africa, Australia, and India. The sinews of that empire necessarily stretched across the oceans of the world, in the form of a dominant navy, a vast merchant fleet, and the ships of the East India Company, which blurred the lines of military and commercial power: half state and half corporation. Having repeatedly bested all its would-be naval rivals—Spain, the Netherlands, and France—Britain had achieved an indisputable dominance of the sea. Testing the Waters The potential advantages of fusing steam power with naval power were clear: sailing ships were slaves to the whims of the atmosphere. A calm left them helpless, a strong storm drove them on helplessly, and adverse winds could trap them in port for days on end. The fickleness of the wind made travel times unpredictable and could steal the opportunity for a victorious battle from even the strongest fleet. In 1814, Sir Walter Scott took a cruise around Scotland, and the vicissitudes of travel by sail are apparent on page after page of his memoirs:  4th September 1814… Very little wind, and that against us; and the navigation both shoally and intricate. Called a council of war; and after considering the difficulty of getting up to Derry, and the chance of being windbound when we do get there, we resolve to renounce our intended visit to that town… 6th September 1814… When we return on board, the wind being unfavourable for the mouth of Clyde, we resolve to weigh anchor and go into Lamlash Bay. 7th September, 1814 – We had amply room to repent last night’s resolution, for the wind, with its usual caprice, changed so soon as we had weighed anchor, blew very hard, and almost directly against us, so that we were beating up against it by short tacks, which made a most disagreeable night…[1] As it had done for power on land, as it had done for river travel, so steam could promise to do for sea travel: bring regularity and predictability, smoothing over the rough chaos of nature. The catch lay in the supply of fuel. A sailing ship, of course, needed only the “fuel” it gathered from the air as it went along. A riverboat could easily resupply its fuel along the banks as it travelled. A steamship crossing the Atlantic would have to bring along its whole supply. Plan of the Savannah. It is evident that she was designed as a sailing ship, with the steam engine and paddles as an afterthought. Early attempts at steam-powered sea vessels bypassed this problem by carrying sails, with the steam engine providing supplementary power. The American merchant ship Savannah crossed the Atlantic to Liverpool in this fashion in 1819. But the advantages of on-demand steam power did not justify the cost of hauling an idle engine and its fuel across the ocean. Its owners quickly converted the Savannah back to a pure sailing ship.[2] MacGregor Laird had a better-thought-out plan in 1832 when he dispatched the two steamships built at his family’s docks, Quorra and Alburkah, along with a sailing ship, for an expedition up the River Niger to bring commerce and Christianity to central Africa. Laird’s ships carried sails for the open ocean and supplied themselves regularly with wooden fuel when coasting near the shore. The steam engines achieved their true purpose once the little task force reached the river, allowing the ships to navigate easily upstream.[3] Brunel Laird’s dream of transforming Africa ended in tatters, and in the death of most of his crew. But Laird himself survived, and he and his homeland would both have a role to play in the development of true ocean-going steamships. Laird, like the great Watt himself, was born in Greenock, on the Firth of Clyde, and Britain’s first working commercial steamboats originated on the Clyde, carrying passengers among Glasgow, Greenock, Helensburgh, and other towns. Scott took passage on such a ferry from Greenock to Glasgow in the midst of his Scottish journey, and the contrast is stark in his memoirs between his passages at sea and the steam transit on the Clyde that proceeded “with a smoothness of motion which probably resembles flying.”[4] The shipbuilders of the Clyde, with iron and coal closet a hand, would make such smooth, predictable steam journeys ever more common in the waters of and around Britain.  By 1822, they had already built forty-eight steam ferries of the sort on which Scott had ridden; in the following decade ship owners extended service out into the Irish Sea and English Channel with larger vessels, like David Napier’s 240-ton, 70-horsepower Superb and 250-ton and 100-horsepower Majestic.[5] Indeed, the most direct path to long-distance steam travel lay in larger hulls. Because of the buoyancy of water, steamships did not suffer rocket-equation-style negative returns on fuel consumption with increasing size. As the hull grew, its capacity to carry coal increased in proportion to its volume, while the drag the engines had to overcome (and thus the size of engine required) increased only in proportion to the surface area. Mark Beaufoy, a scholar of many pursuits but with a deep interest in naval matters, had shown this decisively in a series of experiments with actual hulls in water, published posthumously by his son in 1834.[6] In the late 1830s, two competing teams of British financiers, engineers, and naval architects emerged, racing to be the first to take advantage of this fact by creating a large enough steamship to make transatlantic steam travel technically and commercially viable. In a lucky break for your historian, the more successful team was led by the more vibrant figure, Isambard Kingdom Brunel: even his name oozes character. (His rival’s name, Junius Smith, begins strong but ends pedestrian.) Brunel’s unusual last name came from his French father, Marc Brunel; his even more unusual middle name came from his English mother, Sophia Kingdom; and his most unusual first name descends from some Frankish warrior of old.[7] The elder Brunel came from a prosperous Norman farming family. A second son, he was to be educated for the priesthood, but rebelled against that vocation and instead joined the navy in 1786. Forced to flee France in 1793 due to his activities in support of the royalist cause, he worked for a time as a civil engineer in New York before moving to England in 1799 to develop a mechanized process for churning out pulley blocks for the British navy with one of the great rising engineers of the day, Henry Maudslay.[8] The most famous image of Brunel, in front of the chains of his (and the world’s) largest steamship design in 1857. Young Isambard was born in 1806, began working for his father in 1822, and got the railroad bug after riding the Liverpool and Manchester line in 1831.  The Great Western Railway (GWR) company named Brunel as chief engineer in 1833, when he just twenty-seven years old. The GWR originated with a group of Bristol merchants who saw the growth of Liverpool, and feared that without a railway link to central Britain they would lose their status as the major entrepôt for British trade with the United States. It spanned the longest route of any railway to date, almost 120 miles from London to Bristol, and under Brunel’s guidance the builders of the GWR leveled, bridged, and tunneled that route at unparalleled cost). Brunel insisted on widely spaced rails (seven feet apart) to allow a smooth ride at high speed, and indeed GWR locomotives achieved speeds of sixty miles-per-hour, with average speeds of over forty miles-per-hour over long distances, including stops. Though the broad-gauge rails Brunel stubbornly fought for are long gone, the iron-ribbed vaults of the train sheds he designed for each terminus—Paddington Station in London and Temple Meads in Bristol—still stand and serve railroad traffic today.[9] The Great Western Railway " data-medium-file="https://cdn.accountdigital.net/Fhf3soIAhhg2rN0oqVUZMO0J_BSv" data-large-file="https://cdn.accountdigital.net/Fijn2JZJiY1iiPlxXFJdkKowbSRt?w=739" loading="lazy" width="1024" height="640" src="https://cdn.accountdigital.net/FhOeLf78Au4ONW9H2uRKenokq84m" alt="" class="wp-image-14501" srcset="https://cdn.accountdigital.net/FhOeLf78Au4ONW9H2uRKenokq84m 1024w, https://cdn.accountdigital.net/Fi_GgFaxxSyF_JLeBaV4adatJV4f 150w, https://cdn.accountdigital.net/Fhf3soIAhhg2rN0oqVUZMO0J_BSv 300w, https://cdn.accountdigital.net/Ft7LTlTqReE0HzUJR2_XF8PFniC4 768w, https://cdn.accountdigital.net/Fijn2JZJiY1iiPlxXFJdkKowbSRt 1154w" sizes="(max-width: 1024px) 100vw, 1024px">An engraving of Temple Mead, Bristol terminus of the Great Western Railway. According to legend, Brunel’s quest to build a transatlantic steamer began with an off-hand quip at a meeting of the Great Western directors in October 1835.[10] Someone grumbled over the length of the railway line, Brunel said something to the effect of: “Why not make it longer, and have a steamboat to go from Bristol to New York?” Though perhaps intended as a joke, Brunel’s remark spoke to the innermost dreams of the Bristol merchants, to be the indispensable link between England and America.  One of them, Thomas Guppy, decided to take the idea seriously, and convinced Brunel to do the same. Brunel, never lacking in self-confidence, did not doubt that his heretofore landbound engineering skills would translate to a watery milieu, but just in case he pulled Christopher Claxton (a naval officer) and William Patterson (a shipbuilder) in on the scheme. Together they formed a Great Western Steam Ship Company.[11] The Race to New York Received opinion still held that a direct crossing by steam from England to New York, of over 3,000 miles, would be impossible without refueling. Dionysius Lardner took to the hustings of the scientific world to pronounce that opinion. Dionysius Lardner, Brunel’s nemesis. One of the great enthusiasts and promoters of the railroad, Lardner was nonetheless a long-standing opponent of Brunel’s: in 1834 he had opposed Brunel’s route for the Great Western railway on the grounds that the gradient of Box Hill tunnel would cause trains to reach speeds of 120 miles-per-hour and thus suffocate the passengers.[12] He gave a talk to the British Association for the Advancement of Science in August 1836 deriding the idea of a Great Western Steamship, asserting that “[i]n proportion as the capacity of the vessel is increased, in the same ratio or nearly so must the mechanical power of the engines be enlarged, and the consumption of fuel augmented,” and that therefore a direct trip across the Atlantic would require a far more efficient engine than had ever yet been devised.[13] The Dublin-born Lardner much preferred his own scheme to drive a rail line across Ireland and connect the continents by the shortest possible water route: 2,000 miles from Shannon to Newfoundland. Brunel, however, firmly believed that a large ship would solve the fuel problem. As he wrote in a preliminary report to the company in 1836, certainly drawing on Beaufoy’s work: “…the tonnage increases as the cubes of their dimensions, while the resistance increases about as their squares; so that a vessel of double the tonnage of another, capable of containing an engine of twice the power, does not really meet with double the resistance.”[14] He, Patterson and Claxton agreed to target a 1400 ton, 400 horsepower ship. They would name her, of course, Great Western. In the post-Watt era, Britain boasted two great engine-building firms: Robert Napier’s in Glasgow in the North, and Maudslay’s in London in the south. After the death of Henry Maudslay, Marc Brunel’s former collaborator, in 1831, the business’ ownership passed to his sons. But they lacked their father’s brilliance; the key to  the firm’s future lay with the partner he had also bequeathed  to them, Joshua Field. Brunel and his father both had ties to Maudslay, and so they tapped Field to design the engine for their great ship. Field chose a “side-lever” engine design, so-called because a horizontal beam on the side of the engine rocking on a central pivot delivered power from the piston to the paddle wheels. This was the standard architecture for large marine engines, because it allowed the engine to be mounted deep in the hull, avoiding deck obstructions and keeping the ship’s center of gravity low. Field, however, added several novel features of his own devising. The most important of them was the spray condenser, which recycled some of the engine’s steam for re-use as fresh water for the boiler. This ameliorated the second-most pressing problem for long-distance steamships: the build-up of scale in the engine from saltwater.[15] The 236-foot-long, 35-foot-wide hull sported iron bracings to increase its strength (a contribution of Brunel), and cabins for 128 passengers. The extravagant, high-ceiling grand saloon provided a last, luxurious Brunel touch. By far the largest steamship yet built, Great Western would have towered over most other ships in the London docks where she was built.[16] The competing group around Junius Smith had not been idle. Smith, an American-born merchant who ran his business out of London had dreamed of a steam-powered Atlantic crossing ever since 1832, when while idling on a fifty-four day sail from England to New York; almost twice the usual duration. He formed the British and American Steam Navigation Company, and counted among his backers Macgregor Laird, the Scottish shipbuilder of the Niger River expedition. Their 1800-ton British Queen would boast a 500-horsepower engine, built by the Maudslay company’s Scottish rival, Robert Napier.[17] But Smith’s group fell behind the Brunel consortium (this despite the fact that Brunel still led the engineering on the not-yet-completed Great Western Railway); the Great Western would launch first. In a desperate stunt to be able to boast of making the first Atlantic crossing, British and American launched the channel steamer Sirius on April 4, 1838 from Cork on the west coast of Ireland, laden with fuel and bound for New York. Great Western left Bristol just four days later, with fifty-seven crew (fifteen of them just for stoking coal) to serve a mere seven passengers, each paying the princely sum of 35 guineas for passage.[18] The Steamer Great Western. H.R. Robinson. PAH8859 " data-medium-file="https://cdn.accountdigital.net/FjhWpIE1aFlqt2GYL0Hc-SWJMgY9" data-large-file="https://cdn.accountdigital.net/Flj-dfOyg0Ro7Cz2Hb7HjGMwK6WT?w=739" loading="lazy" width="1024" height="730" src="https://cdn.accountdigital.net/Fu9OpVwe0NXae4jdZHJyPyuWk0m9" alt="" class="wp-image-14505" srcset="https://cdn.accountdigital.net/Fu9OpVwe0NXae4jdZHJyPyuWk0m9 1024w, https://cdn.accountdigital.net/FuVJjI8esdiqejo2CGzqD47Y3gkI 150w, https://cdn.accountdigital.net/FjhWpIE1aFlqt2GYL0Hc-SWJMgY9 300w, https://cdn.accountdigital.net/FnNNJhcavKPniiW0ouC1ntqj7v0o 768w, https://cdn.accountdigital.net/Flj-dfOyg0Ro7Cz2Hb7HjGMwK6WT 1280w" sizes="(max-width: 1024px) 100vw, 1024px">A Lithograph of the Great Western. Despite three short stops to deal with engine problems and a near-mutiny by disgruntled coal stokers working in miserable conditions, Great Western nearly overtook Sirius, arriving in New York just twelve hours behind her. In total the crossing took less than sixteen days—about half the travel time of a fast sailing packet—with coal to spare in the bunkers. The ledger was not all positive: the clank of the engine, the pall of smoke and the ever-present coating of soot and coal dust drained the ocean of some of its romance; as historian Stephen Fox put it, “[t]he sea atmosphere, usually clean and bracing, felt cooked and greasy.” But sixty-six passengers ponied up for the return trip: “Already… ocean travelers had begun to accept the modernist bargain of steam dangers and discomforts in exchange for consistent, unprecedented speed.”[19] In that first year, Great Western puffed alone through Atlantic waters. Itmade four more round trips In 1838, eking out a small profit. The British Queen launched at last in July 1839, and British and American launched an even larger ship, SS President, the following year. Among the British Queen’s first passengers on its maiden voyage to New York was Samuel Cunard, a name that would resonate in ocean travel for a century to come, and an object lesson in the difference between technical and business success. In 1840 his Cunard Line began providing transatlantic service in four Britannia-class paddleships. Imitation Great Westerns (on a slightly smaller scale), they stood out not for their size or technical novelty but for their regularity and uniformity of service. But the most important factor in Cunard’s success was outmaneuvering the Great Western Steam Company in securing a contract with the Admiralty for mail service to Halifax. This provided a steady and reliable revenue stream—starting at 60,000 pounds a year—regardless of economic downturns. Moreover, once the Navy had come to depend on Cunard for speedy mail service it had little choice but to keep upping the payments to keep his finances afloat.[20] Thanks to the savvy of Cunard, steam travel from Britain to America, a fantasy in 1836 (at least according to the likes of Dionysius Lardner), had become steady business four years later. Brunel, however, had no patience for the mere making of money. He wanted to build monuments; creations to stand the test of time, things never seen or done before. So, when, soon after the launching of the Great Western, he began to design his next great steam ship, he decided he would build it with a hull of solid iron.

Read more
Internet Ascendant, Part 2: Going Private and Going Public

In the summer of 1986, Senator Al Gore, Jr., of Tennessee introduced an amendment to the Congressional Act that authorized the  budget of the National Science Foundation (NSF). He called for the federal government to study the possibilities for “communications networks for supercomputers at universities and Federal research facilities.” To explain the purpose of this legislation, Gore called on a striking analogy: One promising technology is the development of fiber optic systems for voice and data transmission. Eventually we will see a system of fiber optic systems being installed nationwide. America’s highways transport people and materials across the country. Federal freeways connect with state highways which connect in turn with county roads and city streets. To transport data and ideas, we will need a telecommunications highway connecting users coast to coast, state to state, city to city. The study required in this amendment will identify the problems and opportunities the nation will face in establishing that highway.1In the following years, Gore and his allies would call for the creation of an “information superhighway”, or, more formally, a national information infrastructure (NII). As he intended, Gore’s analogy to the federal highway system summons to mind a central exchange that would bind together various local and regional networks, letting all American citizens communicate with one another. However, the analogy also misleads – Gore did not propose the creation of a federally-funded and maintained data network. He envisioned that the information superhighway, unlike its concrete and asphalt namesake, would come into being through the action of market forces, within a regulatory framework that would ensure competition, guarantee open, equal access to any service provider (what would later be known as “net neutrality”), and provide subsidies or other mechanisms to ensure universal service to the least fortunate members of society, preventing the emergence of a gap between the information rich and information poor.2Over the following decade, Congress slowly developed a policy response to the growing importance of computer networks to the American research community, to education, and eventually to society as a whole. Congress’ slow march towards an NII policy, however, could not keep up with the rapidly growing NSFNET, overseen by the neighboring bureaucracy of the executive branch. Despite its reputation for sclerosis, bureaucracy was created exactly because of its capacity, unlike a legislature, to respond to events immediately, without deliberation. And so it happened that, between 1988 and 1993, the NSF crafted the policies that would determine how the Internet became private, and thus went public. It had to deal every year with novel demands and expectations from NSFNET’s users and peer networks. In response, it made decisions on the fly, decisions which rapidly outpaced Congressional plans for guiding the development of an information superhighway. These decisions rested largely in the hands of a single man – Stephen Wolff.Acceptable UseWolff earned a Ph.D. in electrical engineering at Princeton in 1961 (where he would have been a rough contemporary of Bob Kahn), and began what might have been a comfortable academic career, with a post-doctoral stint at Imperial College, followed by several years teaching at Johns Hopkins. But then he shifted gears, and took a position  at the Ballistics Research lab in Aberdeen, Maryland. He stayed there for most of the 1970s and early 1980s, researching communications and computing systems for the U.S. Army. He introduced Unix into the lab’s offices, and managed Aberdeen’s connection to the ARPANET.3In 1986, the NSF recruited him to manage the NSF’s supercomputing backbone – he was a natural fit, given his experience connecting Army supercomputers to ARPANET. He became the principal architect of NSFNET’s evolution from that point until his departure in 1994, when he entered the private sector as a manager for Cisco Systems. The original intended function of the net that Wolff was hired to manage had been to connect researchers across the U.S. to NSF-funded supercomputing centers. As we saw last time, however, once Wolff and the other network managers saw how much demand the initial backbone had engendered, they quickly developed a new vision of NSFNET, as a communications grid for the entire American research and post-secondary education community.However, Wolff did not want the government to be in the business of supplying network services on a permanent basis. In his view, the NSF’s role was to prime the pump, creating the initial demand needed to get a commercial networking services sector off the ground. Once that happened, Wolff felt it would be improper for a government entity to be in competition with viable for-profit businesses. So he intended to get NSF out of the way by privatizing the network, handing over control of the backbone to unsubsidized private entities and letting the market take over.This was very much in the spirit of the times. Across the Western world, and across most of the political spectrum, government leaders of the 1980s touted privatization and deregulation as the best means to unleash economic growth and innovation after the relative stagnation of the 1970s. As one example among many, around the same time that NSFNET was getting off the ground, the FCC knocked down several decades-old constraints on corporations involved in broadcasting. In 1985, it removed the restriction on owning print and broadcast media in the same locality, and two year later it nullified the fairness doctrine, which had required broadcasters to present multiple views on public-policy debates. From his post at NSF, Wolff had several levers at hand for accomplishing his goals. The first lay in the interpretation and enforcement of the network’s acceptable use policy (AUP). In accordance with NSF’s mission, the initial policy for the NSFNET backbone, in effect until June 1990, required all uses of the network to be in support of “scientific research and other scholarly activities.” This is quite restrictive indeed, and would seem to eliminate any possibility of commercial use of the network. But Wolff chose to interpret the policy liberally. Regularly mailing list postings about new product releases from a corporation that sold data processing software – was that not in support of scientific research? What about the decision to allow MCI’s email system to connect to the backbone, at the urging of Vint Cerf, who had left government employ to oversee the development of MCI Mail. Wolff rationalized this – and other later interconnections to commercial email systems such as CompuServe’s – as in support of research by making it possible for researchers to communicate digitally with a wider range of people that they might need to contact in the pursuit of their work. A stretch, perhaps. But Wolff saw that allowing some commercial traffic on the same infrastructure that was used for public NSF traffic would encourage the private investment needed to support academic and educational use on a permanent basis. Wolff’s strategy of opening the door of NSFNET as far as possible to commercial entities got an assist from Congress in 1992, when Congressman Rick Boucher, who helped oversee NSF as chair of the Science Subcommittee, sponsored an amendment to the NSF charter which authorized any additional uses of NSFNET that would “tend to increase the overall capabilities of the networks to support such research and education activities.” This was an ex post facto validation of Wolff’s approach to commercial traffic, allowing virtually any activity as long as it produced profits that encouraged more private investment into NSFNET and its peer networks.  Dual-Use NetworksWolff also fostered the commercial development of networking by supporting the regional networks’ reuse of their networking hardware for commercial traffic. As you may recall, the NSF backbone linked together a variety of not-for-profit regional nets, from NYSERNet in New York to Sesquinet in Texas to BARRNet in northern California. NSF did not directly fund the regional networks, but it did subsidize them indirectly, via the money it provided to labs and universities to offset the costs of their connection to their neighborhood regional net. Several of the regional nets then used this same subsidized infrastructure to spin off a for-profit commercial enterprise, selling network access to the public over the very same wires used for the research and education purposes sponsored by NSF. Wolff encouraged them to do so, seeing this as yet another way to accelerate the transition of the nation’s research and education infrastructure to private control. This, too, accorded neatly with the political spirit of the 1980s, which encouraged private enterprise to profit from public largesse, in the expectation that the public would benefit indirectly through economic growth. One can see parallels with the dual-use regional networks in the 1980 Bayh-Dole Act, which defaulted ownership of patents derived from government-funded research to the organization performing the work, not to the government that paid for it. The most prominent example of dual-use in action was PSINet, a for-profit company initially founded as Performance Systems International in 1988. William Schrader and Martin Schoffstall, the co-founder of NYSERNet and one of vice presidents’, respectively, created the company. Schofstall, a former BBN engineer and co-author of the Simple Network Management Protocol (SNMP) for managing the devices on an IP network, was the key technical leader. Schrader, an ambitious Cornell biology major and MBA who had helped his alma mater set up its supercomputing center and get it connected to NSFNET, provided the business drive. He firmly believed that NYSERNet should be selling service to businesses, not just educational institutions. When the rest of the board disagreed, he quit to found his own company, first contracting with NYSERNet for service, and later raising enough money to acquire its assets. PSINet thus became one of the earliest commercial internet service providers, while continuing to provide non-profit service to colleges and universities seeking access to the NSFNET backbone.4Wolff’s final source of leverage for encouraging a commercial Internet lay in his role as manager of the contracts with the Merit-IBM-MCI consortium that operated the backbone. The initial impetus for change in this dimension came not from Wolff, however, but from the backbone operators themselves.  A For-Profit BackboneMCI and its peers in the telecommunications industry had a strong incentive to find or create more demand for computer data communications. They had spent the 1980s upgrading their long-line networks from coaxial cable and microwave – already much higher capacity than the old copper lines – to fiber optic cables. These cables, which transmitted laser light through glass, had tremendous capacity, limited mainly by the technology in the transmitters and receivers on either end, rather than the cable itself. And that capacity was far from saturated. By the early 1990s, many companies had deployed OC-48 transmission equipment with 2.5 Gbps of capacity, an almost unimaginable figure a decade earlier. An explosion in data traffic would therefore bring in new revenue at very little marginal cost – almost pure profit.5The desire to gain expertise in the coming market in data communications helps explains why MCI was willing to sign on to the NSFNET bid proposed by Merit, which massively undercut the competing bids (at $14 million for five years, versus the $40 and $25 millions proposed by their competitors6), and surely implied a short-term financial loss for MCI and IBM. But by 1989, they hoped to start turning a profit from their investment. The existing backbone was approaching the saturation point, with 500 million packets a month, a 500% year-over-year increase.7 So, when NSF asked Merit to upgrade the backbone from 1.5 Mbps T1 lines to 45Mbps T3, they took the opportunity to propose to Wolff a new contractual arrangement.T3 was a new frontier in networking – no prior experience or equipment existed for digital networks of this bandwidth, and so the companies argued that more private investment would be needed, requiring a restructuring that would allow IBM and Merit to share the new infrastructure with for-profit commercial traffic – a dual-use backbone. To achieve this, the consortium would from a new non-profit corporation, Advanced Network & Services, Inc. (ANS), which would supply T3 networking services to NSF. A subsidiary called ANS CO+RE systems would sell the same services at a profit to any clients willing to pay. Wolff agreed to this, seeing it as just another step in the transition of the network towards commercial control. Moreover, he feared that continuing to block commercial exploitation of the backbone would lead to a bifurcation of the network, with suppliers like ANS doing an end-run around NSFNET to create their own, separate, commercial Internet. Up to that point, Wolff’s plan for gradually getting NSF out of the way had no specific target date or planned milestones. A workshop on the topic held at Harvard in March 1990, in which Wolff and many other early Internet leaders participated, considered a variety of options without laying out any concrete plans.8 It was ANS’ stratagem that triggered the cascade of events that led directly to the full privatization and commercialization of NSFNET.It began with a backlash. Despite Wolff’s good intentions, IBM and MCI’s ANS maneuver created a great deal of disgruntlement in the networking community. It became a problem exactly because of the for-profit networks attached to the backbone that Wolff had promoted. So far they had gotten along reasonably with one another, because they all operated as peers on the same terms. But with ANS, a for-profit company held a de-facto monopoly on the backbone at the center of the Internet.9 Moreover, despite Wolff’s efforts to interpret the AUP loosely, ANS chose to interpret it strictly, and refused to interconnect the non-profit portion of the backbone (for NSF traffic) with any of their for-profit networks like PSI, since that would require a direct mixing of commercial and non-commercial traffic. When this created an uproar, they backpedaled, and came up with a new policy, allowing interconnection for a fee based on traffic volume.PSINet would have none of this. In the summer of 1991, they banded together with two other for-profit Internet service providers – UUNET, which had begun by selling commercial access to Usenet before adding Internet service; and the California Education and Research Federation Network, or CERFNet, operated by General Atomics – to form their own exchange, bypassing the ANS backbone. The Commercial Internet Exchange (CIX) consisted at first of just a single routing center in Washington D.C. which could transfer traffic among the three networks. They agreed to peer at no charge, regardless of the relative traffic volume, with each network paying the same fee to CIX to operate the router. New routers in Chicago and Silicon Valley soon followed, and other networks looking to avoid ANS’ fees also joined on.DivestitureRick Boucher, the Congressman whom we met above as a supporter of NSF commercialization, nonetheless requested an investigation of the propriety of Wolff’s actions in the ANS affair by the Office of the Inspector General. It found NSF’s actions precipitous, but not malicious or corrupt. Nevertheless, Wolff saw that the time had come to divest control of the backbone. With ANS + CORE and CIX privatization and commercialization had begun in earnest, but in a way that risked splitting the unitary Internet into multiple disconnected fragments, as CIX and ANS refused to connect with one another. NSF therefore drafted a plan for a new, privatized network architecture in the summer of 1992, released it for public comment, and finalized it in May of 1993. NSFNET would shut down in the spring of 1995, and its assets would revert to IBM and MCI. The regional networks could continue to operate, with financial support from the NSF gradually phasing out over a four year period, but would have to contract with a private ISP for internet access.But in a world of many competitive internet access providers, what would replace the backbone? What mechanism would link these opposed private interests into a cohesive whole? Wolff’s answer was inspired by the exchanges already built by cooperatives like CIX – NSF would contract out the creation of four Network Access Points (NAPs), routing sites where various vendors could exchange traffic. Having four separate contracts would avoid repeating the ANS controversy, by preventing a monopoly on the points of exchange. One NAP would reside at the pre-existing, and cheekily named, Metropolitan Area Ethernet East (MAE-East) in Vienna, Virginia, operated by Metropolitan Fiber Systems (MFS). MAE-West, operated by Pacific Bell, was established in San Jose, California; Sprint operated another NAP in Pennsauken, New Jersey, and Ameritech one in Chicago. The transition went smoothly10, and NSF decommissioned the backbone right on schedule, on April 30, 1995.11The Break-upThough Gore and others often invoked the “information superhighway” as a metaphor for digital networks, there was never serious consideration in Congress of using the federal highway system as a direct policy model. The federal government paid for the building and maintenance of interstate highways in order to provide a robust transportation network for the entire country. But in an era when both major parties took deregulation and privatization for granted as good policy, a state-backed system of networks and information services on the French model of Transpac and Minitel was not up for consideration.12Instead, the most attractive policy model for Congress as it planned for the future of telecommunication was the long-distance market created by the break-up of the Bell System between 1982 and 1984. In 1974, the Justice Department filed suit against AT&T, its first major suit against the organization since the 1950s, alleging that it had engaged in anti-competitive behavior in violation of the Sherman Antitrust Act. Specifically, they accused the company of using its market power to exclude various innovative new businesses from the market – mobile radio operators, data networks, satellite carriers, makers of specialized terminal equipment, and more. The suit thus clearly drew much of its impetus from the ongoing disputes since the early 1960s (described in an earlier installment), between AT&T and the likes of MCI and Carterfone.When it became clear that the Justice Department meant business, and intended to break the power of AT&T, the company at first sought redress from Congress. John de Butts, chairman and CEO since 1972, attempted to push a “Bell bill” – formally the Consumer Communications Reform Act – through Congress. It would have enshrined into law AT&T’s argument that the benefits of a single, universal telephone network far outweighed any risk of abusive monopoly, risks which in any case the FCC could already effectively check. But the proposal received stiff opposition in the House Subcommittee on Communications, and never reached a vote on the floor of either Congressional chamber. In a change of tactics, in 1979 the board replaced the combative de Butts – who had once declared openly to an audience of state telecommunications regulators the heresy that he opposed competition and espoused monopoly – with the more conciliatory Charles Brown. But it was too late by then to stop the momentum of the antitrust case, and it became increasingly clear to the company’s leadership that they would not prevail. In January 1982, therefore, Brown agreed to a consent decree that would have the presiding judge in the case, Harold Greene, oversee the break-up of the Bell System into its constituent parts.The various Bell companies that brought copper to the customer’s premise, which generally operated by state (New Jersey Bell, Indiana Bell, and so forth) were carved up into seven blocks called Regional Bell Operating Companies (RBOCs). Working clockwise around the country, they were NYNEX in the northeast, Bell Atlantic, Bell South, Southwestern Bell, Pacific Telesis, US West, and Ameritech. All of them remained regulated entities with an effective monopoly over local traffic in their region, but were forbidden from entering other telecom markets. AT&T itself retained the “long lines” division for long-distance traffic. Unlike local phone service, however, the settlement opened this market to free competition from any entrant willing and able to pay the interconnection fees to transfer calls in and out of the RBOCs. A residential customer in Indiana would always have Ameritech as their local telephone company, but could sign up for long-distance service with anyone.However, splitting apart the local and long-distance markets meant forgoing the subsidies that AT&T had long routed to rural telephone subscribers, under-charging them by over-charging wealthy long-distance users. A sudden spike in rural telephone prices across the nation was not politically tenable, so the deal preserved these transfers via a new organization, the non-profit National Exchange Carrier Association, which collected fees from the long-distance companies and distributed them to the RBOCS.   The new structure worked. Two major competitors entered the market in the 1980s, MCI and Sprint, and cut deeply into AT&T’s market share. Long-distance prices fell rapidly. Though it is arguable how much of this was due to competition per se, as opposed to the advent of ultra-high-bandwidth fiber optic networks, the arrangement was generally seen as a great success for de-regulation and a clear argument for the power of market forces to modernize formerly hidebound industries. This market structure, created ad hoc by court fiat but evidently highly successful, provided the template from which Congress drew in the mid-1990s to finally resolve the question of what telecom policy for the Internet era would look like. Second Time Isn’t The CharmPrior to the main event, there was one brief preliminary. The High Performance Computing Act of 1991 was important tactically, but not strategically. It advanced no new major policy initiatives. Its primary significance lay in providing additional funding and Congressional backing for what Wolff and the NSF already were doing and intended to keep doing – providing networking services for the research community, subsidizing academic institutions’ connections to NSFNET, and continuing to upgrade the backbone infrastructure.  Then came the accession of the 104th Congress in January 1995. Republicans took control of both the Senate and the House for the first time in forty years, and they came with an agenda to fight crime, cut taxes, shrink and reform government, and uphold moral righteousness. Gore and his allies had long touted universal access as a key component of the National Information Infrastructure, but with this shift in power the prospects for a strong universal service component to telecommunications reform diminished from minimal to none. Instead, the main legislative course would consist of regulatory changes to foster competition in telecommunications and Internet access, with a serving of bowdlerization on the side. The market conditions looked promising. Circa 1992, the major players in the telecommunications industry were numerous. In the traditional telephone industry there were the seven RBOCs, GTE, and three large long distance companies – AT&T, MCI, and Sprint – along with many smaller ones. The new up-and-comers included Internet service providers, such as UUNET, and PSINET as well as the IBM/MCI backbone spin-off, ANS; and other companies trying to build out their local fiber networks, such as Metropolitan Fiber Systems (MFS). BBN, the contractor behind ARPANET, had begun to build its own small Internet empire, snapping up some of the regional networks that orbited around NSFNET – Nearnet in New England, BARRNet in the Bay area, and SURANet in the southeast of the U.S. To preserve and expand this competitive landscape would be the primary goal of the 1996 Telecommunications Act, the only major rewrite of communications policy since the Communications Act of 1934. It intended to reshape telecommunications law for the digital age. The regulatory regime established by the original act siloed industries by their physical transmission medium – telephony, broadcast radio and television, cable TV; in each in its own box, with its own rules, and generally forbidden to meddle in each other’s business. As we have seen, sometimes regulators even created silos within silos, segregating the long-distance and local telephone markets. This made less and less sense as media of all types were reduced to fungible digital bits, which could be commingled on the same optical fiber, satellite transmission, or ethernet cable. The intent of the 1996 Act, shared by Democrats and Republicans alike, was to tear down these barriers, these “Berlin Walls of regulation”, as Gore’s own summary of the act put it.13 A complete itemization of the regulatory changes in this doorstopper of a bill is not possible here, but a few examples provide a taste of its character. Among other things it:allowed the RBOCs to compete in long-distance telephone markets,lifted restrictions forbidding the same entity from owning both broadcasting and cable services,axed the rules that prevented concentration of radio station ownership.The risk, though, of simply removing all regulation, opening the floodgates and letting any entity participate in any market, was to recreate AT&T on an even larger scale, a monopolistic megacorp that would dominate all forms of communication and stifle all competitors. Most worrisome of all was control over the so-called last mile – from the local switching office to the customer’s home or office. Building an inter-urban network connecting the major cities of the U.S. was expensive but not prohibitive, several companies had done so in recent decades, from Sprint to UUNET. To replicate all the copper or cable to every home in even one urban area, was another matter. Local competition in landline communications had scarcely existed since the early wildcat days of the telephone, when tangled skeins of iron wire criss-crossed urban streets. In the case of the Internet, the concern centered especially on high-speed, direct-to-the-premises data services, later known as broadband. For years, competition had flourished among dial-up Internet access providers, because all the end user required to reach the provider’s computer was access to a dial tone. But this would not be the case by default for newer services that did not use the dial telephone network. The legislative solution to this conundrum was to create the concept of the “CLEC” – competitive local exchange carrier. The RBOCs, now referred to as “ILECs” (incumbent local exchange carriers), would be allowed full, unrestricted access to the long-distance market only once the had unbundled their networks by allowing the CLECs, which would provide their own telecommunications services to homes and businesses, to interconnect with and lease the incumbents’ infrastructure. This would enable competitive ISPs and other new  service providers to continue to get access to the local loop even when dial-up service became obsolete – creating, in effect, a dial tone for broadband. The CLECs, in this model, filled the same role as the long-distance providers in the post-break-up telephone market. Able to freely interconnect at reasonable fees to the existing local phone networks, they would inject competition into a market previously dominated by the problem of natural monopoly. Besides the creation of the CLECS, the other major part of the bill that affected the Internet addressed the Republicans’ moral agenda rather than their economic one. Title V, known as the Communications Decency Act, forbade the transmission of indecent or offensive material – depicting or describing “sexual or excretory activities or organs”, on any part of the Internet accessible to minors. This, in effect, was an extension of the obscenity and indecent rules that governed broadcasting into the world of interactive computing services. How, then, did this sweeping act fare in achieving its goals? In most dimensions it proved a failure. Easiest to dispose with is the Communications Decency Act, which the Supreme Court struck down quickly (in 1997) as a violation of the First Amendment. Several parts of Title V did survive review however, including Section 230, the most important piece of the entire bill for the Internet’s future. It allows websites that host user-created content to exist without the fear of constant lawsuits, and protects the continued existence of everything from giants like Facebook and Twitter to tiny hobby bulletin boards. The fate of the efforts to promote competition within the local loop took longer to play out, but proved no more successful than the controls on obscenity. What about the CLECs, given access to the incumbent cable and telephone infrastructure so that they could compete on price and service offerings? The law required FCC rulemaking to hash out the details of exactly what kind of unbundling had to be offered. The incumbents pressed the courts hard to dispute any such ruling that would open up their lines to competition, repeatedly winning injunctions on the FCC, while threatening that introducing competitors would halt their imminent plans for bringing fiber to the home. Then, with the arrival of the Bush Administration and new chairman Michael Powell in 2001, the FCC became actively hostile to the original goals of the Telecommunications Act. Powell believed that the need for alternative broadband access would be satisfied by intermodal competition among cable, telephone, power communications networks, cellular and wireless networks. No more FCC rules in favor of CLECs would be forthcoming. For a brief time around the year 2000, it was possible to subscribe to third-party high-speed internet access using the infrastructure of your local telephone or cable provider. After that, the most central of the Telecom Act’s  pro-competitive measures became, in effect, a dead letter. The much ballyhooed fiber-to-the home only began to actually reach a significant number of homes after 2010, and the only with reluctance on the part of the incumbents.14 As author Fred Goldstein put it, the incumbents had “gained a fig leaf of competition without accepting serious market share losses.”15During most of the twentieth century, networked industries in the U.S. had sprouted in a burst of entrepreneurial energy and then been fitted into the matrix of a regulatory framework as they grew large and important enough to affect the public interest. Broadcasting and cable television had followed this pattern. So had trucking and the airlines. But with the CLECs all but dead by the early 2000s, the Communications Decency Act revoked, and other attempts to control the Internet such as the Clipper chip16 stymied, the Internet would follow an opposite course. Having come to life under the guiding hand of the state, it would now be allowed to develop in an almost entirely laissez-faire fashion. The NAP framework established by the NSF at the hand-off of the backbone would be the last major government intervention in the structure of the Internet. This was true at both the transport layer – the networks such as Verizon and AT&T that transported raw data, and the applications layer – software services from portals like Yahoo! to search engines like Google to online stores like Amazon.  In our last chapter, we will look at the consequences of this fact, briefly sketching the evolution of the Internet in the U.S. from the mid-1990s onward. [Previous] [Next]Quoted in Richard Wiggins, “Al Gore and the Creation of the Internet” 2000.“Remarks by Vice President Al Gore at National Press Club“, December 21, 1993.Biographical details on Wolff’s life prior to NSF are scarce – I have recorded all of them that I could find here. Notably I have not been able to find even his date and place of birth.Schrader and PSINet rode high on the Internet bubble in the late 1990s, acquiring other businesses aggressively, and, most extravagantly, purchasing the naming rights to the football stadium of the NFL’s newest expansion team, the Baltimore Ravens. Schrader tempted fate with a 1997 article entitled “Why the Internet Crash Will Never Happen.” Unfortunately for him, it did happen, bringing about his ouster from the company in 2001 and PSINet’s bankruptcy the following year.To get a sense of how fast the cost of bandwidth was declining – in the mid-1980s, leasing a T1 line from New York to L.A. would cost $60,000 per month. Twenty years later, a OC-3 circuit with 100 times the capacity cost only $5,000, more than a thousand-fold reduction in price per capacity. See Fred R. Goldstein, The Great Telecom Meltdown, 95-96. Goldstein states that the 1.55 mpbs T1/DS1 line has 1/84th the capacity of OC-3, rather than 1/100th, a discrepancy I can’t account for. But this has little effect on the overall math.Office of Inspector General, “Review of NSFNET,” March 23, 1993.Fraser, “NSFNET: A Partnership for High-Speed Networking, Final Report”, 27.Brian Kahin, “RFC 1192: Commercialization of the Internet Summary Report,” November 1990.John Markoff, “Data Network Raises Monopoly Fear,” New York Times, December 19, 1991.Though many other technical details had to be sorted out, see  Susan R. Harris and Elise Gerich, “Retiring the NSFNET Backbone Service: Chronicling the End of an Era,” ConneXions, April 1996.The most problematic part of privatization proved to have nothing to do with the hardware infrastructure of the network, but instead with handing over control over the domain name system (DNS). For most of its history, its management had depended on the judgment of a single man – Jon Postel. But businesses investing millions in a commercial internet would not stand for such an ad hoc system. So the government handed control of the domain name system to a contractor, Network Solutions. The NSF had no real mechanism for regulatory oversight of DNS (though they might have done better by splitting the control of different top-level domains (TLDs) among different contractors), and Congress failed to step in to create any kind of regulatory regime. Control changed once again in 1998 to the non-profit ICANN (Internet Corporation for Assigned Names and Numbers), but the management of DNS still remains a thorny problem.The only quasi-exception to this focus on fostering competition was a proposal by Senator Daniel Inouye to reserve 20% of Internet traffic for public use: Steve Behrens, “Inouye Bill Would Reserve Capacity on Infohighway,” Current, June 20, 1994. Unsurprisingly, it went nowhere. Al Gore, “A Short Summary of the Telecommunications Reform Act of 1996”.Jon Brodkin, “AT&T kills DSL, leaves tens of millions of homes without fiber Internet,” Ars Technica, October 5, 2020.Goldstein, The Great Telecom Meltdown, 145.The Clipper chip was a proposed hardware backdoor that would give the government the ability to bypass any U.S.-created encryption software.Further ReadingJanet Abatte, Inventing the Internet (1999)Karen D. Fraser “NSFNET: A Partnership for High-Speed Networking, Final Report” (1996)Shane Greenstein, How the Internet Became Commercial (2015)Yasha Levine, Surveillance Valley: The Secret Military History of the Internet (2018)Rajiv Shah and Jay P. Kesan, “The Privatization of the Internet’s Backbone Network,” Journal of Broadcasting & Electronic Media (2007)Share this: Click to share on X (Opens in new window) X Click to share on Facebook (Opens in new window) Facebook Like Loading...

Read more
The Computer as a Communication Device

Over the first half of the 1970s, the ecology of computer networking diversified from its original ARPANET ancestry along several dimensions. ARPANET users discovered a new application, electronic mail, which became the dominant activity on the network. Entrepreneurs spun-off their own ARPANET variants to serve commercial customers. And researchers from Hawaii to l’Hexagone developed new types of network to serve needs or rectify problems not addressed by ARPANET. Almost everyone involved in this process abandoned the ARPANET’s original stated goal of allowing computing hardware and software to be shared among a diverse range of research sites, each with its own specialized resources. Computer networks became primarily a means for people to connect to one another, or to remote systems that acted as sources or sinks for human-readable information, i.e. information databases and printers. This was a possibility foreseen by Licklider and Robert Taylor, though not what they had intended when they launched their first network experiments. Their 1968 article,”The Computer as a Communication Device” lacks the verve and timeless quality of visionary landmarks in the history of computing such as Vannevar Bush’s “As We May Think” or Turing’s “Computing Machinery and Intelligence.” Nonetheless, it provides a rather prescient glimpse of a social fabric woven together by computer systems. Licklider and Taylor described a not-to-distant future in which1 You will not send a letter or a telegram; you will simply identify the people whose files should be linked to yours and the parts to which they should be linked-and perhaps specify a coefficient of urgency. You will seldom make a telephone call; you will ask the network to link your consoles together. …Available within the network will be functions and services to which you subscribe on a regular basis and others that you call for when you need them. In the former group will be investment guidance, tax counseling, selective dissemination of information in your field of specialization, announcement of cultural, sport, and entertainment events that fit your interests, etc. The first and most important component of this computer-mediated future – electronic mail – spread like a virus across ARPANET in the 1970s, on its way to taking over the world. Email To understand how electronic mail developed on ARPANET, you need to first understand an important change that overtook the network’s computer systems in the early 1970s.  When ARPANET was first conceived in the mid-1960s, there was almost no commonality among the hardware and operating software running at each ARPA site. Many sites centered on custom, one-off research systems, such Multics at MIT, the TX-2 at Lincoln Labs, and the ILLIAC IV, under construction at the University of Illinois. By 1973, on the other hand, the landscape of computer systems connected to the network had acquired a great deal of uniformity, thanks to the wild success of Digital Equipment Corporation (DEC) in penetrating the academic computing market.2 DEC designed the PDP-10, released in 1968, to provide a rock-solid time-sharing experience for a small organization, with an array of tools and programming languages built-in to aid in customization. This was exactly what academic computing centers and research labs were looking for at the time. Look at all the PDPs! BBN, the company responsible for overseeing the ARPANET, then made the package even more attractive by creating the Tenex operating system, which added paged virtual memory to the PDP-10. This greatly simplified the management and use of the system, by making it less important to exactly match the set of running programs to the available memory space. BBN supplied the Tenex software free-of-charge to other ARPA sites, and it soon became the dominant operating system on the network. But what does all of this have to do with email? Electronic messaging was already familiar to users of time-sharing systems, most of which offered some kind of mailbox program by the late 1960s. They provided a form of digital inter-office mail; their reach extended only to other users of the same computer system. The first person to take advantage of the network to transfer mail from one machine to another was Ray Tomlinson, a BBN engineer and one of the authors of the Tenex software. He had already written a SNDMSG program for sending mail to other users on a single Tenex system, and a CPYNET program for sending files across the network. It required only a leap of imagination for him to see that he could combine the two to create a networked mail program. Previous mail programs had only required a user name to indicate the recipient, so Tomlinson came up with the idea of combining that local user name and the (local or remote) host name with an @ symbol3, to create an email address that was unique across the entire network. Ray Tomlinson in later years, with his signature “at” sign Tomlinson began testing his new program locally in 1971, and in 1972 his networked version of SNDMSG was bundled into the Tenex release, allowing Tenex mail to break the bonds of a single site and spread across the network. The plurality of machines running Tenex made Tomlinson’s hybrid program available instantly to a large proportion of ARPANET users, and it became an immediate success. It did not take long for ARPA’s leaders to integrate email into the core of their working life. Stephen Lukasik, director of ARPA, was an early adopter, as was Larry Roberts, still head of the agency’s computer science office. The habit inevitably spread to their subordinates, and soon email became a basic fact of life of the culture of ARPANET. Tomlinson’s mail software spawned a variety of imitations and elaborations from other users looking to improve on its rudimentary functionality. Most of the early innovation focused on the defects of the mail reading program. As email spread beyond a single computer, the volume of mail received by heavy users scaled with the size of the network, and the traditional approach of treating the mailbox as a raw text file was no longer effective. Larry Roberts himself, unable to deal effectively with the deluge of incoming messages, wrote his own software to manage his inbox called RD. By the mid-1970s, however, the most popular program by far was MSG, written by John Vittal of USC. We take for granted the ability to press a single button to fill out the title and recipient of outgoing message based on an incoming one. But it was Vittal’s MSG that first provided this killer “answer” feature in 1975; and it, too, was a Tenex program. The diversity of efforts led to a need for standards. This marked the first, but far from the last, time that the computer networking community would have to develop ex post facto standards. Unlike the basic protocols for ARPANET, a variety of email practices already existed in the wild prior to any standard setting. The inevitable result was controversy and political struggle, centering around the main email standard documents, RFC 680 and 720. In particular, non-Tenex users expressed a certain prickly resentment about the Tenex-centric assumptions built into the proposals. The conflict never grew terribly hot – everyone on ARPANET in the 1970s was still part of the same, relatively small, academic community and the differences to be reconciled were not large. But it provided a taste of larger struggles to come. The sudden success of email represented the most important development of the 1970s in the application layer of the network, the level most abstracted from the physical details of the network’s layout. At the same time, however, others had set out to redifine the foundational “link” layer, where bits flowed from machine to machine. ALOHA In 1968, Norman Abramson arrived at the University of Hawaii from California to serve a combined appointment as electrical engineering and computer science professor. The University he joined consisted of a main campus in Oahu as well as a secondary Hilo campus, and several other community colleges and research sites spread across Oahu, Kauai, Maui, and Hawaii. In between lay hundreds of miles of water and mountainous terrain. A brawny IBM 360/65 powered computer operations at the main campus, but ordering up an AT&T dedicated line to link a terminal to it from one of the community colleges was not so simple a matter as on the mainland. Abramson was an expert in radar systems and information theory who did a stint as an engineer for Hughes Aircraft in Los Angeles. This new environment, with all the physical challenges it presented to wireline communications, seems to have inspired Abramson to a new idea – what if radio were actually a better way of connecting computers than the phone system, which after all was designed with the needs of voice, not data, in mind? Abramson secured funding from Bob Taylor at ARPA to test this idea, with a system he called ALOHAnet. In its initial incarnation, it was not a computer network at all, but rather a medium for connecting remote terminals to a single time-sharing system, designed for the IBM machine at the Oahu campus. Like ARPANET, it had a dedicated minicomputer for processing packets sent and received by the 360/65 – Menehune, the Hawaiian equivalent of the IMP. ALOHAnet, however, dealt away with all the intermediate point-to-point routing used by ARPANET to get packets from one place to another. Instead any terminal wishing to send a message simply broadcast it into the ether in the allotted transmission frequency. ALOHAnet in its full state of development later in the 1970s, with multiple computers The traditional way for a radio engineer to handle a shared transmission band like this would have been to carve it up into time or frequency-based slots, and assign each terminal to its own slot. But to handle hundreds of terminals in such a scheme would mean limiting each to a small fraction of the available bandwidth, even though only a few might be in active use at any given moment. Instead, Abramson decided to do nothing to prevent more than one terminal from sending at the same time. If two or more messages overlapped they would become garbled, but the central computer would detect this via error-correcting codes, and would not acknowledge those packets. Failing to receive their acknowledgement, the sender(s) would try again after some random interval. Abramson calculated that this simple protocol could sustain up to a few hundred simultaneously active terminals, whose numerous collisions would still leave about 15% of the usable bandwidth. Beyond that, though, his calculations showed that the whole thing would collapse into a chaos of noise. The Office Of The Future Abramson’s “packet broadcasting” concept did not make a huge splash, at first. But it found new life a few years later, back on the mainland. The context was Xerox’s new Palo Alto Research Center (PARC), opened in 1970 just across from Stanford University, in a region recently dubbed “Silicon Valley.” Some of Xerox’s core xerography patents stood on the verge of expiration, and  the company risked being trapped by its own success, unable or unwilling to adapt to the rise of computing and integrated circuits. Jack Goldman, head of research for Xerox, had convinced the bigwigs back East that a new lab – distanced from the influence of HQ, nestled in an attractive climate, and with premium salaries on offer – would attract the talent needed to keep Xerox’s edge, by designing the information architecture of the future. PARC certainly succeeded in attracting top computer science talent, due not only to the environment and the generous pay, but also the presence of Robert Taylor, who had set the ARPANET into motion as head of ARPA’s Information Processing Technology Office in 1966. Robert Metcalfe, a prickly and ambitious young engineer and computer scientist from Brooklyn, was one of many wooed to PARC via an ARPA connection. He joined the lab in June 1972 after working part-time for ARPA a a Harvard graduate student, building the interface to connect MIT to the network. Even after joining PARC, he continued to work as an ARPANET ‘facilitator’, traveling around the country to help new sites get started on the network, and on the preparations for ARPA’s coming out party at the 1972 International Conference on Computer Communications. Among the projects percolating at PARC when Metcalfe arrived was a plan by Taylor to link dozens, or even hundreds, of small computers via a local network.  Year after year, computers continued to decrease in price and size, as if bending to the indomitable will of Gordon Moore. The forward-looking engineers at PARC foresaw a not-far-distant future when every office worker would have his own computer. To that end, they designed and built a personal computer called Alto, a copy of which would be supplied to every researcher in the lab. Taylor, who had only become more convinced of the value of networking over the previous half-decade, also wanted these computers to be interconnected. The Alto. The computer per se was housed in the cabinet at bottom, about the size of a mini-fridge. On arriving at PARC, Metcalfe took over the task of connecting up the lab’s PDP-10 clone to ARPANET, and quickly acquired a reputation as the “networking guy”. Therefore when Taylor asked for an Alto network, his peers turned to Metcalfe. Much like the computers on ARPANET, the Altos at PARC didn’t have much to say to one another. The compelling application for the network, once again, was in enabling human communication – in this case in the form of word and images printed by laser. The core idea behind the laser printer did not originate at PARC, but back East, at the original Xerox research lab in Webster, New York. There a physicist named Gary Starkweather proved that the coherent beam of a laser could be used to deactivate the electrical charge of a xerographic drum, just like the diffuse light used in photocopying up to that point. Properly modulated, the beam could paint a image of arbitrary detail onto the drum, and thus onto paper (since only the uncharged areas of the drum picked up toner). Controlled by a computer, such a machine could produce any combination of images and text that a person might conceive, rather than merely reproducing existing documents like the photocopier. Starkweather received no support for these wild ideas from his colleagues or management in Webster, however, so he got himself transferred to PARC in 1971, where he found a far more receptive audience. The laser printer’s ability to render arbitrary images dot-by-dot provided the perfect mate for the Alto workstation, with its bit-mapped monochrome graphics. With a laser printer, the half-million pixels on a user’s display could be directly rendered onto paper with perfect fidelity. The bit-mapped graphics experience on the Alto. Nothing like this had been seen on a computer display before. Within about a year Starkweather, with the help of several other PARC engineers, had overcome the main technical challenges and built a working prototype of a laser printer, based on the chassis of the workhorse Xerox 7000 printer. It produced pages at the same rate – one per second – at 500 dots per linear inch. A character generator attached the printer crafted text from pre-defined fonts. Free-form imagery (other than what could be generated with custom fonts) was not yet supported, so the network did not need to carry the full 25 million bits-per-second or so required to feed the laser; nonetheless, a tremendous of amount of bandwidth would be needed to keep the printer busy at a time when the 50,000 bits-per-second ARPANET represented the state-of-the-art. PARC’s second generation “Dover” laser printer, from 1976 The Alto Aloha Network How would Metcalfe bridge this huge gap in speed? Finally, we come back to ALOHAnet, for it turns out that Metcalfe knew packet broadcasting better than anyone. The previous summer, while staying in Washington with Steve Crocker on ARPA business, Metcalfe had pulled down volume of the proceedings of the Fall Joint Computer Conference, and came across Abramson’s ALOHAnet paper. He immediately realized that the basic idea was brilliant, but the implementation under-baked. With a few tweaks in the algorithm and assumptions – notably having senders listen for a clear channel before trying to broadcast, and exponentially increasing the re-transmission interval in response to congestion – he could achieve a bandwidth utilization of 90%, rather than the 15% calculated by Abramason. Metcalfe took a short leave from PARC to visit Hawaii, where he integrated his ideas about ALOHAnet into a revised version of his PhD thesis, after Harvard had rejected the original due to a lack of theoretical grounding. Metcalfe originally called his plan to bring packet broadcasting to PARC the “ALTO ALOHA network”. Then, in a memo in May 1973, he rechristened it as Ether Net, invoking the luminiferous ether which nineteenth-century physicists had suposed to carry all electromagnetic radiation. “This will keep things general,” he wrote, “and who knows what other media will prove better than cable for a broadcast network; maybe radio or telephone circuits, or power wiring or frequency-multi-plexed CATV, or microwave environments, or even combinations thereof.” A sketch from Metcalfe’s 1973 Ether Net memo. Starting in June 1973, Metcalfe worked with another PARC engineer, David Boggs, to turn his theoretical concepts for a new high-speed network into a working system. Rather than sending signals over the air like ALOHA, they would bind the radio spectrum within the confines of a coaxial cable, greatly increasing the available bandwidth from the limited radio band allocated to the Menehune. The transmission medium itself was entirely passive, requiring no switching equipment at all for routing messages. It was cheap and easy to connect it hundreds of workstations – PARC engineers just ran coax cable through the building and added taps as needed – and it could handle three million bits per second. Robert Metcalfe and David Boggs in the 1980s, several years after Metcalfe founded 3Com to sell Ethernet technology By the fall of 1974, the complete prototype of the office of the future was up and running in Palo Alto, California – the initial batch of thirty altos with drawing, email, and word processing software, Starkweather’s prototype printer, and Ethernet to connect it all together. A central file server for storing data too large for the Alto’s local disk provided the only other shared resource. PARC originally offered the Ethernet controller as an optional accessory on the Alto, but once the system went live it became clear that it was essential, as the coax coursed with a steady flow of messages, many of them emerging from the printer as technical reports, memos, or academic papers. Simultaneously with the development of the Alto, another PARC project attempted to carry the resource-sharing vision forward in a new direction. The PARC On Line Office System (POLOS), designed and implemented by Bill English and other refugees from Doug Engelbart’s oN-Line System (NLS) project at Stanford Research Institute, consisted of a network of Data General Nova minicomputers. Rather than dedicating each machine to a particular user’s needs, however, POLOS would shuttle work around among them, in order to serve the needs of the system as a whole as efficiently as possible. One machine might be rendering displays for several users, while another handled ARPANET traffic, and yet another ran word processing software. The complexity and coordination overhead of this approach proved unmanageable, and the scheme collapsed under its own weight. Meanwhile, nothing more clearly showed Taylor’s emphatic rejection of the resource-sharing approach to networking than his embrace of the Alto. Alan Kay, Butler Lampson, and the other minds behind the Alto had brought all the computational power a user might need onto an independent computer at their desk, intended to be shared with no one. The function of the network was not to provide access to a heterogeneous set of computer resources, but to carry messages among these islands, each entire of itself, or perhaps to deposit them on some distant shore – for printing or long-term storage. While both email and ALOHA developed under the umbrella of ARPA, the emergence of Ethernet was one of several signs in the first half of the 1970s that computer networking had become something too large and diverse for a single organization to dominate, a trend that we’ll continue to follow next time. [Previous] [Next] Further Reading Michael Hiltzik, Dealers of Lightning (1999) James Pelty, The History of Computer Communications, 1968-1988 (2007) [http://www.historyofcomputercommunications.info/] M. Mitchell Waldrop, The Dream Machine (2001)    

Read more
The Unraveling, Part 2

After authorizing private microwave networks in the Above 890 decision, the FCC might have hoped that they could leave those networks penned in their quiet little corner of the market and forget about them. But this quickly proved impossible. New challengers continued to press against the existent regulatory framework. They proposed a variety of new ways to use or sell telecommunications services, and claimed that the telecommuncations incumbents were obstructing their path. The FCC responded by steadily slicing away portions of AT&T’s monopoly, allowing competitors into various parts of the telecommunications market. AT&T responded with actions and rhetoric designed to counter, or at least mitigate the effects of, the new competition: publicly propounding their opposition to further FCC action, and setting new rates that sliced profits to the bone. From within the company, these seemed like natural responses to new competitive threats, but from the outside they only served as evidence that stronger measures would be needed to curb an insidious monopoly. When regulators pushed for telecom competition, they did not mean to encourage a struggle between for dominance between contending parties, and may the best company win. Their intent was to create and support lasting alternatives to AT&T. AT&T’s efforts to escape the net closing around it thus only served to ensnare it more deeply.1 The new threats came at both the edge and the the center of AT&T’s network, tearing away AT&T’s control over the terminal equipment attached to its lines by customers, and over the long-distance lines that interlinked the whole United States into a single telephone system. Each of these threats started with lawsuits by two small, seemingly insignificant upstarts: Carter Electronics and Microwave Communications, Incorporated (MCI), respectively. But the FCC not only favored the upstarts, but chose to interpret their cases expansively, as representing the needs of a whole new class of competitor which AT&T would have to accept and respect. Yet, in terms of the legal framework of regulation, nothing had changed since the Hush-a-Phone case of the 1950s. At that time the FCC had firmly rejected the claims of a far more innocuous challenger than Carter or MCI. The same 1934 Communications Act that had created the FCC in the first place still governed its actions in the 1960s and 70s. The shift in FCC policy did not come from new congressional action, but from a change in political philosophy within the commission. That change, in turn, was prompted to a large degree by the rise of the electronic computer. The emerging hybridization between computers and communication networks helped to set the conditions of its own further development. An Information Society For decades, the FCC had seen its main responsibility as maximizing access and fairness within a relatively stable and uniform telecommunications system. From the mid-1960s, however the FCC staff developed a different idea of their mission, and increasingly focused on maximizing innovation within a dynamic and diverse market. In large part this change can be attributed to the emergence of the new, though relatively tiny, market in data services. The data service industry originally had nothing to do with the telecommunications business at all. Its origins lay in service bureaus – companies that did data-processing on behalf of clients, then shipped them the results, a concept that in predated the modern computer by decades. IBM, for example, had offered on-demand processing for clients who couldn’t afford to lease their own mechanical tabulating equipment since the 1930s. In 1957, as part of an anti-trust deal with the U.S. Justice Department, they spun this business off into its a separate subsidiary, the Service Bureau Corporation, by that time running on modern electronic computers. Likewise, Automatic Data Processing (ADP), began as a manual payroll processing business in the late 1940s before computerizing in the late 1950s. In the 1960s, however, the first on-line data services began to appear, which allowed users to interact with a remote computer by terminal over a private, leased telephone line. Most famous of these was SABRE, a derivative of SAGE, designed to handle reservations for American Airlines using IBM computers. Just as with the first time-sharing systems, however, once you had multiple users talking to the same computer, it was a small step to letting those users talk to each other. It was this new way of using computers, as a mailbox, that brought computers to the attention of the FCC. In 1964, Bunker-Ramo2, a company best known as a defense contractor, decided to diversify into data services by acquiring Teleregister. Among Teleregister’s lines of business was a service called Telequote, which had provided stock information to brokers over telephone lines since 1928. Teleregister, however, was not itself a regulated common carrier. It relied on private lines leased from Western Union for communications between its users and its data center. Bunker-Ramo Telequote III terminal. It could display information about requested stocks, as well as market summary data. " data-medium-file="https://cdn.accountdigital.net/FgXKyn3jmYrb3JR4kL9Y5Z8Bu2y4?w=300" data-large-file="https://cdn.accountdigital.net/FgXKyn3jmYrb3JR4kL9Y5Z8Bu2y4?w=472" loading="lazy" class=" size-full wp-image-13324 aligncenter" src="https://cdn.accountdigital.net/FgXKyn3jmYrb3JR4kL9Y5Z8Bu2y4?w=739" alt="102716113.01.01.lg_" srcset="https://cdn.accountdigital.net/FgXKyn3jmYrb3JR4kL9Y5Z8Bu2y4 472w, https://cdn.accountdigital.net/FgXKyn3jmYrb3JR4kL9Y5Z8Bu2y4?w=150&h=135 150w, https://cdn.accountdigital.net/FgXKyn3jmYrb3JR4kL9Y5Z8Bu2y4?w=300&h=269 300w" sizes="(max-width: 472px) 100vw, 472px">Bunker-Ramo Telequote III terminal. It could display information about requested stocks, as well as market summary data. Telequote’s state-of-the-art system in the 1960s, Telequote III, allowed users to use a terminal with a tiny CRT to screen to punch up the price of a stock stored on Telequote’s remote computer. In 1965, Bunker-Ramo proposed the next iteration, Telequote IV, with the additional feature of allowing brokers in different offices to submit buy or sell orders to one another via their terminals. Western Union, however, refused to allow their lines to be used for this purpose. They claimed that using the computer to transmit messages between users would turn a purported private line into a de facto common carrier message-switching service (not unlike Western Union’s own telegraph service), and requiring the operator (Bunker-Ramo) to be regulated by the FCC. The FCC decided to turn this dispute into an opportunity to answer a broader question – how should the growing contingent of on-line data services be treated, vis-a-vis telecommunications regulation? The resulting investigation is now known simply as the Computer inquiry. The ultimate conclusions of that inquiry are less important for us at this point than their effects on the mentality of the FCC staff. Long-established boundaries and definitions seemed liable to be redrawn or abandoned, and this shake-up conditioned the FCC’s mindset for the challenges to come. Every so often, over the previous decades, a new communications technology had emerged. Each developed independently and acquired its own distinct character and its own regulatory rules: telegraphy, telephony, radio, television. But with the emergence of the computers these distinct lines of development began to converge on the imagined horizon into an intertwined information society. Not just the FCC, but the intelligentsia in general anticipated major changes afoot. Sociologist Daniel Bell wrote of the coming “post-industrial society”, management expert Peter Drucker spoke of “knowledge workers” and the “age of discontinuity.” Books, papers, and conferences abounded in the second half of the 1960s on the topic of a coming world based in information or knowledge rather than material production. The authors of these works referred often to emergence of high-speed, general-purpose computers, and the new ways that they would allow data to be transmitted and processed within communications networks in the coming decades. Some of the newer FCC commissioners, appointed by Presidents Kennedy and Johnson, were themselves active in these intellectual circles. Kenneth Cox and Nicholas Johnson both participated in a Brookings Institute symposium on “Computers, Communications, and the Public Interest,” whose chair imagined “a national or regional communication network that connects video and computer facilities at universities to homes and classrooms in local communities …The citizenry could be students ‘from cradle to coffin…” Johnson later wrote a book on the prospect of using computers to transform broadcast TV into an interactive medium, entitled How to Talk Back to Your Television Set. Beyond these general intellectual currents that were pushing communications regulation in new directions, one man in particular had a particular interest in steering regulation onto a new course, and played a major role in shifting the FCC’s attitudes. Bernard Strassburg belonged to the layer of the FCC bureaucracy just below the seven politically-appointed commissioners. The career civil servants who populated most of the FCC were divided into bureaus based on the technological area that they regulated. The commissioners relied on the legal and technical expertise of the bureaus to guide them in the rulings process. The domain of the Common Carrier Bureau, to which Strassburg belonged, lay in the wireline telephone and telegraph industry, consisting primarily of AT&T and Western Union. Strassburg joined the Common Carrier Bureau during World War II, rose to become its Chairman by 1963, and played a major role in pushing the FCC to chip away at AT&T’s dominance over the following decade. His distrust of AT&T originated with the anti-trust suit that the Justice Department launched against the company in 1949. At issue, as we’ve mentioned before, was the question of whether Western Electric, AT&T’s manufacturing arm, inflated its prices in order to allow AT&T, in turn, to artificially inflate its profits. Strassburg became convinced during the investigation that it was simply impossible to answer the question, given AT&T’s near-total monopsony in telephone equipment. There was no telephone equipment market to compare against to determine what constituted reasonable prices. AT&T was simply too large and powerful to effectively regulate, he concluded3. Much of his advice to the Commission in the coming years could be traced to this belief that competition needed to be forced into AT&T’s world, to weaken it sufficiently to make it regulable. Challenge at the Center: MCI The first serious challenge to AT&T’s long distance network, since its inception at the turn of the twentieth century, began with an unlikely man. John Goeken was a salesman and small businessman with at least as much enthusiasm as good sense. Like many boys of his time, he had developed an interest in radio equipment as a youth. He joined the Army out of high school as a microwave radio technicia, and, after completing his active service, he went into radio sales for General Electric (GE) in Illinois. His day job didn’t fill his need for entrepreneurship, however, so he also developed a side business with a group of friends, selling more GE radios in other parts of Illinois outside of his assigned territory. Goeken in the mid-90s, when he was working on an in-flight telephone When GE got wind of the operation and shut it down in 1963, Goeken began to look for other ways to supplement his income. He decided to build a microwave line from Chicago to St. Louis, selling radio access to the line to truckers, bargemen, flower delivery vans, and other small businesses along the route who had a need for inexpensive, mobile communications. He believed that AT&T’s private-line service was “gold-plated” – over-staffed and over-engineered – and that by being leaner and more cost-conscious he could provide lower prices and better service to the smaller users neglected by Ma Bell. Goeken’s concept did not conform to then-current FCC rules – the Above 890 ruling had authorized private companies to build microwave systems for their own use. Under pressure from smaller businesses without the wherewithal to build and maintain a whole system, a 1966 ruling had allowed multiple entities to share a single private microwave systems. But this still did not authorize them to become common carriers themselves, retailing service to third parties. Moreover, the reason that AT&T’s prices appeared excessive was not due to gilded wastefulness, but regulated cost-averaged rates. AT&T charged for private line service according to the distance and number of lines leased, whether those lines lay along the high-density Chicago-St. Louis route or a low-density route with little traffic across the Great Plains. Regulators and telephone companies had intentionally devised this structure to level the playing field between areas with differing population densities. MCI was thus proposing to engage in a form of arbitrage – taking advantage of the differential between the market and the regulated price on a high-traffic route to extract guaranteed profits. AT&T called this cream-skimming, a term that served as their primary rhetorical touchstone in the debates to come. It’s not clear whether Goeken did not initially know these facts, or chose blithely to ignore them. In any case, he went after his new idea with gusto, on a shoestring budget funded mainly by credit cards. He and his partners, all of similarly modest means, nonetheless dared to form a company to take on the over-mighty AT&T, which they called Microwave Communications, Inc . Goeken flew around the country looking for investors with deeper pockets, with little success. He had better luck, however, arguing MCI’s case before the FCC. The first hearings on the case began in 1967. Strassburg was intrigued. He saw in MCI an opportunity to achieve his goal of weakening AT&T, by further prying open the market for private lines. But he wavered at first about whether to follow through. Goeken did not impress him as a serious and effective businessman. MCI, he worried, might not be the best test case. He was nudged off the fence by an economist from the University of New Hampshire named Manley Irwin. Irwin had a steady consulting gig with the Common Carrier Bureau, and had helped to formulate the terms of the Computer inquiry. He convinced Strassburg that the nascent on-line data service market revealed by that inquiry needed companies like MCI that would provide new offerings; that AT&T alone would never be able to fulfill all the potential of the coming information society. Strassburg later reflected that “the ‘fallout’ from the Computer Inquiry… substantiated MCI’s claim that its entry into the specialized intercity market would be in the public interest.”4 With the blessings of the Common Carrier Bureau in-hand, MCI breezed through the initial hearing, then squeaked by with approval before the full commission in 1968, which split 4-3 along party lines. All the Democrats (Cox and Johnson included) voted in favor of approving MCI’s license. The Republicans, led by the chair, Rosel Hyde, dissented. The Republicans did not want to disrupt a well-balanced regulatory system with a scheme concocted by fly-by-night operators of questionable technical and business savvy. They pointed out that the decision, though limited on its face to a single company and a single route, carried profound implications that would transform the telecommunications market. Strassburg and the pro-approval commissioners treated the MCI case as an experiment, to see if a business could successfully operate alongside AT&T in the private line services market. But in fact it was a precedent, and, once approved, dozens of other companies would immediately come out of the woodwork to file their own applications. Reversing the experiment, the Republicans saw, would effectively be impossible. Morever, MCI and similar specialized entrants could scarcely survive with just a scattering of disconnected routes like the one from Chicago to St. Louis. They would demand interconnection with AT&T, and force the FCC to continue making changes to the regulatory structure. The land rush predicted by Hyde and the other Republicans did indeed ensue, with thirty-one companies filing 1713 separate applications for a total of 40,000 miles of microwave network within two years of the MCI decision.5 The FCC lacked the capacity to carry out individual hearings on all of these applications, and so it gathered them all together as a single docket on Specialized Common Carrier Services. In May 1971, with Hyde out, they unanimously decided to open the market fully to competition. Meanwhile, MCI, still starved for money, found a new wealthy investor to set its finances in order, William C. McGowan. McGowan was virtually Goeken’s opposite, a sophisticated and established businessman with a degree from Harvard, who had built successful consulting and venture capital businesses in New York City.  Within a few years, McGowan took effective control of MCI and pushed Goeken out. He had a very different vision for the company from that of his predecessor. He had no intention of messing around with bargemen and florists, nibbling around the periphery of the telecommunications market wherever AT&T deigned not to notice him. Instead he would go right for the heart of the regulated network, competing directly in all forms of long-distance communications. Bill McGowan in later years The stakes and implications of the original experiment with MCI thus continued to ratchet upward. Having committed itself to seeing MCI succeed, the FCC now found itself taken for a ride, as McGowan’s demands continued to broaden. Arguing (again, as predicted), that MCI could not survive as a small collection of disconnected routes, he demanded a wide variety of interconnection rights into the AT&T network; for example the right to connect to what was called a “foreign exchange,” allowing MCI’s network to connect directly into AT&T’s local telephone exchanges at the terminus of MCI private lines. AT&T’s responses to the new specialized common carriers did not help its cause. It answered the intrusion of competitors by introducing much lower rates on private lines along high-traffic routes, abandoning regulated, rate-averaged prices. If it thought this would appease the FCC by showing competitive spirit, it misconstrued the FCC’s purpose. Strassburg and his allies were not trying to help consumers by reducing communications prices, at least not directly. Instead they were trying to help new producers enter the market, thereby weakening AT&T’s power. Thus AT&T’s new competitive rates were seen by the FCC and other observers, especially at the Justice Department, as vindictive and anti-competitive, because they threatened the financial stability of new entrants like MCI. AT&T’s combative new president, John deButts, also did himself no favors with his aggressive rhetorical responses to competitive incursions. In a 1973 speech before the National Association of Regulatory Utility Commissioners, he belittled the FCC with his call for “a moratorium on further experiments in economics.” This kind of intransigence infuriated Strassburg, and further convinced him of the necessity of taking AT&T down a peg. The FCC duly ordered the interconnections requested by MCI in 1974. McGowan’s escalation climaxed with Execunet, launched the following year. Advertised as as a new kind of metered service for sharing private lines among small businesses, it gradually became apparent to both the FCC and AT&T that Execunet was in fact a competing long-distance phone network. It allowed a customer in one city to pick up a phone, dial a number, and reach arbitrary customers in another city (taking advantage of MCI’s foreign exchange connections) for a charge based on the distance and duration of the call. No dedicated point-to-point line came into the picture at all. Execunet connected MCI customers directly to any AT&T customer in any major city. At this point the FCC finally balked. It had intended to use MCI as a cudgel to beat back the complete dominance of AT&T, but this was a blow too far.  By this time, however, AT&T had other allies in the courts and the Justice Department, and continued to advance its case. The unraveling of the AT&T monopoly, once begun, was not easily stopped. Challenge at the Periphery: Carterfone While the MCI case was playing out, another threat approached. The similarities between the Carterfone and MCI stories are striking. In both cases, an upstart entrepreneur – possessed of more gumption and grit than good business sense – brought a successful challenge against the largest corporation in the United States. Both men, however – Jack Goeken and our new protagonist, Tom Carter – were shortly thereafter eased out of their companies by sharper operators and then faded into obscurity. Both men began as protagonists, but ended as pawns. Tom Carter was born in 1924 in Mabank, Texas, south east of Dallas. Another young radio enthusiast, he joined the Army at 19, becoming, like Goeken, a radio technician. He spent the latter years of World War II manning a broadcasting station in Juneau, providing news and entertainment to troops at far-flung outposts across Alaska. After the war he returned to Texas and formed Carter Electronics Corporation in Dallas, operating a two-way radio station that he leased out to other businesses – florists with delivery vans; oil companies with operators out at drilling rigs. Over and over, Carter heard requests from clients for a way to patch their mobile radios directly into the phone network, rather than having to relay messages to people in town through the base station operator. Carter devised an instrument to satisfy this need, which he called Carterfone. It consisted of a black plastic lozenge with a molded top designed to cradle a telephone handset, containing a microphone and a speaker, both wired to the radio transmitting/receiving station. To connect someone in the field with someone on the telephone, the base station operator still had to place a call manually, but then they could then rest the handset in the cradle, and the two parties could converse uninterrupted. A voice-activated switch tripped the radio’s send/receive mode, sending when the person on the telephone was speaking and receiving otherwise. He began selling the device in 1959, with a manufacturing operation that consisted of a small brick building in Dallas where senior citizens assembled Carterfones on plain wooden tables. A 1959 Caterfone. The phone handset would rest in the cradle and activate the device via the small switch at top. Carter’s invention was not entirely novel. Bell had its own mobile radio telephone service, which it first offered in St. Louis, Missouri in 1946. Twenty years later it served 30,000 customers. But there was plenty of room for a competitor like Carter – AT&T only offered the service in about a third of the United States, and the waiting list could be years long. Moreover, Carter offered a significantly cheaper price, if (large caveat) one already had a access to a radio tower: a one-time $248 purchase for the equipment, versus a $50-60 lease for a Bell mobile phone. Carterfone was, from AT&T’s point-of-view, a “foreign attachment”, a piece of third-party equipment attached to its network, a practice that it forbade. In the earlier Hush-a-Phone case, the courts had forced AT&T allow simple mechanical attachments to a telephone, but Carterfone did not fall in that category, being acoustically-coupled to the network – that is, it transmitted and received sound over the telephone line. Due to the small scale of Carter’s operations, it was two years before AT&T took notice, and started to warn retailers carrying Carterfone that their customers risked having their telephone service shut off – the same angle used to attack Hush-a-Phone over a decade earlier. With these kinds of tactics, AT&T chased Carter out of one market after another. Unable to reach any kind of deal with his antagonists, Carter decided to sue in 1965. None of the big Dallas firms would take the case, so Carter ended up at  the small office of Walter Steele, with only three lawyers to its name. One of them, Ray Besing, later painted this character portrait of the man who arrived in his office: He fancied himself a handsome man, with his side-combed white hair, which was all the whiter thanks to Grecian Formula, but his double-knit suit and cowboy boots presented a different kind of image. He was a self taught man, handy with any kind of electronic, radio, or telephone equipment. Not much of a business man. A strict family man with an equally strict wife. Yet he sought to appear a cool, successful businessman even though he was basically broke. The case came before the FCC’s preliminary examiner in 1967. AT&T and its allies (primarily the other, smaller telephone companies and the state telephone regulatory agencies) argued that Carterfone was not a simple attachment at all, but a piece of interconnection equipment, that unlawfully coupled AT&T’s network into local mobile radio networks. This violated the telephone company’s end-to-end responsibility for communications within its system. But as with MCI, the Common Carrier Bureau issued a statement decisively in favor of Carter. Once again the belief in a coming world of digital information services, simultaneously integrated and diversified, loomed in the background. How could a single monopoly supplier foresee and satisfy all the market needs for terminals and other equipment for all these coming applications? The final decision of the commission, on June 26, 1968, concurred with the CCB and found that AT&T’s foreign attachments rule was not only unlawful, but had been unlawful from its inception – therefore Carter stood eligible for back damages. AT&T, the FCC ruled, had failed to properly distinguish potentially harmful attachments (ones that might send errant control signals into the network, for example) from essentially harmless ones such as Carterfone. AT&T would have to allow Carterfones immediately, and devise technical standards for the safe interconnection of third-party devices. Shortly after the decision, Carter tried to exploit his success by going into business with two partners, including one of his lawyers, forming Carterfone Corporation. After pushing Carter out of the company, his partners made millions by selling to the British telecom giant Cable and Wireless. The Carterfone itself disappeared; the company continued on selling teletypewriters and computer terminals. Carter’s story has a curious epilogue. In 1974, he actually went into business with Jack Goeken, founding the Florist Transworld Delivery system to send flowers on demand. It was just the kind of market – using telecommunications to support small businesses – that both men had wanted to serve in the first place. Carter soon quit that company, too, however, and moved back to his roots southeast of Dallas, where, in the mid-80s, he operated a small radio telephone company called Carter Mobilefone. He remained there until his death in 1991.6 Unraveled Like Carter and Goeken, the FCC had set into motion forces it could not control or even fully understand. By the mid 1970s, Congress, the Justice Department and the courts took the debate over AT&T’s future out of the FCC’s hands. The climax of AT&T’s great unraveling, of course, came with final break-up of AT&T, carried out in 1984. But we have already gotten well ahead of the rest of our story. The world of computer networking did not feel the full implications of MCI’s victory, and the intrusion of competition into the long-distance market, until the 1990s, when private data networks began to proliferate. The decisions on terminal equipment had a more immediate effect. Acoustically coupled computer modems could now be manufactured by anyone and connected to the Bell system, under the sheltering hand of the Carterfone ruling, making them less expensive and easier to find. But the most important implication of AT&T’s unraveling lay in the big picture, rather than the particulars of individual rulings. Many of the early visionaries of the information age imagined a single, unified American computer-communications network, under the aegis of AT&T, or perhaps even the federal government itself. Instead computer networks developed piecemeal, in fragments, which were only gradually connected, or, “inter-networked.” No single overarching corporation controlled the various sub-networks as had been the case with Bell and its local operating companies; they came to one another  not as master and subordinate, but as peers. But that, too, is getting ahead of ourselves. To continue our story we must turn back to the mid-1960s, to see where computer networks came from in the first. [Previous part] Further Reading Ray G. Bessing, Who Broke Up AT&T? (2000) Philip L. Cantelon, The History of MCI: The Early Years (1993) Peter Temin with Louis Galambos, The Fall of the Bell System: A Study in Prices and Politics (1987) Richard H. K. Vietor, Contrived Competition: Regulation and Deregulation in America (1994)

Read more