VISUALIZATION BY DONNA COX & ROBERT PATTERSON, NCSA, UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN.
A visualization study of inbound traffic measured in billions of bytes on the NSFNET T1 backbone for the month of September 1991. The traffic volume range is depicted from purple (zero bytes) to white (100 billion bytes). It represents data collected by Merit Network, Inc.
By the early 1980s, the bones of what would become the internet were in place. This new and growing network, with continued support by DARPA and the DOD, became available to the academic community, which led to the launch of other government networks. In 1983, the first part of what would become the National Science Foundation’s (NSF) computing backbone, NSFNET, was laid down using an optical fiber network. NSFNET grew rapidly, connecting thousands and then millions of university computers around the United States by the early 1990s. The success of the network convinced the Department of Energy (DOE) and NASA to establish their own networks using the same protocols, which led to the birth of the DOE’s Energy Sciences Network and the NASA Science Internet. These new networks joined NSF’s to become the four major networks of the U.S. internet. Commercial internet service did not exist until 1989. Cerf noted that commercially available software and hardware products available from firms like Proteon, and later Juniper and IBM, were available for dial-up use. “You could buy those [products]. You couldn’t buy [internet] service, because it was only available to people who were sponsored by the U.S. government research agencies,” Cerf said. In 1988, Cerf, who was then vice president of the Corporation for National Research Initiatives, received permission from the Federal Networking Council to connect the MCI Mail service, which had been providing commercial email services over phone lines via modem since
1983, to the internet. As president of MCI’s Digital Information Services from 1982 to 1986, Cerf led the development of MCI Mail. This served to break the restriction prohibiting commercial traffic from traveling over the government backbone, he explained. This new commercial, rather than government-based, email service launched in 1989. It opened the floodgates for commercial players, and the major email providers of the time, such as AOL, CompuServe, and Telenet’s Telemail, which were providing messaging via telephone modems, soon followed suit. This rapid expansion of commercial internet services prompted the NSF to shut down its dedicated backbone in 1995.
Powering the Growing Web The 1990s saw the development of web search technology, driven by users’ need for more efficient ways to find information as the World Wide Web expanded exponentially. Work in web search technology derived largely from earlier work in information retrieval from text, such as DARPA’s 1991 TIPSTER program. These earlier programs helped set the stage for a line of DARPA investments aimed at improving information extraction and search that still continues today. By the late ’90s, the need for better machine processing of the information on the growing web, as well as for more precise search capabilities, was becoming critical to the ability of users to find what they were looking for. In the national security context, the challenges to accessing, managing, and sharing data residing on the classified DOD networks were becoming more important to solve. “A lot of information management technologies had been developed by DARPA,” Hendler said, but “the question was, how could we apply them to this exponentially growing space?”