When the National Archives received data in the mid-seventies from the Census Bureau, it was in a 1960’s then state-of-the-art UNIVAC format. At the time “there were only two UNIVAC computers left in the world: one in Japan and the other housed in the Smithsonian Institute as a museum piece. Heroic and costly rescue efforts recovered much, but not all, of the data.” [1]
Detailed info on this case:
As it compiled the decennial census in the early sixties, the Census Bureau retained records for its own use in what it regarded as “permanent” storage. In 1976, the National Archives identified seven series of aggregated data from the 1960 Census files as having long-term historical value. A large portion of the selected records, however, resided on tapes that the Bureau could read only with a UNIVAC type II-A tape drive. By the mid-seventies, that particular tape drive was long obsolete, and the Census Bureau faced a significant engineering challenge in preserving the data from the UNIVAC type II-A tapes. By 1979, the Bureau had successfully copied onto industry-standard tapes nearly all the data judged then to have long-term value. Though largely successful in the end, the data rescue effort was a signal event that helped move the Committee on the Records of Government six years later to proclaim that “the United States is in danger of losing its memory.” The Committee did not bother to describe the actual details of the migration of the 1960 census records. Nor did it analyze the effects on the integrity of the constitutionally-mandated census of the nearly 10,000 (of approximately 1.5 million) records of aggregated data that the rescue effort did not successfully recover. Instead, it chose to register its warning on the dangers of machine obsolescence in apocryphal terms. With more than a little hyperbole, it wrote that “when the computer tapes containing the raw data from the 1960 federal census came to the attention of NARS [the National Archives and Records Service], there were only two machines in the world capable of reading those tapes: one in Japan and the other already deposited in the Smithsonian as a relic” (1985:9, 86-87) [2]
Thanks to Marielle Guercio (University of Urbino) who pointed me to this interesting document [3] a more extensive description is available.
Archive US 1960 Census
Purpose Summary tape files (“micro-aggregation files”) about the 1960 Census
Institution in charge Census Bureau / National Archives and Records AdministrationContents
The US 1960 Census returns were microfilmed in 1961 for long-term storage. In addition to the microfilm, the Census Bureau also creates many reports from the information obtained in each census. These reports are mostly demographic in nature. They describe the ethnic make-up of the U.S. population; they document American migration patterns and eve n tell how many bathrooms are in the average American home.
In 1961, the staff at the Bureau of the Census had access to computers for the first time. In order to simplify some of the data analysis that the Census Bureau must conduct, they used the new computers to create the “micro-aggregation files” that contain statistical information. This information had been entered on punch cards in earlier censuses, but magnetic tape was the storage medium of choice in the ’60s. The Bureau of the Census had the required data stored on 9,121 reels of magnetic tape: 7,297 reels “readable” with UNIVAC II-A tape drives; 1,678 tapes “readable” with UNIVAC III-A tape drives, and another 146 magnetic tapes created on still other brands of tape drives and “readable” with the n contemporary industry-compatible tape drives. The reports needed were generated and printed on paper. Once the reports were completed, the tapes were placed in storage.Relevance of the archive
The U.S. Constitution mandates an official counting of the population every 10 years. While the official purpose of the Constitution’s mandate of a population count is to reapportion congressional districts, the Census also provides a statistical history of the nation and its people and is an economic asset and tool of inestimable value. The Census shows not only where people live at the time of the count, but also their educational levels, income, and other vital data. In 1890, data were recorded with the aid of Hollerith’s punch cards. The 1950 Census of Population used a UNIVAC computer for tabulation of data. For the 1960 Census, the first to use the mail for collection of data, the Census Bureau and the National Bureau of Standards developed FOSDIC (film optical sensing device for input to computers), which was used until the 2000 Census. The surveys, completed by filling in dots opposite the appropriate answers, were photographed onto
microfilm. FOSDIC read the dots and transferred the data to tape for computer input.What happened to it?
An internal 1963 Bureau of the Census technical memorandum listed tape files produced in connection with the 1960 Census of Population and Housing that the Bureau was retaining in “permanent data storage”.
Following consultation with staff of the National Archives in 1975, the Census Bureau created a plan to provide for the adequate retention of the 1960 data. They would retain 132 of the industry compatible tapes and would copy the tape files on 1,273 of the III-A tapes onto industry compatible tapes.
During 1975 and 1976, a member of the National Archives’ Machine Readable Archives Division reviewed the micro-aggregation or derivative files that the Bureau of the Census had preserved from the 1960 Census on the II-A tapes. This review identified seven series of low-level microaggregations as having long-term value. The seven series resided on 642 of the II-A tapes which the Census Bureau agreed to migrate onto more modern storage media, at least modern by 1975 industry standards. But by this time, the UNIVAC II-A tape drives were obsolete, and thus the preservation of these tapes presented a major engineering challenge.Characteristics of the recovery
Despite the challenge, the Census staff managed to find some old tape drives still in use that could read the tapes. These old drives were installed on a computer system that also had newer drives installed, so a tape conversion seemed simple. By 1979, the Census Bureau had successfully copied 640 of the 642 II-A tapes onto newer format tapes. The two tapes that were not copied were, in fact, missing. The missing tapes had 7,488 records, or about 0.5 percent of the total of approximately 1.5 million records on all II-A tapes that had been identified as having long-term value. Of the 640 tapes that were located, only 1,575 records (or less than 0.2 percent of the total number of valuable
records on II-A tapes) could not be copied because of deterioration. Hence a small volume of records from the 1960 Census was lost, and this occurred because of inadequate inventory control and because of the physical deterioration of a minuscule number of
records. The bottom line is that 99.3 percent of the 1960 micro-aggregation data was saved on modern tape formats and can be read today. The findings of the original study had already been published on paper in the 1960s, and that paper is preserved. The original 1960 U.S. Census documents were also recorded on microfilm, and all
that microfilm is still in good condition, locked up at the National Archives. In compliance with U.S. laws, the complete 1960 U.S. Census documents on microfilm will be released to the public in the year 2032, 72 years after the original enumeration.Cost of the recovery
It has been impossible to find information about the overall cost of the recovery.Conclusions
The successful recovery of 99.3% of the micro-aggregation data marks a positive outcome which was made possible due to the availability of hardware similar to that used when the archive was originally created. The data which it was not possible to recovery was stored on tapes which were lost or had deteriorated.
[1] Warner, Dorothy and Buschman, John. Studying the Reader/Researcher Without the Artifact: Digital Problems in the Future History of Books. Library Philosophy and Practice Vol. 7, No. 1 (Fall 2004) http://www.webpages.uidaho.edu/~mbolin/warner-buschman.htm
[2] Garrett, J & Waters, D. Preserving Digital Information. Report of the Task Force on Archiving of Digital Information commissioned by The Commission on Preservation and Access and The Research Libraries Group. 1996 http://www.clir.org/pubs/reports/pub63watersgarrett.pdf p.2
[3] Preservation of digital memories: risks and emergencies
Six case studies. Edited by Alessandra Ruggiero. http://www.iccu.sbn.it/opencms/export/sites/iccu/documenti/emergenze.pdf.