Preserving digital records

The Archives' digital archive server room.
The Archives' digital archive server room.

The Archives ensures that significant Commonwealth records – including digital records – are preserved and remain accessible in the future.

Read the Digital Preservation Policy.

Digital records present many preservation challenges. They are at risk of being lost due to the rapid pace of development in computer hardware, operating systems and application software, coupled with the short effective life of most physical storage media.

The Archives has developed a digital preservation methodology, supported by software, to ensure that important digital records are preserved as national archives.

Digital preservation: a four-step process

To preserve digital records, the Archives uses its digital preservation software to convert proprietary file formats to openly specified preservation file formats.

There are four steps in this process:

  1. Manifest – a list is created of all digital records to be preserved. 
  2. Quarantine – records are checked for viruses and integrity. 
  3. Preservation – records are converted to preservation file formats. 
  4. Storage – records are stored in the digital archive. 

The Archives' digital preservation software:

  • captures the essential elements of digital records
  • allows digital records to be retrieved from the digital archive at any time
  • continually checks the integrity of records in the digital archive.

Dissecting the Digital Preservation Software Platform (pdf, 600kB or doc, 2.05MB) details the methods employed by the Archives to ensure the effective preservation of digital records. It covers our approach to preserving digital records, how we develop our software, what the software does to preserve digital records and how we maintain the integrity of the records in our digital archive.

Preservation file formats

The basis of the National Archives' digital preservation strategy is the conversion of non-open file formats to open file formats.

The Archives uses preservation file formats based on open standards, which:

  • have full specifications that are publicly documented
  • are interoperable with a range of software applications
  • are not affected by changes in commercial property rights over software in the marketplace.

Digital records based on open standards thus have a greater potential lifespan.

The digital preservation file formats the Archives has selected are listed in this table (PDF, 464kb).

National Archives of Australia software

When researching and developing our approach to digital preservation, we made some important decisions about the software we required. As a result, the software:
  • is licensed under an open source license (the GNU General Public License)
  • converts proprietary file formats to preservation file formats based on open standards.

Open source development allows the Archives to build on the efforts of other open source projects, so we can achieve our goals more quickly and with fewer resources. Australian Government agencies and other stakeholders benefit too as the software developed by the Archives is freely available to download and use.

Our suite of software is referred to as the Digital Preservation Software Platform. This is made up of the following applications:

  • Xena
  • Manifest Maker
  • Digital Preservation Recorder
  • Checksum Checker.

Xena

Xena (Xml Electronic Normalising for Archives) converts proprietary file formats to preservation file formats. It can also be used to view and export digital records.

Xena preserves digital records in a three-step process:

  1. Xena determines the file format of the digital record.
  2. Based on the file format, Xena either converts the digital record to a preservation file format or, if the record is already in a preservation file format, preserves it as is.
  3. Xena then stores the digital record, with its preservation metadata, as a Xena file.

Xena handles a range of formats, including office documents, email, images and audio files.

Manifest Maker

Manifest Maker supports the transfer of digital records from agencies to the Archives by producing a manifest file. The manifest lists all the digital records that are being sent to the Archives.

The manifest is used to verify the integrity of digital records and track them when they are being processed by Digital Preservation Recorder.

Digital Preservation Recorder

Digital Preservation Recorder manages the processing of records into the digital archive. Its main features are:

  • step-by-step guided workflow from quarantine to preservation to storage in the digital archive
  • conversion of digital records to preservation file formats using Xena
  • storage of each original record and its preservation version in the digital archive
  • access to records stored in the digital archive
  • audit information about each transfer
  • reports on users, transfers and records stored in the digital archive.

Checksum Checker

The Checksum Checker software monitors the contents of the Archives' digital archive for data loss or corruption. It continually calculates the checksums of records in the digital archive and compares them to values stored in the Digital Preservation Recorder database. If the checksums do not match, staff are alerted to fix the issue.

Future directions

The Archives' development priorities are driven by changes in technology, standards and our own business processes. We are continually improving our software so it can handle the exponential growth in  file sizes, and convert an increasing variety of file formats to preservation formats, and the volume of digital records.

Copyright National Archives of Australia 2017