Outsourcing digital storage

Storing Commonwealth records in data centres, digital repositories and the cloud

Outsourcing digital data storage can relieve pressure on storage capacity and ICT resources while reducing costs. However, there are potential risks to the viability, access and use of the records. It may also be difficult to ensure that all copies of digital records are accounted for when they are destroyed or removed from storage.

Many of the records management risks can be mitigated once they are recognised. One of the strategies to mitigate risks is to perform due diligence; ie take care when evaluating a provider before entering into a contract. A records management risk assessment template is provided that can be used as a starting point to evaluate digital storage options.

Records management risk assessment template (pdf, 76kb) (Word, 1.1mb)

The Records Issues for Outsourcing including General Disposal Authority 25 (pdf, 111kb) authorises Australian Government agencies to transfer custody of Commonwealth records to contractors and sets out the terms and conditions that apply.

Some of the main information management risks are outlined below.

Breach of regulatory requirements

A number of laws affect how Australian Government agencies create and manage their records and information. Agencies must ensure that their choice of offsite storage provider does not put them in breach of any of the legislation to which they are subject.

Outsourcing storage does not lessen an agency's obligation to ensure that its information is managed, made accessible and disposed of accountably. Management of the information by the storage provider must also be accountable and in accordance with legal requirements.

To reduce the risk of breaching regulatory requirements:

  • consider what information is to be relocated to offsite storage and the legislation that applies to that information
  • investigate any legislative impediments to offsite storage location and control of the information
  • ensure adequate management and control of the records stored offsite, including those records created by the storage provider
  • ensure the storage provider understands its responsibilities in relation to the agency's information.

Consequences of storage location

Risks arise if data is stored in a jurisdiction that does not maintain appropriate standards or is not legislatively comparable to that of the agency. Some jurisdictions may have the power to demand access to all information, including classified information. Some may apply different laws to stored information, even if the information did not originate there. Storing information in such a jurisdiction may be a breach of legislation.

To reduce the risks associated with storing information in unknown jurisdictions:

  • ensure contracts specify where records are stored, for example, the location of servers and their main operations
  • determine if the storage provider uses subcontractors and where the operations of the subcontractor are located
  • determine whether information can be stored legally in jurisdictions outside the Commonwealth of Australia or if there are restrictions relating to particular jurisdictions
  • consider how security of data might be unintentionally affected by legislative and regulatory requirements of other jurisdictions.

Unauthorised access

The consequences of information being accessed without authorisation can be very damaging, particularly if the information is sensitive or personal. This may occur if a storage provider accidentally or deliberately discloses information to parties outside those specified in the contract.

In addition, new information is created when stored information is accessed or updated, or when changes are made to equipment in the storage facility. This transactional and relationship information may include details of the person(s) who has accessed the information, including location (if accessing information through internet portals or intranets), name or identification, and activities carried out. This information also needs to be protected from inappropriate access.

To reduce the risk of unauthorised access to, or use of, information ensure:

  • that the contract outlines conditions for access and use
  • the storage provider maintains system logs and audits, and has the capability to detect unauthorised access
  • the storage provider understands requirements for authentication and authorisation.

Privacy issues

If information is stored offsite, the storage provider must have adequate access controls and security measures, and be able to provide an appropriate level of security for personally sensitive material.

To reduce the risk to privacy:

  • know where the information is physically stored and who has access
  • assess storage providers’ ability to protect the sensitivity of the information
  • consider retaining sensitive information on agency servers.

Loss of access

Access to information may be lost as a result of a disaster if the provider has not performed appropriate backup. Cloud computing services may be affected by internet service disruption.

Reduce the risk of loss of access to information by:

  • writing contracts that outline access requirements
  • confirming agency access requirements can be met by the provider and in agreed times
  • ensuring proper backups are created and maintained by the provider; and 
  • checking disaster recovery and business continuity plans are in place and actively maintained.

The storage provider may cease business

Timely access to information may be lost if the storage provider, whether a data centre, digital repository or cloud computing service, goes out of business or is taken over by another company.

When a storage provider goes out of business, agencies may not be able to access their information and could lose control of vital business records.

If new owners of the storage facility do not honour previous arrangements, agencies may not know who has access to their information.

Reduce the impact by:

  • maintaining all contracts, agreements, licences and transfer documentation throughout the period of the contract
  • checking the storage provider has sound policies and procedures that reflect, and are appropriate for, the type of information stored
  • checking that data can be easily migrated to other providers without impediment if the provider has gone out of business or because an agency wishes to change providers at the end of a contract. 
  • knowing costs involved, if information be exported in an open format, and how long will it take before you can access your data again.

Readability and usability compromised

When storage providers upgrade hardware or software, there is a risk that agency software may no longer be compatible. The use of open formats supports readability and reduces the risk of information becoming unreadable.

Reduce the risk by:

  • specifying the format in which information should be returned when the contract ends
  • specifying the use of open formats for storage
  • requiring notice of any software or hardware changes, including migration of information, that may be undertaken by the storage provider
  • ensuring proper processes are followed when information is migrated.

Evidential value diminished

Evidential value of government records can diminish if their authenticity cannot be proven. Potential risks to authenticity can be unsuccessful migration undertaken by the provider, information not being properly secured, unknown access or access that is not logged and backups not being routinely performed.Reduce the risk of evidential damage for information, check the storage provider:

Metadata not appropriately maintained

Metadata makes information findable, usable and authentic. Mismanaged metadata may result in unusable information because it is difficult to find, understand, or authenticate.

Reduce the risk of metadata not being appropriately maintained:

  • assign metadata to records before they are transferred to the storage provider
  • check that applied metadata is enough to confirm the authenticity of information while being stored offsite
  • confirm contracts include minimum metadata requirements for process management of information.

Destruction or removal of information

When storing information offsite any disposal actions should be transparent and appropriately carried out. Factors to be considered include the removal of copies created during the transfer process and ensuring that removal or destruction of information is in accordance with the appropriate records authority. The Australian Government Protective Security Policy Framework and the Australian Government Information Security Manual set out requirements for rendering backups or copies unreadable and irretrievable. Cloud storage poses a particular problem because, by design, it creates multiple, geographically distributed copies to maintain availability. It may not be possible to verify that records have been destroyed and agencies must factor this risk into their use of the cloud.

When managing the disposal of records stored offsite consider:

  • if disposal is to be carried out by the storage provider, contracts should state what disposal action is to take place and when, see Administrative Functions Disposal Authority (pdf, 5787kb) – Information Management Control [class 1940]).
  • if records are to be returned to the agency, or destroyed, ensure no other version remains with the storage provider
  • records classed as RNA have been transferred to the National Archives at the appropriate time.

Due diligence assessment and service level agreements

Australian Government agencies should undertake appropriate assessment and checks when entering into any contract for an outsourced service. When storing digital information, questions that should be asked include:

  • where will the records and information be located?
  • can the provider meet the records management requirements of the Australian Government?
  • is the storage provider aware of the laws and regulations surrounding Australian Government information, such as the Privacy Act 1988?
  • will records be returned in a timely manner with no copy of the original records retained once the contract has ended?
  • who has access to the records and information, storage location and systems?
  • what backup plans and processes are in place?
  • is the provider certified in, or audited against, any standards?
As data storage is increasingly being offered as a service rather than a product specified in an end-user agreement, it is important that agencies use service level agreements to specify metrics. Doing so ensures that the information stored is protected, costs involved beyond the contract fees and charges are agreed, and what happens during server downtime is outlined.

Storage option definitions

What is a data centre?
A data centre houses computer systems and associated components such as servers, networks and data storage systems. Data centres are purpose built, permanent, shared enterprise facilities that can contain a full range of ICT equipment for agency use.

What is a digital repository?
A digital repository is used to retain and manage digital information, and aims to ensure the usability of stored digital objects over time. The term 'digital repository' is often interchangeable with 'institutional repository' and 'digital archive'. Common types of digital repositories include national libraries and archives, subject-based repositories or scientific-data archives.

What is cloud computing?
Cloud computing can be described as information technology resources delivered as a service through a network. These services may include procurement of software, platforms, infrastructure, or a combination of these. Outsourced cloud storage services may involve sharing, creating or storing information on remote servers accessed through the internet.

Copyright National Archives of Australia 2016