6- What standards should I follow?

What standards should I follow?

This answer assumes that the full question is: I have content on shelves; I want to put it into files and put those files on mass storage of some sort; what standards should I follow?

This answer started out covering four areas where people might ask “what is the standard, or at least recognised best practice?”
  1. the digitisation itself: sampling rate, quantisation, the complexities of colour video, the scanning of film;
  2. the encoding of the results of the digitisation, because there are many options;
  3. the file format to hold the resultant audio and video; again, several options; and
  4. digital audiovisual media: the problem of material that is already digital, but is NOT in a file format. This covers everything from audio CDs to the latest forms of digital videotape — a 30 year span of digital media of many types, with a surprising amount of complexity
To keep answers to one page and in an effort to provide clarity, there are now four FAQs, one for each of the above areas.
The following answer is for the question: what standards do I follow for the archive quality digitisation of analogue audio and video, and of film (real film)?
Audio: this is the easy one. There is good documentation and there is a strong concensus of opinion. The hard part is actually getting the best possible playback in order to do the best to meet the standard!
Archive preservation standard for audio:
  • Quantisation: “24 bit” = 24 bits in the number representing each sample. 
  • Sampling rate: at 48 kHz minimum; 96 kHz is recommended.
  • Don’t compress, at all
  • Save as BWF, the Broadcast Wave version of the WAV file
Source of the standard: IASA Technical Committee, Guidelines on the Production and Preservation of Digital Audio Objects, ed. by Kevin Bradley. Second edition 2009. (= Standards, Recommended Practices and Strategies, IASA-TC 04). 
  • 24-bits: you’ll be lucky to get 19 to 21 bits of true data, but for simplicity (and to have a multiple of 8) people refer to 24-bit quantisation
  • 48kHz minimum: the CD standard, 44.1 kHz is tolerated; it has practical advantages when 44.1 kHz is a working format for an archive or institution
 A useful practical document on playback and digitisation from the British Library Sound Archive is the Endangered Archives Programme Guidelines for the preservation of sound recordings.
Video: there may be arguments about what to do to preserve video, but there are two clear standards for how video should be coded as a digital signal.
1) SD: Video from the 1950’s to the 1990’s was mainly standard definition = SD, which meant 525 lines in the US (and many other countries, particularly those with 60 Hz electricity systems), and 625 lines in the UK (and countries using 50 Hz electricity). The SD standard was agreed in 1980 and has been the basis for digital video equipment ever since. It is ITU-R Recommendation BT.601.
“Rec 601” video is what comes out of the digital connectors on professional video cameras and videotape equipment, and is the standard signal networked across television production. Or was, because since the 1990s video has been changing from SD to high definition = HD.
Rec 601 has variants, but for digitising analogue SD content the version to use is 10-bit data, 4:2:2 allocation of samples to the three components of colour video. Any professional ‘capture card’ will deliver this standard. It has a full bit rate of 270 megabits per second (Mb/s), but that is tied to a real-time signal which has ‘blanking intervals’ which were needed by cathode-ray television sets. For files, blanking no longer applies so the data rate can be chopped to 200 Mb/s.
Analogue video in archives will be SD, and should be digitised to Rec 601. All the arguments about analogue video have to do with encoding and file formats, answered in the FAQ “How should audio and video be encoded for preservation?” and the FAQ “What file format(s) should I use?”
There are also complexities about material which ‘digital but not in files’, such as digital videotape (DV, Digibeta, IMX and more). How to deal with that material is in the answer to the FAQ How do I preserve digital media?
2) HD: There is NO analogue high definition video, so any archive digitisation of analogue video should follow the SD standard, Rec 601. But there is an HD equivalent to Rec 601: ITU-R Recommendation BT.709. There is a lot of complexity regarding HD, which is one reason why dealing with digital media has its own FAQ. All HD is digital, therefore it is either on digital media (eg HD-CAM SR tape) or is already in files on mass storage. 
Film: there are many complexities about film: a range of gauges and image formats, dozens and dozens of particular types of film, at least half a dozen possible versions of ‘the same thing’ ranging from shooting negatives through cut negative to interpositive, internegative and various kinds of prints. 
There is no standard for the digitisation of film for preservation, because film is many things, unlike audio. Whether film needs digitisation for preservation is hotly debated. Certain positions are clear:
  • film that is demaged will have to be digitised to be restored, because all the powerful restoration processes are digital;
  • film that is suffering colour fade or vinegar syndrome can be kept below freezing to slow the chemical reactions, which is not preservation so much as putting off the inevitable digitisation. The expense of sub-zero storage just steals from a digitisation budget, and the time spent in storage just increases the cost of that inevitable digitisation;
  • film in broadcast archives will never be used without digitisation. Older broadcasters have a lot of film: the Presto 2002 survey of 10 major European broadcasters found that 1/3 of their television archives were on film, not videotape 
There are technical guidelines and examples of good practice. When the Dutch national audiovisual archive NISV started a major project of film digitisation, they looked at many options and have contributed their findings to the PrestoCentre: Film Scanning Considerations — which also has a six page digest.
The requirements in broadcasting for film scanning were investigated by the European Broadcasting Union, in two reports also available from PrestoCentre:
The underlying issue of what image information is on a negative and what scanning technology is needed to recover that information was studied by the ITU standards body in 2001-2002. Their work and other highly technical studies of all the components of the ‘optical chain’ between object and film, between film and viewer and between film and scanner are reviewed in a report from the scanner company DFT: What Digital Resolution is Needed to Scan Motion Picture Film: 4K, or Higher?. However there is evidence that for ‘technically perfect’ still-image negatives, there is information that can be gained from an 8k scan instead of a 4k scan: Understanding Image Sharpness
One way to deal with complexity is to ignore it. Here is a simple answer to a complex issue; the plan is for the complexities (and rebuttals!) to be dealt with in more detailed documents that will be referenced here as they are produced.
Standard practice for digitisation of film at archive preservation quality:
  • 16mm film: scanning at 2k with 10-bit quantisation using a log scale (the NISV approach)
  • 35 mm film: scanning at 4k with 14-bit linear quantisation recommended (the DFT approach). For exceptionally high-value and ‘technically perfect’ items, scan at 8k.
The PrestoCentre recommendation for all analogue content, including film: digitise now; it will only get harder and more expensive in the future. 

Information and Consultancy on Audiovisual Preservation