Coping with errors in audio and video files

My first encounter with technology for coping with errors in audiovisual files was at the AES conference in Budapest (1) in 2001 – where Jorg Houpert of Cube-Tec had his laptop stolen. That conference had a paper on minimising the effects of losing an entire block of audio data, by spreading the data across the whole file. Loss of a block of data produces a reduction in signal-to-noise ratio of the whole file, rather than a gap in the audio. The process was considered by NOA of Vienna, though never commercially implemented.

But that wasn’t the first audio technology to reduce effects of errors. Sony Walkman CD players used buffering to allow re-reading of tracks. The more expensive the player, the more buffering and the greater the likelihood of uninterrupted audio even when jogging.

Similar real-time or near real-time corrections are built into most video playback decks, though often the technology is proprietary, undocumented and inaccessible. One of the main advantages of the work of the DAVID project ( is to produce open and well-documented error-recovery processes.

The error indications produced by playback equipment for the video DV format are documented, and much has been gained from examining and using DV error information. The whole issue of how to get the best DV playback has been the subject of useful work by Dave Rice (2), independent consultant, and Chris Lacinak, AV Preserve (3); both in New York City.

But coping with playback errors in video was already part of videotape playback technology in analogue equipment, where methods of storage of one line of video would allow ‘concealment’ of a missing line (or part of a line). With digital videotape these methods just became easier and more powerful, leading to the ability to conceal whole frames of bad data.

The ability of videotape players to conceal errors – many errors – while a video file with a single error was unreadable, was brought to my attention by Jean-Hugue Chenot of INA around ten years ago. INA had tried to interest technology companies in doing something to make video files more resilient, but there had been no take-up.

Some years after that, I was invited to give a keynote talk at a conference I wouldn’t usually attend, specialising in still images: the Image Science and Technology conference Archiving 2008, held in Bern, Switzerland. There was a paper by Prof Manfred Thaller of Cologne, reporting on the striking difference in the effects on an image of one or a small number of file errors, depending upon whether the encoding was compressed or uncompressed. For any kind of compression, lossless or lossy, a single error was magnified by the computations used to turn the data back into an image. For an uncompressed file, one error basically affected one pixel. For a compressed file, large parts of an image were often destroyed. Volker Heydegger (4), a student of Prof Thaller, had subjected hundreds of thousands of files to various levels of readback error, and compiled statistics on the average number of pixels affected. In general compression magnified the effects of errors by 100 to 1000 times, or even more – depending upon the type of file and the type of compression. A one-byte error would, on average, corrupt 17% of a lossless JPEG2000 file, and 33% of a lossy JPEG2000 file.

This paper caused quite a stir. The head of technology at Adobe said it was ‘bad science’, because it would be better to store multiple compressed copies than to abandon all use of compression. Of course Adobe was a leading producer of PDF files, which are uncompressed yet still have the property of magnifying errors, when compared to .txt files. One could say .txt files were too primitive to be a fair comparison, as they have no structure, no markup. An obvious way to introduce markup without also introducing error magnification is by using XML files, which are just as resilient at .txt files. In fact, they are .txt files, really. And now, years later, Microsoft does use XML for its WORD files, and then re-introduces error magnification by packaging the XML files in a ZIP file – which it renames as DOCX.

Regarding whether it was better to have one resilient copy or three compressed copies – taking up the same space but without readback error resilience – it wasn’t until PrestoPRIME and the tools for modelling storage and file management (5) developed by ITI, that we were able to get statistical answers to such questions.

Meanwhile, Joanneum had been working with INA – and for a while also with the BBC – on image, video and film restoration. There is a whole range of technology available for restoration, and the DIAMANT system developed by Joanneum and marketed by HS-ART (6) is the market leader.

Back in Vienna, Christoph Bauer at ORF was encountering their first (and let’s hope only) major outbreak of file errors. An encoding problem had produced incorrect IMX (D10) files (7). Previously, Laurent Boch of the R&D department of the Italian broadcaster RAI had already identified a range of problems with the IMX format, so the ORF people were able to get help, and sympathy, from partners in EU projects and in FIAT/IFTA, the professional organisation for television archives. ORF had problems with fully half of the files from 23,000 hours of video. ORF needed more than sympathy, and one of the first successes of the DAVID project was to develop software to re-wrap the video in a standards-compliant fashion. This solution has become the Cube-Tec MXF Legalizer (8).

Related work on transferring 300,000 hours of video at ORF had a quality-control problem. The BBC approach of “completely manual, watch every frame and listen to every second” quality control was expensive and slow. Joanneum Research had the technology to automatically detect defects, because that is the starting point for image restoration systems such as DIAMANT. However there is a long route from fault detection to a working quality-control system, as evidenced by the BBC’s own struggle to introduce automation into their workflow. A principal problem is generating too many false alarms, each of which has to then be manually checked – obviating the whole purpose of the automation. The only solution to false alarms is better detection, which in turn requires hundreds and eventually thousands of hours of manually-corrected training and test data.

ORF has worked over many years with Joanneum, but the DAVID project brought the technology and the practical application close enough together, for sufficient time and with sufficient training and test material, to finally develop an effective process, which is now the Vidicert (9) product.

This has been my “view from a distance” of the DAVID project, and there is much more to the project that I haven’t mentioned. I hope interested parties will take the time find out more because I’ve only mentioned two of the seven areas that the DAVID website lists as ‘spotlights‘. What I have most enjoyed seeing from DAVID is solutions, at last, to problems that many interrelated projects have been working on since, originally, the Aurora video restoration project of nearly 20 years ago.

My congratulations to the DAVID partners.

(1) Stocker, Daniel. Protective Nonlinear File Structure. In Proceedings of the AES 20th International Conference: Archiving, Restoration, and New Methods of Recording, Budapest, Hungary, 2001 October 5-7
(2) Dave Rice:
(3) Digital Tape Preservation Strategy: Preserving Data or Video? By David Rice and Chris Lacinak – December 2, 2009
(4) Heydegger, Volker; Analysing the Impact of File Formats on Data Integrity. Archiving 2008, Bern
(5) Digital Preservation Tools from IT Innovation
(6) Diamant and DustBuster
(7) D6.2: Longitudinal CoP Impact Analysis.
(9) VidiCert

Leave a Reply

Your email address will not be published. Required fields are marked *