HARMONICA
Harmonised Access to Music and Music Information in Libraries
Libraries Project: PROLIB/HARMONICA 10453
Commission of the European Communities
LIBRARIES PROGRAMME
Practical Guidelines for the Future Development of Music Services in Libraries and Archives
Deliverable Number: D3.6.4
Version: / 1.0Date: / 15. 01. 2000
Authors: / Werner A. Deutsch (OEAW),
Confidentiality: / Public
Status: / Final
This document consists of <19> pages plus this cover
PAGE / DOCUMENT / VERSION
1 / D3.6.4 Guidelines / 1.0
CONTENTS
1. INTRODUCTION 1
1.1 General 1
1.2 Purpose and Scope 2
2. DIGITISATION and PRESERVATION 3
2.1 Preservation and Handling 3
2.2 Digitisation 4
2.2.1 General 4
2.2.2 Audio Technical Issues 5
2.2.3 Soundfile or "Bitstream" Formats for Ingestion 5
2.2.4 Images Associated to Sound 6
3. IN-HOUSE DIGITISATION or OUTSOURCE? 9
3.1.1 In-house pros: 9
3.1.2 In-house cons: 9
3.1.3 Outsourcing pros: 9
3.1.4 Outsourcing cons: 9
4. STORAGE 10
4.1 Storage Media for Data Acquisition (Ingestion) 10
4.2 Digital Preservation 10
5. DATA ACQUISITION (Ingestion) 12
6. ACCESS AIDS 13
6.1 Web Page Search 13
6.2 Interactivity: Means of Production 13
6.3 The Role of Audio and Music Descriptors (MPEG 7) 14
7. FURTHER READING 15
7.1.1 From LC-Audio Visual Prototyping Briefing Documents: 16
8. APPENDIX 17
8.1.1 Examples of PERL CGI-scripts for Web access to soundfiles. 17
8.1.2 A selection of Subcommittees, working groups, and officers of the Audio Engineering Society (AES) Standards Committee 19
PAGE / DOCUMENT / VERSION1 / D3.6.4 Guidelines / 1.0
1. INTRODUCTION
1.1 General
Digital music projects open new and exciting opportunities for music libraries and sound archives making their collections more accessible to designated communities as well as to the public. HARMONICA has been initiated in order to advance the knowledge on means to collect, store and organise music and music information in digital formats and make it available for searching, retrieving and processing via communication networks.
HARMONICA Workpackage 3 Networking and Digitisation currently provides the following deliverables covering topics as:
· D 3.1:
Analogue Documents, Carriers and Formats
· D 3.2:
Digital Information Including Transfer
· D 3.3:
Archiving and Managing Digital Information
· D 3.4:
Local and networked access to digital information collections
· D 3.6.1 & 3.6.2:
Developments of Digital Libraries
& Steps to a Digital Library
· D 3.6.3:
Remote Access and Transfer of Audio Recordings –
· D3.6.4
Practical Guidelines for the Future Development of Music Services
in Libraries and Archives (this deliverable)
WP3 deliverables D3.1 to 3.6.3 already offer an overview and references to information on document preservation, transfer of analogue audio recordings into digital formats (digitisation), local and remote storage of data as well as access and retrieval in a library or archive environment. Descriptions of current state of the art data acquisition, quality control and archive reference models are given. New developments in network capabilities and security providing better service to selected network traffic over various technologies have been presented.
1.2 Purpose and Scope
The purpose of this document (D3.6.4) is to provide links to practical guidelines for the development of digital music repositories, temporarily setting back considerations such as document type theory and access models in favour of describing practicable steps in regard to small and medium sized digital music services. Due to the diversity of problems arising from the integration of different media such as sound, text and images it seems appropriate to emphasise steps and actions applicable commonly to all without loosing the view for requirements specific to different media categories.
It is the perception of HARMONICA WP3 that by splitting up libraries and archives into independent sectors (and international organisations) as specialised in books, sheet music, still images or sound and video may have made easier to identify individual technical and internal management problems. Nevertheless, in order to maintain the role of libraries as most important part of the future information society and in order to develop integrated multimedia services based on binary objects carrying information from different media, close co-operation across sectors is urgently needed.
Increased interoperability of libraries and archives has been identified as a requirement for economical and information management reasons. The advance of WWW has made interoperability easily achievable. To allow libraries interoperate seamlessly standards, standards, standards[1] and broad commitment to follow the standards accepted is a necessary condition. Interoperability further requires agreement on organisational procedures such as exchange of material, purchase of material and mutual recognition of each other's users.
Vast amounts of relevant information has been published by academic institutions and by several professional library and archive associations on the topics under consideration. The current deliverable does not provide a comprehensive list of detailed practices and methods but gives exemplary basic information and links for further reading. This deliverable is not intended to be used in isolation. It completes the information provided in WP3-deliverables D3.1-D3.6 and relates to references and bibliographies cited within them.
On the other hand, the hardware and software used to store, process, manage and analyse music and music information has evolved a great deal since the last couple of years. Digital library and archive documents will no longer solely remain in the domain of computer specialists and, therefore, guidelines should also aim at the non-specialist.
Finally, as technical developments converge to feasable system solutions, a change of emphasis towards user’s requirements is essential. The current work should be seen in close relation to HARMONICA Workpackage 1 and 2. There, research directed specifically at end user requirements is included. This could initiate the production of model library systems incorporating the core functionality required by all music libraries and their users.
2. DIGITISATION and PRESERVATION
2.1 Preservation and Handling
Although guidelines for the preservation and handling of library material is not the main subject of this deliverable, an update to recently published documents ist provided:
IFLA Principles for the Care and Handling of Library Material.
International Preservation Issues Number One. May 1999
[Adobe Acrobat PDF format 374 KB – 72 pages]
Contains a section on Audio-visual Carriers
http://ifla.inist.fr/VI/4/news/pchlm.pdf
PRESERVATION OF LIBRARY & ARCHIVAL MATERIALS: A MANUAL
Third Edition; Revised and Expanded
Edited by Sherelyn Ogden
[Adobe Acrobat PDF format - 443 pages] http://www.nedcc.org/plam3/presman.pdf
Northeast Document Conservation Center, Andover, Massachusetts 1999
http://www.nedcc.org/plam3/newman.htm
Both publications provide extended coverage of threads to library materials, security and disaster planning as well as preventive measures and precautions to reduce risks.
Harmonica Deliverable 3.1 provides a description on handling of sound carriers. In practical terms, the advice for libraries and archives has been summarised as follows:
· The most expensive parts of preservation and transfer activities are (1) Selecting the documents to preserve, using criteria developed from risk evaluation as well as access and user requirement studies (2) The physical task of turning the pages of the selected book for item-by-item chemical treatment, filming, or digitising (3) Preparing the appropriate reproduction equipment and selecting the tapes, discs or cylinders for high quality controlled transfer etc. To produce a microfilm image, digital scan or audio file, the major cost is expended in the time required to gain access to each item so that each book, each film, each tape etc. is handled only once.
· To assure archival copying quality, document format specific plans are needed to recommend and review laboratory preservation work, establish technical guidelines and to develop generally accepted procedures for large preservation and transfer programs. These must take the new scientific understanding of document deterioration and transfer technology into account.
· Libraries and archives must recognize the importance of saving the original discs, original film and images, even after copying, unless they have deteriorated beyond any use. Saving the original documents remains a basic principle.
For further reading:
THE CARE AND HANDLING OF RECORDED SOUND MATERIALS
By Gilles St-Laurent
Music Division; National Library Of Canada January 1996
http://palimpsest.stanford.edu/byauth/st-laurent/care.html
2.2 Digitisation
2.2.1 General
It has already been pointed out in Deliverable D3.6.3 that many digitisation projects will be done for purposes of increased access to sound archives and library collections primarily, nevertheless, preservation can be obtained as a natural by-product. Digitisation should, therefore, generally be performed with a "preservation mindset."
· Determination of document and metadata file naming as well as directory structure convention
· Performing digitisation at the highest image resolution or sample rate appropriate to the nature and the informative content of the originals
· Creating and storing a linear coded master file that can be used to produce derivative and compressed files in order to serve a variety of current and future user needs
· Using system components that are non-proprietary
· Using file formats, editing systems and data compression techniques that conform with industry standards
· Creating backup copies of all files on a stable medium
· Creating meaningful metadata for files and associated documents including cataloguing issues (if appropriate)
· Monitoring and recapturing data if necessary
· Outlining a migration strategy for transferring data across generations of archive and access technology (plan for obsolescence of current hard- and software technology)
· Anticipating and planning for future usage and technological developments
2.2.2 Audio Technical Issues
This section describes envisaged formats and actions for ingestion and access of audio documents. The use of a specified Digital Audio Workstation is not mandatory although certain minimum sound quality requirements and controls are necessary. These include the use of pre-existing technical metadata as analogue calibration signals, preemphasis, time code signals as well as item specific data associated with the recordings and their reproduction devices. For detailed description of digitisation methods, especially the 24/16 bit word length discussion, WAVE file formats (BWF), narrow soundfile segmentation and creation of links to metadata see deliverable D 3.6.3: Remote Access and Transfer of Audio Recordings –
2.2.3 Soundfile or "Bitstream" Formats for Ingestion[2]
· MASTER SOUNDFILE
The master formats are defined as those under consideration as potential preservation-copy formats, intended to ultimately replace the current analogue audio formats such as open- reel tape analogue preservation masters and other professional recording studio quality analogue sound carriers: Uncompressed PCM bitstream, 96 or 48 kHz sample rate, 24 bit word length, WAVE file format
· DERIVATIVE SERVICE SOUNDFILES
Service files are converted versions or manifestations of the master file, at lower resolutions or lower sample rates to permit more convenient sound document delivery online or on consumer format sound carriers (44.1 kHz, 16 bit word length, Audio CD).
· Uncompressed soundfile
The uncompressed soundfile will be produced if the system is disabled to deliver the master soundfiles to high-end audio workstations in an efficient manner. Uncompressed PCM bitstream, 48 or 22 kHz sample rate, less-than- 24 bit word length (probably 16 bit), WAVE file format
· Higher quality compressed soundfile (Compression Ratio ≤1:12)
This category of soundfiles is intended to be used most effectively if presented in streaming or pseudo-streaming mode: Compressed MP3 bitstreams, possible use of variable bit rate encoding, in a mode called "HiFi" at sites like MP3.com.
· Lower quality compressed soundfile (Compression Ratio > 1:12)
This soundfile format is very likely to be useful for the Internet access. It will be most effective if presented in streaming or pseudo-streaming mode.
Option A:
Compressed MP3 bitstreams, in a mode called "LoFi" at sites like MP3.com.
Option B:
RealAudio file at a data rate at the high end of the RealAudio range.
Option C:
several additional compression schemes have to be considered.
· MEANS OF PRODUCTION
It is assumed to produce the derivative soundfiles at the time of ingestion, at some other "post-processing" moment, or even on-the-fly on the time of delivery. Some libraries have decided to produce sets of three files at the time of digitisation:
(1) the master file,
(2) the uncompressed soundfile, and
(3) the higher quality compressed soundfile.
Note 1:
Lower quality (lossy coded) soundfiles occupy extremly low amount of disk storage capacity on account of the high compression ratio they provide. The lower quality compressed soundfile may either be produced at the time of ingestion or, in any case, ought to be easy to be produced in a later post-process.
Note 2:
At the time of master soundfile creation, the principal decision has to be made which type of metadata (if any) have to be stored in the soundfile format itself or on additional metadata files to be referenced by the database system. The specification of user chunks in the RIFF/WAVE file format provides a convenient method for linking of metadata and adjunct material.
Note 3:
Special attention must be taken that metadata links are adjusted perfectly equal to different uncompressed and compressed soundfile representations of the same signal. Internal soundfile addresses and segment boundaries as well as cue in and cue out points should be counted in sample numbers relative to signal starting point, offset and segment duration of the corresponding uncompressed master file. An arbitrary time code can be applied for external use.
2.2.4 Images Associated to Sound
This section provides scanning recommendations judged as minimum necessary for responsible conversion and for achieving an acceptable level of image quality by the Colorado Digitisation Project[3]. These recommendations rely upon technical capabilities and limitations corresponding the status quo. Decisions on image quality and resolution should be based on the needs of users, how the images will be used, and the nature of the materials to be scanned (dimensions, color, tonal range, format, material type, etc.). Reduced resolutions probably are sufficiant for images associated to sound documents, recognizing that images of sound collections differ in the ways they are used and accessed in comparison to the usage of high quality photographs in institutions and the clientele focusing on image contents primarily.
· MASTER IMAGE FILE
Many digital imaging projects scan a high-quality "master" or archival image and then derive multiple versions in smaller sizes or alternative formats for a variety of uses. A high-quality master image will make the investment in the image capture process worthwhile. Since user expectations and technology change over time, a digital master must be available and rich enough to accommodate future needs and applications. The master image should be the highest quality affordable; it should not be edited or processed for any specific output; and it should be uncompressed. Intensive quality control should be applied in creating master image files