Acoustic Data Format Manual

21-06-00 Author: Martin van Gijzen

Acoustic data format manual TNO-FEL

ROBLINKS

ACOUSTIC DATA FORMAT MANUAL

ROBLINKS MAST III PROJECT

MAS3-CT97-0110

1 Introduction 3

2 Documentation 4

3 Quality checks 5

4 Acoustic data format 6

5 Sample MATLAB code to retrieve the data 9

6 Overview of the ROBLINKS NetCDF-files. 10

1 Introduction

The EC MAST III project ROBLINKS is a collaboration between Thomson Marconi Sonar (TMS-SAS), the Ruhr University Bochum (RUB) and the Physics and Electronics Laboratory of the Dutch Organisation for Applied Scientific Research (TNO-FEL). The aim of the ROBLINKS project is to develop robust coherent communication concepts and algorithms based on continuous identification of the channel response. The algorithms and waveforms are tested and evaluated with real data, collected under shallow water conditions.

To collect this data set a trial has been held in the North Sea, near the Meetpost Noordwijk (MPN, located 52016’26”N 04017’46”E) from April 30 to May 7 1999. During this trial a great number of acoustic and environmental experiments have been performed which resulted in a rich and valuable data set.

One of the two main deliverables of the ROBLINKS project is a documented and quality-checked data-set that is available for future use by other interested parties, to test their own algorithms on real sea-data. The creation of this data set is a joint effort of TNO (responsible within the ROBLINKS project for data management), MARIS (advice), and IFREMER/SISMER (banking and dissemination after the project).

This manual describes in detail how the acoustic data gathered during the ROBLINKS trial are stored and how they can be retrieved. Further information about the formats and the dissemination of the complete data set (acoustic and non-acoustic) can be obtained from http://www.ifremer.fr/sismer/program/roblinks/.

2 Documentation

The complete documentation of the acoustic data set is composed of the following documents:

Data management plan [3]. This document describes data-management procedures and responsibilities of the different partners for the proper implementation of the procedures. The report also details the transmission hardware and the data-acquisition system that has been used during the trial.
Calibration report [1]. This report describes the procedures followed to calibrate the transmitter (the entire chain from PC to acoustic source), and the receiver array, and gives the response functions of the equipment.
Measurement plan [2]. This document specifies the different acoustic experiments and environmental measurements.
Cruise report [4] and Cruise Summary Report [5]. These reports describe al significant events that occurred during the trial. The Cruise report also outlines the mounting of receiving equipment and the deployment of the transmitter.
ROBLINKS Second Year Scientific Report [6]. This report gives the theoretical background of the transmitted signals, describes possible receiver algorithms and gives results of the analysis of received signals.

3 Quality checks

The selected data correspond to the most interesting and thoroughly analysed signals. Hence their value and usefulness has thus been established. Results are reported for example in [6].

The only processing that has been performed on the data that will be disseminated is the conversion to NetCDF format. Hence the NetCDF files contain the raw data. During the conversion process additional checks have been performed (calculation of spectrum, visual and aural inspection of the converted signal) to check that no mistakes have been made.

4 Acoustic data format

All acoustic data are stored in NetCDF files [7]. A NetCDF file is a stored list of named, multi-dimensional, numerical arrays called "variables". Their sizes are established by zero-or-more named "dimensions", one of which may be extensible ("record-dimension"). Variables may contain other named information called "attributes", and the file as a whole may carry similar information of its own,

termed "global attributes". NetCDF files are platform independent and therefore highly portable. Interface libraries exist for Fortran, C and MATLAB. The ROBLINKS NetCDF files have been generated using the MATLAB NetCDF Toolbox that can be obtained from [8].

The ROBLINKS NetCDF-files have a very simple structure. They are composed of three main blocks:

· A header block of global attributes, describing the experiment (keywords, time, date and location, cruise reference number, and data manager);

· An equipment block of global attributes, giving the settings and units of the hardware (high-pass filter of the amplifier, gain of the amplifier, sensitivity of the recorder, sample frequency);

· The actual acoustic data, stored as two-dimensional variable (channels times samples). The first dimension specifies the channel or hydrophone number. The hydrophone closest to the surface corresponds to channel 1, the deepest hydrophone to channel 20. In some files additional channels are included which correspond to SWAN hydrophones. Three extra channels correspond to the three leftmost hydrophones (see [4] for the layout of the SWAN hydrophones), counting from top to bottom. Six channels indicate that all SWAN hydrophones are included, counting from left to right then top to bottom. Occasionally some of the channels are left out to limit the size of the file below 2 Gb.

Table 1 specifies all the entities that compose a ROBLINKS NetCDF file. Column 1 gives the names of the entities, column 2 what they are (dimension, variable, or attribute), and column 3 gives their data-type. Note that all entities corresponding to the header and to the settings of the equipment are character data. Column 4 explains the meaning of the entity if this is not obvious from the name already. Column 5 gives example values for the entities.

Entity name / Dimension/variable/attribute / Data type / Meaning of the entity / Example
Description / Global attribute / Char / Some keywords / TNO3, 2km, primary track
start_date / Global attribute / Char / 30-04-1999
start_time / Global attribute / Char / 19:13:00 UTC
end_date / Global attribute / Char / 30-04-1999
end_time / Global attribute / Char / 19:23:00 UTC
Latitude / Global attribute / Char / 52:17.0 N
Longitude / Global attribute / Char / 004:13.0 E
geograph_area / Global attribute / Char / North Sea
cruise_ref_nmb / Global attribute / Char / FI641999550010
data_manager / global attribute / Char / the chief of the French IFREMER/SISMER NODC
hp_filter / global attribute / Char / Cut-off frequency high pass filter / 100 Hz
ampl_gain / global attribute / Char / Amplifier gain / 12 dB
rec_sens / global attribute / Char / Recorder sensitivity / 5 V
samp_freq / global attribute / Char / Sample frequency / 48000 Hz
comment1 / global attribute / Char
comment2 / global attribute / Char
comment3 / global attribute / Char
nr_of_chnls / dimension / Long / Number of hydrophones / 20
data_length / dimension / Long / Number of time samples / 28960000
channel_data / variable / Short
[nr_of_chnls data_length] / Actual acoustic data

Table 1. Structure of a ROBLINKS NetCDF-file, containing acoustic data of one experiment.

Figure 1 shows the lay-out of a ROBLINKS NetCDF-file, viewed with the MATLAB-tool ncbrowser.

Figure 1. Layout of the NetCDF-file, viewed with the MATLAB-tool ncbrowser.

5 Sample MATLAB code to retrieve the data

This section gives sample MATLAB code to read out the data from a ROBLINKS NetCDF-file. The name of the NetCDF-file should be stored in the MATLAB character variable nc_filename.

% Open netcdf-file (name in variable nc_filename)

nc = netcdf( nc_filename , 'nowrite');

% Short description to identify the experiment

description = nc.description(:);

% Date and time of the transmission:

start_date = nc.start_date(:);

start_time = nc.start_time(:);

end_date = nc.end_date(:);

end_time = nc.end_time(:);

% Location of HNLMS Tydeman:

latitude = nc.latitude(:);

longitude = nc.longitude(:);

geograph_area = nc.geograph_area(:);

% Trial identification:

cruise_ref_nmb = nc.cruise_ref_nmb(:);

data_manager = nc.data_manager(:);

% Settings of the equipment:

% Cut-off of high-pass filter (always 100 Hz):

hp_filter = nc.hp_filter(:);

% Amplifier gain in dB, always either 6 or 12;

ampl_gain = nc.ampl_gain(:);

% SONY recorder sensitivity (always 5 V)

rec_sens = nc.rec_sens(:);

% Sample frequency of SONY recorder (always 48000 Hz)

samp_freq = nc.samp_freq(:);

% Number of hydrophone channels (20 or 26, depending on the day),

% first dimension of variable channel_data

nr_of_chnls = size(nc{'channel_data'},1);

% Further remarks:

% comment1 is always used to warn that due to the transmission

% hardware all frequencies are upwardly shifted by a factor of 1.02629

comment1 = nc.comment1(:);

comment2 = nc.comment2(:);

comment3 = nc.comment3(:);

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% End of header, now read a portion of the acoustic data,

% for example from channel 11, sample 11000-12000

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

channel = 11;

first_sample = 11000;

last_sample = 12000;

chan_data = nc{'channel_data'}(channel,first_sample:last_sample);

6 Overview of the ROBLINKS NetCDF files.

The total number of ROBLINKS NetCDF files is 30. The files correspond to ten different transmitted signals, and to three transmission ranges for each signal. Four of the ten signals have been defined by TMS-SAS, four by RUB, and two by TNO. The selected signals and transmission ranges comprise the most interesting and illustrative part of the total data set gathered during the ROBLINKS trial. The signals are described in [6] where also analysis results are given. Table 2 gives an overview of the selected signals and ranges.

Signal / Name / Range / Tracks / Channels
included / Scientific experiment
1 / TNO1_MAY1.nc
TNO1_MAY2.nc
TNO1_MAY6.nc / 2 km
5 km
sailed / Primary
Primary
Primary / 20 VLAS
20 VLAS
20 VLAS + 6 SWAN / 40 FM sweeps
2 / TNO3_MAY1.nc
TNO3_MAY2.nc
TNO3_MAY6.nc / 2 km
5 km
sailed / Primary
Primary
Primary / 20 VLAS
20 VLAS
20 VLAS + 6 SWAN / Conventional signals (BPSK, QPSK, PAM).
3 / RUB1_APRIL30.nc
RUB1_MAY2.nc
RUB1_MAY5.nc / 1 km
5 km
2 km / Primary
Primary
Secondary / 19 VLAS[1]
20 VLAS
20 VLAS +
3 SWAN / MSK1
4 / RUB5_APRIL30.nc
RUB5_MAY2.nc
RUB5_MAY5.nc / 1 km
5 km
2 km / Primary
Primary
Secondary / 20 VLAS
20 VLAS
20 VLAS +
3 SWAN / OQPSK1
5 / RUB9_APRIL30.nc
RUB9_MAY2.nc
RUB9_MAY5.nc / 1 km
5 km
2 km / Primary
Primary
Secondary / 20 VLAS
20 VLAS
20 VLAS +
3 SWAN / DPSK1
6 / RUB12_APRIL30.nc
RUB12_MAY3.nc
RUB12_MAY5.nc / 1 km
5 km
2 km / Primary
Primary
Secondary / 20 VLAS
20 VLAS
20 VLAS +
3 SWAN / DPSK4
7 / TMS02_APRIL30.nc
TMS02_MAY2.nc
TMS02_MAY5.nc / 1 km
5 km
2 km / Primary
Primary
Secondary / 20 VLAS
20 VLAS
20 VLAS +
3 SWAN / FBO131
8 / TMS04_APRIL30.nc
TMS04_MAY2.nc
TMS04_MAY5.nc / 1 km
5 km
2 km / Primary
Primary
Secondary / 20 VLAS
20 VLAS
10 VLAS[2] +
6 SWAN / FBO511
9 / TMS07_APRIL30.nc
TMS07_MAY2.nc
TMS07_MAY5.nc / 1 km
5 km
2 km / Primary
Primary
Secondary / 20 VLAS
20 VLAS
20 VLAS +
3 SWAN / HBO263
10 / TMS14A_APRIL30.nc
TMS14A_MAY1.nc
TMS14B_MAY3.nc / 1 km
2 km
5 km / Primary
Primary
Primary / 20 VLAS
20 VLAS
20 VLAS / IMG01

Table 2. Overview of signals in the data set.

References

[1] A. Gerk and P. A. van Walree, Calibration report, http://www.tno.nl/instit/fel/roblinks (project secretariat)

[2] M. B. van Gijzen, Measurement plan, http://www.tno.nl/instit/fel/roblinks (project secretariat)

[3] M. B. van Gijzen, Data management plan, http://www.tno.nl/instit/fel/roblinks (project secretariat)

[4] M.B. van Gijzen and P.A. van Walree, ROBLINKS Cruise Report, http://www.tno.nl/instit/fel/roblinks (project secretariat)

[5] ROBLINKS Cruise Summary Report http://www.ifremer.fr/sismer/program/roblinks/camlst.htql

[6] ROBLINKS Second Year Scientific Report http://www.tno.nl/instit/fel/roblinks (project secretariat)

[7] UNIDATA, NetCDF documentation, http://www.unidata.ucar.edu/packages/netcdf/docs.html

[8] MexCDF and NetCDF Toolbox For Matlab-5 http://crusty.er.usgs.gov/~cdenham/MexCDF/nc4ml5.html

[1] Channel 20 (deepest hydrophone) of VLAS array not included.

[2] Only odd hydrophones of VLAS array included.