1523649

(1)

Peer-Reviewed Technical Communication

The Watermark Benchmark for Underwater Acoustic Modulation Schemes

Paul A. van Walree , Member, IEEE, Franc¸ois-Xavier Socheleau, Member, IEEE, Roald Otnes, Senior Member, IEEE, and Trond Jenserud

Abstract—Watermark is a freely available benchmark for physical-layer schemes for underwater acoustic communications. It allows researchers to test and compare algorithms for the physical layer under realistic and reproducible conditions. The benchmark is a shell around the validated channel simulator Mime, which is driven by at-sea measurements of the time-varying impulse response. The first release of Watermark is issued with a library of channels measured in Norway (two sites), France, and Hawaii, offering three frequency bands (4–8, 10–18, and 32.5–37.5 kHz), single-hydrophone and array receivers, and play times varying from 33 s to 33 min.

Index Terms—Acoustic communication, benchmark testing, channel simulator, signal fluctuations, wideband systems.

I. INTRODUCTION

N

UMEROUS modulation schemes have been proposed for underwater acoustic communications during the past decades. In the majority of cases it is impossible to compare the performances of these schemes in a meaningful way, as field tests are performed in different environments. Moreover, the variation in channel characteristics is large even in a given environment [1], [2]. Unlike terrestrial radio- frequency communications, underwater acoustic communications is a technological field deprived of standard test channels. A variety of channel models and channel simulators have been proposed for acoustic communications, e.g., [3]–[10]. These models have varying degrees of realism, completeness, and availability, and in most cases they are tuned to a single environment and platform configuration.

The underWater AcousTic channEl Replay benchMARK (Water- mark) is a realistic simulation tool now made available to the underwater communications community. It is a shell around the validated FFI channel simulator “Mime” [7], [11], which convolves user waveforms with measured channels. The initial release is provided with a library of channels from four environments. The primary objective of the present initiative is to provide an easy-to-use benchmark that enables development, testing, and comparison of physical-layer algorithms under realistic and reproducible conditions.

Watermark was presented at UComms 2016 with two test channels for a single receiver [12]. Following the presentation at UComms, IMT Atlantique (formerly known as Telecom Bretagne) generously offered array data [13] for release with the benchmark, and, soon after, Scripps Manuscript received January 9, 2017; revised April 11, 2017; accepted April 21, 2017. Date of publication May 23, 2017; date of current version October 11, 2017. (Corresponding author: Paul A. van Walree.)

Guest Editor: I. Akyildiz

P. A. van Walree, R. Otnes, and T. Jenserud are with the Maritime Sys- tems Division, Norwegian Defence Research Establishment, Horten NO- 3191, Norway (e-mail: paul.vanwalree@ffi.no; roald.otnes@ffi.no; trond.

jenserud@ffi.no).

F.-X. Socheleau is with IMT Atlantique, Lab-STICC, Bretagne Loire Uni- versity, Brest 29238, France (e-mail: fx.socheleau@imt-atlantique.fr).

Digital Object Identifier 10.1109/JOE.2017.2699078

Institution of Oceanography approved a request to include array data from the Kauai Acomms MURI 2011 (KAM11) experiment [14], [15].

This technical communication summarizes the principle of channel replay, presents the available test channels, discusses the operation of Watermark, and gives examples of its use. Format descriptions and detailed user instructions are provided in a separate report [16]. The software, test channels and user manual are available for download from [17]. Watermark is programmed in MATLAB and has been successfully tested on Windows, Linux, and Mac OS.

II. MIMECHANNELSIMULATOR

Mime distorts input waveforms by convolving them with measured channels, or statistical variations thereof. Its operation principle is known as channel replay

y(t) =

_∞

−∞

ˆh(t, τ)x(t−τ) dτ+n(t) (1) wherex(t)is the input signal,ˆh(t, τ)is a time-varying impulse response (TVIR),n(t)is a noise term, andy(t)is the distorted output signal. Channel replay can be performed in several ways [7], [8], [11], [18], [19], and recently the term has also been used for network simulations with measured packet error statistics [20].

Mime has a direct-replay mode whereˆh(t, τ)is the measured channel, which is an estimate of the true channelh(t, τ). The maximum simulation (play) time with direct replay equals the measurement time.

Mime also has a stochastic replay mode, which generates TVIR realizations that are statistically consistent with the measured channel.

This mode can run forever on a single input measurement. The realism of direct-replay simulations depends primarily on the quality of the channel measurement. The measurement errors have been well documented in the case of correlative channel sounders [21], and the combination of Mime and correlative sounders has been well validated [7], [11]. Simulated bit and packet error ratios (BERs and PERs), and receiver output signal-to-noise ratio (SNR), are close to the corresponding values measured at sea. The stochastic replay mode has also yielded good validation results, but has limitations regarding the reproduction of nonstationary channels, time-varying delays, and correlated scattering. To guarantee realistic simulations under all conditions, Watermark is released only with the direct-replay mode of Mime. This mode faithfully reproduces all effects mentioned in [2], including channel nonstationarity, correlated scattering, time-varying delays, and ultrawideband channels. The one type of channel that cannot be reproduced is the overspread channel, because it cannot be measured properly.

Another advantage of direct replay over stochastic replay is that the code is simple, fast, and free of user parameters. Its disadvantage is that a significant play time requires a significant measurement time, and a correspondingly bulky channel library. Acoustic channels are usually not stationary over longer periods, for reasons such as platform motion, 0364-9059 © 2017 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution

requires IEEE permission. See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

(2)

1008 IEEE JOURNAL OF OCEANIC ENGINEERING, VOL. 42, NO. 4, OCTOBER 2017

TABLE I

TESTCHANNELPARAMETERS ANDSOUNDINGCONDITIONS

Name NOF1 NCS1 BCH1 KAU1 KAU2

Environment Fjord Shelf Harbor Shelf Shelf

Time of year June June May July July

Range 750 m 540 m 800 m 1080 m 3160 m

Water depth 10 m 80 m 20 m 100 m 100 m

Transmitter depl. Bottom Bottom Suspended Towed Towed Receiver depl. Bottom Bottom Suspended Suspended Suspended Probe signal type LFM train Pseudonoise Pseudonoise LFM train LFM train

−3-dB frequency band 10–18 kHz 10–18 kHz 32.5–37.5 kHz 4–8 kHz 4–8 kHz

Roll-off fact. 1/8 1/8 1/10 1/8 1/8

Sounding duration 32.9 s 32.6 s 59.4 s 32.9 s 32.9 s Delay coverage 128 ms 32 ms 102 ms 128 ms 128 ms Doppler coverage 7.8 Hz 31.4 Hz 9.8 Hz 7.8 Hz 7.8 Hz

Type SISO SISO SIMO SIMO SIMO

# hydrophones 1 1 4 16 16

Element spacing — — 1 m 3.75 m 3.75 m

# cycles 60 60 1 1 1

Cycle time 400 s 600 s — — —

Total play time 33 min 33 min 1 min 33 s 33 s

weather variability, movement of the tides, etc. Nonstationarity of test channels is a price paid for the realism, just like testing at sea has to deal with channel nonstationarity. For a fair and reproducible comparison between modulation schemes, it is recommended that performance results reported for Watermark channels are averaged over the full play time of the replay channel.

III. CHANNELLIBRARY

Watermark is released with sounding (TVIR measurement) data for five channels. Table I summarizes measurement conditions and test channel parameters, while Fig. 1 provides schematic drawings of the platform deployment. The transmitter is bottom-mounted in Norway—

Oslofjord (NOF1) and Norway—Continental Shelf (NCS1), suspended in the water column in Brest Commercial Harbor (BCH1), and towed by a surface ship in Kauai (KAU1 and KAU2). The receiver is a bottom- mounted hydrophone (NOF1, NCS1) or a vertically suspended hydrophone array (BCH1, KAU1, KAU2). NOF1 and NCS1 are single- input–single-output (SISO), whereas BCH1, KAU1, and KAU2 are single-input–multiple-output (SIMO). Both types of channels are rep- resented by a set of channel files. A channel file contains a single- hydrophone TVIR measurement, typically with a duration of 30–60 s.

A set of channel files represents consecutive soundings in the SISO case, and a simultaneous sounding on consecutive hydrophones in the SIMO case.

The frequency band and sounding duration in Table I are properties of the employed channel probe signal, but they also give the nominal band and the maximum signal duration for Watermark simulations, as explained in Section IV-B. Probing waveforms are cyclic transmissions of pseudonoise (m-sequence) or linear frequency modulated (LFM) pulses. Both signal types have a (root) raised-cosine spectrum [22, pp. 607–608] with a small rolloff factor, which yields a flat spectrum over a large part of the given band. The importance of a flat probe signal spectrum is explained in Section IV-B.

The delay coverage is the pulse repetition period of the channel sounder, and the Doppler coverage is its reciprocal. These numbers give the maximum possible delay spread and Doppler spread in the replay channel. Energy in the delay-Doppler spread function of the true channel that is outside these limits is aliased in the channel estimate [21], [23]. Aliasing is often unavoidable in acoustic channels probed

Fig. 1. Sketch of the deployment setups.

with cyclic sounders, but the measurement error is small when the aliased energy is a small fraction of the total energy. The channels in Table I are well behaved and offer realistic replay.

Sections II-A–E characterize the test channels with delay-Doppler spread functions, i.e., 2-D power densities that give the distribution of received signal power versus time delayτ and frequency shiftυ. The figures zoom in on the main multipath arrivals and the mean Doppler shift has been removed. More comprehensive channel analyses are shown in the user manual [16].

A. Norway—Oslofjord

NOF1 is a channel measured in a shallow stretch of Oslofjorden between a stationary source and a stationary single-hydrophone receiver. The data consist of a TVIR measurement over 32.9 s, repeated 60 times at 400-s intervals, yielding a total play time of≈ 33 min. Note that the play time is discontinuous. For instance, a 5-s

(3)

Fig. 2. Delay-Doppler spread function of the first NOF1 sounding.

communication packet fits six times in a single TVIR (see Section V-C) and yields 60×6=360 independent packets in simulation. The first six packets are transmitted betweent0 andt0+ 32.9s, packets 7–12 betweent0+ 400s andt0 + 432.9s, etc.

Fig. 2 characterizes the channel, where zero delay is arbitrarily placed just before the first arrival. This arrival has no frequency spread, whereas later arrivals are Doppler spread due to sea surface interac- tions. There is also a streak of delayed clutter atυ=0, due to stationary scatterers. Despite the scattering, most energy is concentrated in a narrow delay-Doppler window (betweenτ =3 ms andτ =5 ms, at υ=0 Hz). NOF1 is a relatively benign communication channel.

B. Norway—Continental Shelf

NCS1 was measured on Norway’s continental shelf between a stationary source and a stationary single-hydrophone receiver.

The data consist of a TVIR measurement over 32.6 s, repeated 60 times at 600-s intervals, yielding a total play time of≈33 min. This play time is discontinuous. For instance, a 5-s communication packet fits six times in a single TVIR and yields 60×6=360 independent packets in simulation. The first six packets are transmitted betweent0

andt0+ 32.6s, packets 7–12 betweent0+ 600s andt0+ 632.6s, etc.

Fig. 3 characterizes the channel, which has in common with NOF1 that most energy is concentrated at the start of the impulse response.

Otherwise it is a completely different channel. There are no stable paths, and the coherence time is of order 100 ms [16]. NCS1 is more challenging than NOF1, in particular for coherent communication schemes, which need to estimate the phase of a signal.

Challenging channels are difficult to measure, and NCS1 has the largest measurement errors among the Watermark test channels. The true channel has long diffuse tails in delay and in Doppler, which do not suddenly stop at the boundaries of the measurement box in Fig. 3.

These tails are aliased in the measurement. However, the fraction of

Fig. 3. Delay-Doppler spread function of the first NCS1 sounding.

the total energy that is aliased is not necessarily large. A validation has been performed to assess the impact of aliasing and other measurement errors for NCS1, concluding that Watermark simulation with NCS1 yields realistic communication performance [12]. At any rate, a channel library with only perfect measurements would imply a library with only benign channels. Some measurement errors have to be tolerated to give the community a taste of less forgiving channels.

C. Brest Commercial Harbor

BCH1 is a SIMO channel measured in the commercial harbor of Brest, France. A source and a four-element array were lowered into the water column from two docks [13]. A single probe transmission over a range of 800 m resulted in a TVIR measurement over 59.4 s, simultaneously recorded on the four hydrophones. Fig. 4 characterizes the channel. Similarly to NOF1, the channel is a mixture of stable and fluctuating arrivals, but with a larger number of distinct trailing paths.

The spatial correlation is relatively high. For instance, for the two main arrivals (τ =1.8 ms andτ =7 ms) the Pearson correlation coef- ficient across the array ranges from 0.5 to 0.9. Although correlated, the SIMO channel shows some spatial selectivity, e.g., a≈9-dB difference in propagation loss between hydrophone channels 1 and 3.

D. Kauai 1

KAU1 is a SIMO channel from the Kauai Acomms MURI 2011 (KAM11) experiment. This experiment was conducted in shallow water off the western side of Kauai, HI, USA [14], [15]. The channel was probed between 4 and 8 kHz, between a towed source and a vertically suspended array with 16 receivers (one of which was broken). The play time is 33 s. Fig. 5 characterizes the channel for a hydrophone near the center of the array. Arrivals clusters are observed, whose Doppler spread increases with an increasing delay. This is caused by an

(4)

Fig. 4. Delay-Doppler spread function of BCH1, hydrophone channel 2.

increasing number of surface bounces occurring at increasing grazing angles.

In addition to Doppler spreading due to variability of the medium, there is a time-varying Doppler shift (TVD) due to tow ship motion.

This also affects the direct path and explains its Doppler spread. This can be verified by resampling the received probe signal with a time- varying resampling factor derived from the phase of the direct path [7], [23]. The resulting channel is shown in Fig. 6, revealing elimination of the Doppler spread from the first arrival. Although difficult to ob- serve, the resampling also reduces the Doppler spread on the trailing clusters. In KAU1, communication schemes may benefit from adaptive resampling or a phase-locked loop (PLL) [24], assisting the receiver in dealing with TVD. Note that TVD removal is not included in Wa- termark; it is applied separately here to illustrate a property of the channel.

E. Kauai 2

KAU2 is another SIMO channel from the KAM11 experiment. It concerns the same probe signal transmission as KAU1, but recorded on a different array. The main difference with KAU1 is a longer range, which attenuates delayed arrival clusters more rapidly: Fig. 7. TVD contributes significantly to the total Doppler spread in this channel, similarly to KAU1.

IV. TECHNICALREMARKS

A. Propagation Loss and Time Synchronization

Watermark is not calibrated for propagation loss, but propagation loss differences between consecutive soundings (NOF1, NCS1) or consecutive hydrophones (BCH1, KAU1, KAU2) are preserved. This means that the signal level of simulated waveforms may fluctuate in time and depth. Watermark also preserves time synchronization of

Fig. 5. Delay-Doppler spread function of KAU1, hydrophone channel 8.

signals received on the vertical arrays. This means, for example, that beamforming Watermark data gives correct results. Whether the correct results are desirable results is a different question, as the element spacing is much larger than the acoustic wavelength in the three SIMO channels.

B. Transfer Function

Input signals have to comply with the frequency bands listed in Table I. Watermark does not use explicit filters, but the channel files contain an implicit bandpass filter. Correlative sounders yield ultrawideband channel estimatesˆh(t, τ)[25], where the transfer functions of the channel, the sounder hardware, and the probe signal are superim- posed on one another. The composite transfer function is reproduced in direct replay. For this reason, the probe signals of Table I have a near rectangular spectrum. In the flat region of the probe signal spectrum (e.g., 10.5–17.5 kHz for NOF1 and NCS1), the spectral distortion of received signals is not affected by probing waveform characteristics, but entirely due to frequency-dependent propagation loss and hardware effects.

The probe signal spectrum, and thus the composite transfer function, falls off rapidly outside the nominal band (10–18 kHz in the case of NOF1 and NCS1). Excess bandwidth of user waveforms will be strongly attenuated. Communication schemes using a different band than the nominal band of a test channel should be adapted, because Watermark will not adapt to these schemes.

The frequency response is revealed by passing white noise over a test channel. Fig. 8 illustrates this for an NCS1 sounding. The top panel shows a spectrogram of the received probe signal, and the middle panel that of the input signalx(t)[see (1)].¹ A spectrogram of the

1Note that the noise is the input signalx(t)in this case, not the additive term n(t).

(5)

Fig. 6. Delay-Doppler spread function of KAU1, hydrophone channel 8, after TVD removal.

Fig. 7. Delay-Doppler spread function of KAU2, hydrophone channel 8.

output signaly(t)is shown in the bottom panel. It strongly resembles that ofx(t), as it should, and shows that simulation is only possible where there is signal energy in the probe signal spectrogram. This makes sense, as it is impossible to reproduce something which has not been measured. The main difference between the spectrograms is the weak reverberation tail, seen at the end of the probe signal, which

is not reproduced in simulation. This energy is aliased in the channel estimate.²

C. Doppler Effects

The mean Doppler shift V0 is removed from the sounding data before the channel estimation. This is done by resampling the raw acoustic data by a resampling factor 1−V0/c, wherec=1500 m/s is the nominal sound speed. The used sign convention is such that a positive velocity corresponds to a positive range rate, i.e., time dilation of the signal. This Doppler shift is missing from the estimates ˆh(t, τ) in the channel library, and from Figs. 2–7 for that matter.

The reason for removing the mean Doppler shift is that this mini- mizes the measurement errors of the channel sounder. Instantaneous Doppler spreading and TVD shifts around the mean value are reproduced in direct replay, whereas the mean shiftV0 is reinstalled when Watermark retrieves the simulated packets. These packets are resam- pled by a factor1/(1−V0/c). This ensures that all Doppler effects experienced by the channel probe signal in the real channel are trans- ferred onto the input signalx(t)in the simulated channel. The Doppler spread in a replay channel may be slightly reduced compared with the original channel [11].

Watermark does not discriminate between true Doppler shifts due to platform motion, and apparent Doppler shifts due to clock frequency offsets (CFOs). Both effects affect the duration of received acoustic signals. A CFO is a mismatch between the nominal (assumed) and the true sample rate of a device. There can be an offset in both sender and receiver, resulting in an apparent Doppler shiftV0 whose magni- tude depends on the quality of the clocks. Offsets corresponding to an apparent velocity of several centimeters per second are not uncommon.

Watermark reproduces the at-sea conditions, including hardware effects, and hence simulated packets can be subject to a small Doppler shift even between bottom-mounted senders and receivers. Communi- cation systems need to deal with such shifts in simulation, just like they need to deal with such shifts in subsea applications.

V. PRINCIPLES OFOPERATION

This section describes the operation of Watermark, omitting detailed instructions concerning file formats, function calls, etc. These are documented in the user manual [16].

A. System Requirements

Watermark runs in MATLAB and works under Windows, Linux, and Mac OS. It uses the Signal Processing toolbox and requires release R2012b or more recent. The reinsertion of small Doppler shifts (Section IV-C) requires systems with sufficient random-access mem- ory. Contemporary work stations should be fine for the channels in Table I.

B. Input Signal

The first step is preparation of a passband waveform, typically a communication packet, for transmission over a given test channel. The packet should fit into a channel estimate in both time and frequency (see Table I). Packets longer than the measured TVIR are refused and frequency contents outside the nominal band is strongly attenuated in

2Actually Watermark does not allow input signals whose length exceed that of the probe signal; the parts ofy(t)beforet=4 s and aftert=37 s have not been simulated.

(6)

Fig. 8. Spectrograms of a recorded probe signal (top), white Gaussian noise as the inputx(t)to Watermark (middle), and the output signaly(t)(bottom).

simulation. Users also need to supply the number of bits in the packet, defined as the number of physical-layer information bits available to higher layers in the protocol stack. It does not include overhead due to synchronization, training, physical-layer header, and error-correcting codes. But if a communication protocol stack is considered, it does include overhead due to layers above the physical layer.

C. Channel Replay

Watermark stacks multiple copies of the input signal packet, as many as fit into the TVIR. It uses a spacing between successive packets to avoid reverberation from one packet spilling into the next one. Since the replay channel has a measurement-imposed upper limit on the delay spread, the required spacing is precisely known. The packet train is convolved with the TVIR using direct replay.

The number of packets returned by Watermark for a single TVIR depends on the input signal durationTs, the play timeTplay, the pulse repetition time of the channel sounderTrep, and a safety marginΔ = 4 ms, which is used accommodate possible sidelobes occurring in the filtering process

N=Tplay/(Ts+Trep+ Δ). (2)

For instance, a communication packet with length 2 s would fit 16 times in a KAU2 sounding with play time 32.9 s, but with the required spacing ofTrep+ Δ =132 ms it fits only 15 times: Fig. 9. For the total number of packets returned by Watermark,Nhas to be multiplied by the number of sounding cycles (SISO channels) or the number of hydrophones (SIMO channels). However, the number of SIMO packets available for array processing is justN.

(7)

Fig. 9. Spectrogram of a filtered packet train in KAU2 (hydrophone 8) for a signal with a length of 2 s.

D. Retrieving Packets

Filtered signal trains (such as in Fig. 9) are stored on disk by Water- mark. Functions are available to retrieve simulated packets for SISO and SIMO applications. SISO packets can be retrieved with additive white Gaussian noise (AWGN) at a specifiedE_b/N0value, whereE_b is the signal energy per information bit andN0the noise power spectral density. The noisy waveform is≈10 s longer than the actual packet, which arrives with a random delay after 4–6 s. Detection and synchronization are essential tasks of modems and modulation schemes, and Watermark aims to enable a realistic comparison of autonomous receivers.

The characteristics of ambient noise in the oceans vary greatly with environment and conditions [5], [26]–[28]. Gaussian statistics may oc- cur in regions where the noise is dominated by sea surface agitation, but it will generally be colored. The reason for using AWGN in Wa- termark is that this permits the use of the unambiguous SNR per bit (Eb/N0) metric. SISO packets can also be retrieved without noise, giv- ing users freedom to apply different noise models, or measured noise, if available. SIMO packets can only be retrieved without noise, because Watermark version 1 lacks a noise model with realistic spatial correlation properties.

Note that retrieving multiple SISO packets at a fixed SNR is not equivalent to at-sea signaling with a constant source level, because both signal level and noise level may vary in the field. Signal level fluctuations are reproduced by Watermark, which simply scales the noise accordingly to realize the target SNR for each packet.

E. Reporting Results

When Watermark is used to demonstrate the performance of a modulation scheme, it is good practice to produce an unambiguous report that allows other people to compare their systems in a meaningful way.

There is a lack of standard terminology in underwater acoustic communications, causing people to use different definitions for seemingly trivial quantities such as SNR, bandwidth, and bit rate. It is recommended to report the following, in addition to whatever one would like to share about the proposed algorithms:

1) the name of the test channel and the Watermark version;

2) the number of information bits in the packet;

3) the effective bit rate reported by Watermark;

4) the signal bandwidth;

5) the total number of packets in the test channel;

TABLE II SIGNALPARAMETERS

QPSK DSSS

Carrier frequency 14 kHz 14 kHz

−3-dB bandwidthB 5.6 kHz 5.6 kHz

# information bits 256 256 Effective bit rateReff 1222 b/s 180 b/s

# packets in NOF1 5760 1260

# packets in NCS1 7920 1320

6) the type of noise in simulation (e.g., the default AWGN or something else);

7) whether communication results are averaged over all packets or a specific subset. Averaging types other than the arithmetic mean should be stated;

8) used definitions of bandwidth, input SNR, output SNR, etc., as appropriate;

9) whether preexisting knowledge is used about signal start or Doppler shift, or whether the receiver has to figure out every- thing itself;

10) to what extent signal and receiver parameters are tuned to the test channel under consideration;

11) anything else that helps to reproduce or appreciate the results.

VI. EXAMPLES

This section provides three example Watermark applications. The objective is to illustrate the use of Watermark, not to present novel modulation schemes or state-of-the-art performances. Descriptions of the schemes are therefore kept short. The modulation format is essen- tially given by the direct-sequence spread spectrum (DSSS) scheme described in [15], including the nonspread case of a spreading code with length 1. Modifications to this format include fitting into the frequency bands of the test channels, and varying preamble lengths and message sizes.

A. Comparing Schemes in NOF1 and NCS1

Table II specifies signals of two modulation schemes, which are tested and compared in NOF1 and NCS1. The first signal is a single- carrier waveform with a quadrature phase-shift keyed (QPSK) symbol constellation. The second signal is DSSS, which is also single-carrier

(8)

Fig. 10. PER in NOF1 and NCS1 versusEb/N0.

QPSK, but with a 15-chip spreading code for enhanced robustness.

Both schemes employ a rate-1/2 convolutional code. Root raised-cosine pulses are used with a rolloff factor of 1/3 at a rate of 5600 s⁻¹.

Both signals carry a message of 256 information bits. They are equipped with a detection preamble and training symbols, which are a considerable overhead at this short message length. The effective bit rates are thus relatively low. The QPSK packet lasts only 0.21 s, which results in thousands of independent packets in NOF1 and NCS1.

The receivers use a filter matched to a bank of Doppler-shifted preamble replicas (with velocities spanning the range from−5 to+5m/s) for detection, synchronization and Doppler estimation. The knowledge that NOF1 and NCS1 are channels between fixed stations is not used.

The chosen detection threshold results in about 1 false alarm per day in AWGN. Demodulation is performed with adaptive equalization. The SISO packet retrieval function is called from a receiver batch job cycling through a range ofEb/N0 values. At each SNR, all available packets are retrieved with AWGN and a random delay, and fed to the receiver.

The packet error ratio (PER) is considered as the performance metric for this investigation. A packet error occurs when one or more bits are in error at the decoder output, or when the packet is not detected. The outcome of the simulations is shown in Fig. 10, leading to several observations.

1) NCS1 is a more challenging channel than NOF.

2) DSSS is more robust than QPSK.

3) DSSS works well in both channels, but requires a 3–6-dB higher SNR in NCS1 to achieve the same performance as in NOF1.

4) The performance of QPSK in NCS1 is severely limited by delay- Doppler spread.

Results can also be examined versus the SNR metric that is more common in underwater acoustics (ratio of signal power to noise power, measured in the frequency band of the signal) with the conversion SNR = (R_eff/B)×(Eb/N0). This increases the gap between DSSS and QPSK, and confirms the notion that data communication at low SNR requires low-rate signaling.

Fig. 11. Receiver output SNR and BER, averaged over 21 (18) packets in KAU1.

The observation that a spreading code increases the robustness to channel distortions and noise is not surprising, but without realistic simulations it would be hard to quantify the improvements. A high-rate coherent scheme that delivers in NCS1 emerges as a challenge.

B. Understanding System Performance

The QPSK scheme of the Section VI-A is adapted to KAU1 by lowering the carrier frequency to 6 kHz, and by lowering the symbol rate to 2800 s⁻¹, which reduces the−3 dB bandwidth to 2.8 kHz. The number of information bits is increased to 2048. The signal length becomes 1.4 s, and the effective data rate 1460 b/s.

Watermark returns 21 SIMO packets, which are fed to an adaptive multichannel equalizer in AWGN. The noise is spatially uncorrelated, and is added at a high (30-dB acoustic) SNR. The detector runs on the top hydrophone, whereas the receiver is applied to the top hydrophone, the top two hydrophones, the top four hydrophones, etc. One hydrophone was broken during the data collection, so the maximum number is 15.

Detection is performed with a Doppler-bank matched filter, cycling through velocities from−5 to+5 m/s in steps of 0.1 m/s. The filter is matched to the signal preamble, which is an m-sequence with a length of 255 chips.

The receiver output SNR is computed as one over the mean square symbol error, after removal of equalizer bias. The blue curve in Fig. 11 shows the output SNR and bit error ratio (BER), averaged over the 21 Watermark packets. The mean BER is 0.44 for the top hydrophone.

Adding elements improves the BER, but it does not drop to zero.

Inspection of individual packets reveals that there are two packets with high BER, which do not improve at all upon adding elements. The cause is revealed in Fig. 12, which shows the detector Doppler estimate. A TVD estimate extracted from the known channel is included as the ground truth. The two BER outliers concern the two packets with the largest Doppler mismatch.

(9)

Fig. 12. Preamble Doppler estimates, and BER for 15 elements. The black curve is the ground truth extracted fromˆh(t, τ).

The receiver removes the detector Doppler estimate by resampling, and leaves the residual Doppler to the equalizer. It appears that a mismatch of≈0.4 m/s is too large a burden for the equalizer, whose taps cannot keep track with the phase rotation of symbols.

Two strategies are explored to improve the performance. The first alternative is to improve the Doppler resolution by increasing the preamble length to 1023 chips. This requires rerunning Watermark, and yields zero BER for all packets (see Figs. 11 and 12). A disadvantage is that the effective bit rate drops from 1460 to 1220 b/s, and that the complexity of the detector increases. The number of Watermark packets also drops, from 21 to 18. The second alternative is to equip the receiver with a PLL [24]. This low-complexity device deals explicitly with the phase rotation, and allows the equalizer to concentrate on path ampli- tude fluctuations and differential Doppler. Fig. 11 shows that alternative 2 outperforms alternative 1. However, note that a long preamble would be required anyway for detection in the low-SNR regime. The two alternatives can also be combined, yielding a minor further improvement in the present high-SNR scenario (not shown).

C. Preamble Detection in BCH1 and KAU2

In Section VI-B, Watermark was used to examine the effect of preamble Doppler resolution. In this section, the benchmark is used to examine preamble detection statistics. Communication schemes deployed in different environments may require different decision statistics for optimal operation.

In underwater acoustic scenarios, the noise variance as well as the Doppler shift and the arrival time of preambles are unknown at reception. In this context, packet detection classically relies on a bank of constant false alarm rate matched filters [29, ch. 4] made of Doppler-shifted preamble replicas. The decision statistic is computed by combining the filter outputs. Several combination alternatives can be chosen, and, depending upon the channel properties, they may behave differently, so compromises must be found.

For single-hydrophone detection, a standard approach is to compare, at each instant, the maximum value of the matched filters output to

Fig. 13. Correct detection rate versus SNR for single-hydrophone detection in BCH1 (channel #2) and KAU2 (channel #8): Impact of time delay averaging.

a threshold (the maximum being taken over all Doppler branches).

This approach, which considers each sample individually, implicitly assumes that the channel contains a single dominant path, which is not always the case as shown in Figs. 5 and 7. Another approach, more robust to multipath, is to consider the contribution of the different paths by averaging each matched filter output over a given time delay window [30, ch. 5.7]. The decision is then made by comparing the maximum value of these averages to a threshold. The optimal averaging duration requires a priori knowledge of the channel power delay profile, which is not available in practice. Therefore, only empirical choices can be made.

When several hydrophones are available at reception, the single- hydrophone statistics can be added up across the array to make a decision. However, if the space diversity offered by the array is known to be deficient because of correlation between impulse responses, other strategies may provide better performance. For instance, the space-time matrix obtained by stacking matched filter outputs can have a rank lower than the dimension of the array. If available, this prior information can be used to apply a multichannel detector of signals having a known rank [31], [32].

For the tests, the DSSS scheme of Section VI-A is adapted to BCH1 and KAU2 by changing the carrier frequency to 35 and 6 kHz, respectively, and by lowering the chip rate to 2800 s⁻¹. A 511-chip m-sequence is used as a preamble and each packet carries 256 information bits. The packet duration is 2.925 s so that the effective bit rate becomes 87.5 b/s. Watermark returns 19 SIMO packets for BCH1 and ten for KAU2, which are then fed to a preamble detector. Spatially uncorrelated WGN is added to the SIMO packets, where the SNR is defined as the ratio of average received signal power (across time and space) to noise power, measured in the−3-dB frequency band of the signal (2.8 kHz). The noise variance as well as the Doppler shift and the arrival time are assumed unknown at reception. The bank of Doppler- shifted replicas spans the velocity range from−5 to+5 m/s in 0.1-m/s steps. The detection thresholds are set to get one false alarm per day on average in AWGN. The results are averaged over all Watermark packets and 500 noise realizations are generated for each packet.

(10)

Fig. 14. SNR gain provided by averaging the matched-filters output of BCH1 (channel #2) and KAU2 (channel #8) over time delay. This gain is measured for a correct detection rate of 90%.

Fig. 13 illustrates the impact of time delay averaging on the correct detection rate for single-hydrophone detection in BCH1 (channel #2) and KAU2 (channel #8). While a 10-ms averaging window does not provide any performance improvement in BCH1, it is highly beneficial for KAU2. The reason is that BCH1 contains one dominant path, whereas KAU has several significant arrivals with a spread in arrival times. The optimal averaging duration for both channels can be deduced from Fig. 14. This figure shows, for a correct detection rate set to 90%, the SNR gain obtained for several averaging durations compared with the case without time-delay averaging. For BCH1, this optimal value is less than 1 ms and averaging over more than 10 ms can be worse than not averaging at all. KAU2 is less sensitive to the averaging duration once it is greater than 5 ms. This is explained by the slower decay of the power delay profile of KAU2 compared with BCH1.

Fig. 15 shows the performance of multichannel detection for BCH1 with four hydrophones and KAU2 with 16 hydrophones, using 100 noise realizations at each SNR value. KAU2 outperforms BCH1, in contrast to Fig. 13, owing to a higher array gain. Several assumptions are made with respect to the rank of the space-time matrix obtained by aggregating the matched filters outputs of the array. For BCH1, assum- ing a lower rank than the dimension of the array is beneficial for detection. For instance, a 1-dB improvement is achieved by limiting the rank to 2 instead of 4. As discussed in [33], although the hydrophone spacing is greater than 20 wavelengths, the correlation between hydrophones is quite high, which explains this performance improvement. KAU2 has much more spatial diversity than BCH1. Therefore, reducing the rank does not provide any gain.

D. Summary

The above examples illustrated applications of Watermark, and the insight it can provide. A given SISO scheme performed satisfactorily in one channel but not in another. A spreading code was required to en- sure a low PER in the tougher channel, even at high SNR. The cause of a few persistent high-BER packets in a SIMO setup was pinpointed to insufficient preamble Doppler resolution, and the scheme was

Fig. 15. Correct detection rate versus SNR for multiple hydrophone detection in BCH1 (4 channels) and KAU2 (16 channels): Impact of rank limitation.

For the simulation, matched filters outputs are aggregated over a 10-ms delay window.

improved correspondingly. Preamble detection statistics were shown to vary between channels, and different methods to improve detection had different effects in different channels.

Watermark makes it easier to understand communication and detection performance, since the channels are known. Other differences with at-sea signaling are the possibility to improve schemes and retransmit signals over the original channel, and the possibility to send multiple signals in parallel in the same frequency band.

VII. FUTUREADDITIONS TOWATERMARK

Watermark version 1 includes a set of channels with different parameters and characteristics, fulfilling several needs. Needs not covered by the first release require extensions. Possible extensions and additions are as follows.

The library of test channels can be extended by performing suitable soundings, using good acoustic measurements practice [34].

Extensions may concern different frequency bands, environments, channel properties, deployment geometries, etc. It is recommended to use a flat probe signal spectrum, whose frequency band falls in a well- behaved region of the hardware transfer function. Correlative sounders are straightforward to use and have documented measurement errors [21], but other channel estimation algorithms may be used as well.

More important than the type of sounder is the requirement that measurement errors are small. Perhaps the most important requirement to a replay communication channel, is that communication performance in simulation is comparable to the communication performance at sea.

It is therefore recommended to transmit different types of communication signals between sounding signals to enable validations such as in [11]. Even validated acoustic propagation models may be used with Watermark, provided that their output is in the form of a TVIR at a suitably high sampling rate, and in the desired format [16].

More realistic noise models will add to the simulation fidelity in the regime of low SNR. Since the characteristics of ambient noise in the oceans vary widely [5], [26]–[28], different types of noise are needed to design robust communication systems. Acoustic noise models should

(11)

be validated, but it is also possible to use measured noise, with a long measurement time to be able to draw a large number of independent realizations.

End-to-end calibration of sounding equipment and noise recorders will enable calibrated channel simulation resulting in simulated signals and noise at sound pressure levels corresponding to their ocean values.

This could be used, for instance, to find the minimum source level that delivers adequate system performance in a given setup.

VIII. CONCLUSION

A realistic benchmark is now available for underwater acoustic communications. It is based on a replay channel simulator driven by measurements of the TVIR. Its initial library of five test channels comprises four geographical areas and three frequency bands. Three of these test channels offer reception on a vertical line array. The primary use of Watermark is benchmarking of the physical layer of acoustic communication systems, but in principle it can also be used for different sonar applications involving the one-way transmission of sound.

Watermark brings the realism of at-sea signaling into the office while adding reproducibility. Applications include the following:

1) test and improve detection and communication algorithms;

2) compare different modulation schemes in the same channel;

3) examine the performance of a given scheme in different channels;

4) document the performance of modulation schemes and the effect of parameter settings, both for in-house evaluations and scientific publications;

5) the quest for a robust high-rate system;

6) extract error statistics for network simulations;

7) study channel characteristics and signal fluctuations.

The benchmark can be extended with channels from different environments and frequency bands, either for own use or for general distribution, depending on the willingness of third parties to perform suitable measurements and share the data.

ACKNOWLEDGMENT

An initial version of this benchmark was developed under the Eu- ropean research project “Robust Acoustic Communications in Under- water Networks” (RACUN) with partners from Germany, Italy, The Netherlands, Norway, and Sweden. The authors would like to thank Kongsberg Maritime for their support with the channel soundings, and the Scripps Institution of Oceanography for approving distribution of KAM11 channel estimates with the benchmark.

REFERENCES

[1] J. Preisig, “Acoustic propagation considerations for underwater acoustic communications network development,” ACM SIGMOBILE Mobile Com- put. Commun. Rev., vol. 11, no. 4, pp. 2–10, Oct. 2007.

[2] P. A. van Walree, “Propagation and scattering effects in underwater acoustic communication channels,” IEEE J. Ocean. Eng., vol. 38, no. 4, pp. 614–631, Oct. 2013.

[3] A. Zielinksi, Y.-H. Yoon, and L. Wu, “Performance analysis of digital acoustic communication in a shallow water channel,” IEEE J. Ocean.

Eng., vol. 20, no. 4, pp. 293–299, Oct. 1995.

[4] C. Bjerrum-Niese and R. L¨utzen, “Stochastic simulation of acoustic com- munication in turbulent shallow water,” IEEE J. Ocean. Eng., vol. 25, no. 4, pp. 523–532, Oct. 2000.

[5] M. Chitre, “A high-frequency warm shallow water acoustic communica- tions channel model and measurements,” J. Acoust. Soc. Amer., vol. 122, no. 5, pp. 2580–2586, Nov. 2007.

[6] M. Siderius, M. B. Porter, P. Hursky, V. McDonald, and the KauaiEx Group, “Effects of ocean thermocline variability on noncoherent under- water acoustic communications,” J. Acoust. Soc. Amer., vol. 121, no. 4, pp. 1895–1908, Apr. 2007.

[7] P. A. van Walree, T. Jenserud, and M. Smedsrud, “A discrete-time channel simulator driven by measured scattering functions,” IEEE J. Sel. Areas Commun., vol. 26, no. 9, pp. 1628–1637, Dec. 2008.

[8] F.-X. Socheleau, C. Laot, and J.-M. Passerieux, “Stochastic replay of non- WSSUS underwater acoustic communication channels recorded at sea,”

IEEE Trans. Signal Process., vol. 59, no. 10, pp. 4838–4849, Oct. 2011.

[9] H. S. Dol, M. E. G. D. Colin, M. A. Ainslie, P. A. van Wal- ree, and J. Janmaat, “Simulation of an underwater acoustic communication channel characterized by wind-generated surface waves and bubbles,” IEEE J. Ocean. Eng., vol. 38, no. 4, pp. 642–654, Oct. 2013.

[10] P. Qarabaqi and M. Stojanovic, “Statistical characterization and compu- tationally efficient modeling of a class of underwater acoustic communi- cation channels,” IEEE J. Ocean. Eng., vol. 38, no. 4, pp. 701–717, Oct.

2013.

[11] R. Otnes, P. A. van Walree, and T. Jenserud, “Validation of replay-based underwater acoustic communication channel simulation,” IEEE J. Ocean.

Eng., vol. 38, no. 4, pp. 689–700, Oct. 2013.

[12] P. van Walree, R. Otnes, and T. Jenserud, “Watermark: A realistic bench- mark for underwater acoustic modems,” in Proc. IEEE 3rd Underwater Commun. Netw. Conf., Aug. 2016, pp. 1–4.

[13] F.-X. Socheleau, A. Pottier, and C. Laot, “Watermark: bch1 dataset de- scription,” Institut Mines-Telecom, TELECOM Bretagne, UMR CNRS 6285 Lab-STICC, Brest, France, Res. Rep. 17331, hal-01404491, 2016.

[14] W. S. Hodgkiss and J. C. Preisig, “Kauai Acomms MURI 2011 (KAM11) experiment,” in Proc. Eur. Conf. Underwater Acoust., Edinburgh, U.K., Jul. 2012, pp. 993–1000.

[15] R. Otnes, P. A. van Walree, H. Buen, and H. Song, “Underwater acoustic network simulation with lookup tables from physical-layer replay,” IEEE J. Ocean. Eng., vol. 40, no. 4, pp. 822–840, Oct. 2015.

[16] P. van Walree, R. Otnes, and T. Jenserud, “The Watermark manual and user’s guide,” Forsvarets Forskningsinstitutt, Horten, Norway, FFI-rapport 2016/01378, 2016.

[17] “The Watermark acoustic modem benchmark. 2016. ” [Online]. Available:

www.ffi.no/watermark

[18] Y. Isukapalli, H. C. Song, and W. S. Hodgkiss, “Stochastic channel sim- ulator based on local scattering function,” JASA Express Lett., vol. 130, no. 4, pp. EL200–EL205, Oct. 2011.

[19] F.-X. Socheleau, C. Laot, and J.-M. Passerieux, “Parametric replay-based simulation of underwater acoustic communication channels,” IEEE J.

Ocean. Eng., vol. 40, no. 4, pp. 796–806, Oct. 2015.

[20] C. Petrioli, R. Petroccia, J. R. Potter, and D. Spaccini, “The SUN- SET framework for simulation, emulation and at-sea testing of under- water wireless sensor networks,” Ad Hoc Netw., vol. 34, pp. 224–238, 2015.

[21] G. Matz, A. F. Molisch, F. Hlawatsch, M. Steinbauer, and I. Gaspard, “On the systematic measurement errors of correlative mobile radio channel sounders,” IEEE Trans. Commun., vol. 50, no. 5, pp. 808–821, May 2002.

[22] J. G. Proakis and M. Salehi, Digital Communications, 5th ed. New York, NY, USA: McGraw-Hill, 2008.

[23] P. van Walree, “Channel sounding for acoustic communications: Tech- niques and shallow-water examples,” Forsvarets Forskningsinstitutt, Horten, Norway, FFI-rapport 2011/00007, 2011.

[24] M. Stojanovic, J. A. Catipovic, and J. G. Proakis, “Phase-coherent digital communications for underwater acoustic channels,” IEEE J. Ocean. Eng., vol. 19, no. 1, pp. 100–111, Jan. 1994.

[25] P. A. van Walree and R. Otnes, “Ultrawideband underwater acoustic com- munication channels,” IEEE J. Ocean. Eng., vol. 38, no. 4, pp. 678–688, Oct. 2013.

[26] G. M. Wenz, “Acoustic ambient noise in the ocean: Spectra and sources,”

J. Acoust. Soc. Amer., vol. 34, no. 12, pp. 1936–1956, Dec. 1962.

[27] P. H. Dahl, J. H. Miller, D. H. Cato, and R. K. Andrew, “Underwater ambient noise,” Acoust. Today, vol. 3, no. 1, pp. 23–33, Jan. 2007.

[28] G. B. Deane and J. C. Preisig, “Very high frequency noise sources in the littoral zone,” in Proc. IEEE 3rd Underwater Commun. Netw. Conf., Aug. 2016, pp. 1–4.

[29] L. L. Scharf, Statistical Signal Processing: Detection, Estimation, and Time Series Analysis. Reading, MA, USA: Addison-Wesley, 1991, pp. 1–524.

[30] S. M. Kay, Fundamentals of Statistical Signal Processing, Detection The- ory, vol. 2, Upper Saddle River, NJ, USA: Prentice-Hall, pp. 1-560, 2009.

[31] O. Besson, S. Kraut, and L. L. Scharf, “Detection of an unknown rank-one component in white noise,” IEEE Trans. Signal Process., vol. 54, no. 7, pp. 2835–2839, Jul. 2006.

[32] S. Sirianunpiboon, S. D. Howard, and D. Cochran, “Multiple-channel detection of signals having known rank,” in Proc. IEEE Int. Conf. Acoust.

Speech Signal Process., May 2013, pp. 6536–6540.

(12)

[33] F. X. Socheleau, A. Pottier, and C. Laot, “Stochastic replay of SIMO under- water acoustic communication channels,” in Proc. MTS/IEEE OCEANS Washington, Oct. 2015, pp. 1–6.

[34] J. R. Potter, M. B. Porter, and J. C. Preisig, “UComms: A conference and workshop on underwater communications, channel modeling, and validation,” IEEE J. Ocean. Eng., vol. 38, no. 4, pp. 603–613, Oct. 2013.

Paul A. van Walree (M’08) received the M.Sc. de- gree in high-frequency phonons and the Ph.D. degree in stimulated emission of phonons from Utrecht Uni- versity, Utrecht, The Netherlands, in 1992 and 1997, respectively.

From 1998 to 2009, he was an Underwater Acous- tician with The Netherlands Organization for Applied Scientific Research, The Hague, The Netherlands. In 2009, he was a Research Scientist at the Department of Maritime Systems, Norwegian Defence Research Establishment, Horten, Norway. His research interests include acoustic communication, channel characterization and simulation, and miscellaneous topics in underwater acoustics.

François-Xavier Socheleau (S’08–M’12) received the Diplôme d’Ingénieur degree in electrical engineering from ESEO, Angers, France, in 2001 and the Ph.D. degree from Telecom Bretagne, Brest, France, in 2011.

From 2001 to 2004, he was a Research Engineer at Thales Communications, France, where he worked on electronic warfare systems. From 2005 to 2007, he was with Navman Wireless (New Zealand/U.K.) as an R&D Engineer. From 2008 to 2011, he was with Thales Underwater Systems, France. In November 2011, he joined ENSTA Bretagne as an Assistant Professor. Since September 2014, he has been an Associate Professor at IMT Atlantique (formerly known as Telecom Bretagne), Brest, France. His research interests include the field of signal processing and underwater acoustics.

Roald Otnes (S’00–M’03–SM’13) received the M.Sc. and Ph.D. degrees in telecommunications from the Norwegian University of Science and Technol- ogy, Trondheim, Norway, in 1998 and 2003, respectively.

Since 2003, he has been a Scientist at the Nor- wegian Defence Research Establishment, Horten, Norway. His current research interests include underwater acoustic communications and sonar signal processing.

Trond Jenserud received the M.Sc. degree in electri- cal engineering and the Ph.D. degree in telecommunications from the Norwegian Institute of Technology, Trondheim, Norway, in 1980 and 2001, respectively.

He is currently with the Norwegian Defence Re- search Establishment, Horten, Norway. His research interests include wave propagation, signal processing, and underwater communications.