Figure 1.5: The disciplines and theory involved in autonomous sampling of the ocean:
robotics, oceanography (the practice), spatial statistics, and information theory. This work encompasses elements of all of these disciplines, building systems that can reason, plan, and strategize data collection in highly uncertain environments.
Design and implementation have been conducted using the autonomous agent archi-tecture T-REX (McGann et al., 2008b,a; Py et al., 2010) and the on board AUV control system DUNE (Pinto et al., 2012). The numerous sea trials (see Table A.1) have primar-ily been conducted in Svalbard, the Trondheimsfjord, and nearby coastal areas using the Light Autonomous Underwater Vehicle (Sousa et al., 2012) [LAUV Harald] – special-ized for water column sampling. Real-world sensing applications and campaigns are not merely concerned about extending current sampling abilities of AUVs, but also retrieving the data itself. Failure to reconcile the two weakens the overall result, thus the aim must be to change the way we retrieve data for the benefit of the scientific context.
1.4 Thesis Contributions and Outline
The principal contributions of this work are related to i) the design and analysis of information-theoretic approaches in upper water column sampling, coupled with
intel-ligent control and ii) testing and validating these methods in the field. This includes development of greedy data-driven sampling strategies and formulation of compact proxy models for use in autonomous exploration of the water column using a single robotic platform. Details on the contributions per paper are listed in more detail in Ch. 6, Section 6.3, stated below the reference to each paper.
This thesis summarizes and complements a number of publications, and is organized as follows. The first part of the thesis presents an overview of the research with background information that helps to fit the individual papers into the broader context of adaptive sam-pling. There are six chapters whose content is briefly described below. Part II contains the papers, which support the discussion presented in Part I. Included here are four authored and two co-authored papers, of which three are journal papers and three are conference papers. The papers are listed in the same order as they appear below. The scientific contri-bution of each paper is given in Ch. 6 together with a summary and future work.
Part I
Chapter 2gives an overview of sampling in the oceans and the space-time variability of the interacting processes before an overview over synoptic marine data sources is given, focusing on remote sensing and numerical ocean models. Chapter 3 provides a brief introduction to spatial statistics and Gaussian processes. Applications and examples are given from the standpoint of modeling and inference in the ocean environment. Building on the basis from the preceding chapters,Chapter 4discusses information-theoretic and behavior-based adaptive sampling in detail, including theory, related work, applications, as well as some concrete examples of data-driven autonomous agents. The theory and concepts are further discussed in an operational context inChapter 5, which provides a discussion about marine robotic platforms and practical aspects related to adaptive sampling, focusing on operational issues and deployment with AUVs in the upper water column. Finally,Chapter 6summarizes the thesis and specifies the scientific contribu-tions before a discussion on potential future research direccontribu-tions. Lastly, Appendix A provides an overview of the field deployments undertaken during the PhD work.
Part II
The papers included in this section cover different aspects of autonomous robotic sam-pling.Papers A-Bpresent two adaptive sampling methods based on Gaussian processes for doing data collection in the water column.Paper Aproposes a greedy adaptive sam-pling algorithm that uses an information-theoretic approach to select and plan samsam-pling efforts. The strategy relies on a Gaussian process model for modeling the environment, formulated on the basis of regional data from an ocean model.Paper Bemploys a similar Gaussian model, but this time for modeling and mapping heterogeneous concentrations of water column parameters. This model is then used for adapting a volumetric AUV survey, targeting regions of interest. Results from field trials are shown, together with corresponding ship-based observations.Paper Cpresents a methodology for leveraging remote sensing data, specifically images of sea-surface temperature, towards developing compact on board models that can be used to inform sampling strategies for marine
1.4. Thesis Contributions and Outline sensing platforms. A case study using data from Monterey Bay and an autonomous surface vehicle is presented, together with statistical validation and analysis of the compact model. Paper D proposes an autonomous agent architecture for inspection, maintenance, and repair applications for ROVs, aided by control and computer vision techniques. Results from field deployment using a full scale integration on board a work class ROV is shown. InPaper E, the greedy and Gaussian framework fromPaper Ais re-applied to an industrial application for tracking and monitoring dispersion dynamics in the water column.Paper Fpresents an approach for autonomous mapping of the seafloor using Hidden Markov Random Fields. Backscatter is used to segment and automatically plan a more detailed camera survey; results from full-scale experiments are given.
List of Included Papers
A: Peer-reviewed Journal Article
Trygve Olav Fossum, Jo Eidsvik, Ingrid Ellingsen, Morten Omholt Alver, Glaucia Moreira Fragoso, Geir Johnsen, Renato Mendes, Martin Ludvigsen, and Kanna Rajan.Information-driven Robotic Sampling in the Coastal Ocean.Journal of Field Robotics, Volume 35, Issue 7, pages 1101–1121, 2018.
B: Peer-reviewed Journal Article
Trygve Olav Fossum, Glaucia Moreira Fragoso, Emlyn J. Davies, Jenny Ullgren, Renato Mendes, Geir Johnsen, Ingrid Ellingsen, Jo Eidsvik, Martin Ludvigsen, and Kanna Rajan. Towards Adaptive Robotic Sampling of Phytoplankton in the Coastal Ocean.Science Robotics, Volume 4, Issue 27, eaav3041, 2019.
C: Peer-reviewed Journal Article
Trygve Olav Fossum, John Ryan, Tapan Mukerji, Jo Eidsvik, Thom Maughan, Martin Ludvigsen and Kanna Rajan.Compact models for Adaptive Sampling in Marine Robotics. Submitted to International Journal of Research Robotics, 9th November 2018.
D: Conference paper
Trygve Olav Fossum, Martin Ludvigsen, Stein M. Nornes, Ida Rist-Christensen and Lars Brusletto.Autonomous Robotic Intervention using ROV: An Experimental Approach.MTS/IEEE OCEANS 2016, Monterey, CA, USA,19th-22th September 2016.
E: Conference paper
Gunhild Elisabeth Berget, Trygve Olav Fossum, Tor Arne Johansen, Jo Eidsvik and Kanna Rajan.Adaptive Sampling of Ocean Processes Using an AUV with a Gaussian Proxy Model.11th IFAC Conference on Control Applications in Marine Systems, Robotics, and Vehicles (CAMS) Opatija, Croatia, 10th-12th September 2018.
F: Conference paper
Øystein Sture, Trygve Olav Fossum, Martin Ludvigsen and Martin Syre Wiig.
Autonomous Optical Survey Based on Unsupervised Segmentation of Acoustic Backscatter.MTS/IEEE OCEANS, Kobe Techno-Oceans (OTO), Kobe, 2018.
The following works are not included in this thesis:
Conference paper
Martin Ludvigsen, Sigurd M. Albrektsen, Krzysztof Cisek, Tor Arne Johansen, Pet-ter Norgren, Roger Skjetne, Artur Zolich, Paulo Sousa Dias, S´ergio Ferreira, Jo˜ao Borges de Sousa,Trygve Olav Fossum, Øystein Sture, Thomas Røbekk Krogstad, Øivind Midtgaard, Vegard Hovstein, and Erlend V˚agsholm.Network of heteroge-neous autonomous vehicles for marine research and management.
In Proc. MTS/IEEE OCEANS, Monterey, CA, USA, 2016.
Technical Report
Øystein Sture, Martin Syre Wiig, and Trygve Olav Fossum. NTNU-FFI Cruise 2017-HUGIN Autonomy Integration (DUNE, T-REX).NTNU Cruise Reports, The Norwegian University of Science and Technology (NTNU).
Technical Report
Trygve Olav Fossum.Intelligent Autonomous Underwater Vehicles: A Review of AUV Autonomy and Data-Driven Sample StrategiesIMT-AURLab-1, Depart-ment of Marine Technology, Centre for autonomous marine operations and systems (AMOS), Norwegian University of Science and Technology (NTNU).
Chapter 2
Ocean Observation
“Most of the previous century could be called a century of under-sampling.”
—Walter Munk, Secretary of the Navy Research Chair in Oceanography (Munk, 2002)
T
HEability to observe the ocean is rapidly improving. The use of high resolution ocean models, remote sensing, and robotic elements has moved oceanographic sensing practices towards a more holistic perspective, where increased presence and information sharing, across a range of different scales, is becoming more feasible (see Fig. 2.1). This chapter takes a closer look at ocean observation and the space-time variability of the inter-acting processes, followed by an overview over synoptic marine data sources focusing on remote sensing and numerical ocean models.2.1 Observing Earth’s Ocean
The study of the ocean covers a multitude of scales and space-time (spatiotemporal) vari-ability, including processes that are episodic (see Fig. 2.2). The primary platform for ob-servation has been – and still is – ships. However, the U.S. federal oceanographic fleet could be reduced to half its size by 2026 as a result of flat budgets and increased costs (Cressey, 2013); a trend that is indicative for the rest of the world. At the same time, trends in science and technology indicate that ship assets are still required (Board et al., 2009, Ch.5) and cannot be completely replaced by new sampling tools. These changes are also reflected in newly developing oceanographic sensing practices, where satellites, robotic elements, ocean models, and ocean sensor networks are increasingly being used as data-gathering tools (Kintisch, 2013). These networked sampling systems are not based on a single platform or observation scale, but rather a complementary ensemble covering a range of scales, building on the principle of sharing information to mutual advantage.
The introduction of remote sensing and large-scale sensor networks have provided a more synoptic perspective of the ocean; however, sensors measurement are still too far apart or cannot resolve the necessary details. The attenuation of radio waves also restrict satellite observation to the very surface. Ocean model accuracy is also not at a level where it can replace actualin-situobservation (Lermusiaux, 2006). Hence, we are still left with a significantly unobserved water column, where it is necessary to combine various individual marine data sources to close the gaps in coverage and resolution. Even
Figure 2.1: Conceptual view of a multi-scale, multi-platform field experiment using: ships, buoys, AUVs, glider, floaters, satellite, and aerial drones. Achieving the ambition of a synoptic understanding of the ocean requires a joint effort between a range of marine data sources.
with numerous deployed instruments, it will still not be conceivable to examine the entire environment in detail, and thus onlyquasi-synoptic(i.e. a semi-holistic recording of an event) coverage is usually possible (Curtin et al., 1993). Observation itself is also not straightforward; sensors are usually only capable of providing proxy measurements of the relevant processes, which means additional uncertainty is introduced. Observations also come at different scales, accuracy, are subject to spatial sampling bias (due to the inherent heterogeneity [patchiness] of the ocean), and cannot be readily transmitted with high bandwidth between sources – making cross-validation and comparison difficult.
Additionally, currents, topography, tides, and turbulent flows constantly move information around, making all observations time dependent. In practice, this means that we are still inclined toundersamplethe environment in both time and space. The term “ground-truth”
is therefore never really fully attainable in ocean sensing, except for very large scale processes (such as tides) or very local processes (such as determining run-off from rivers).
This is the sampling conundrumin oceanography and the lack of sufficient obser-vations is the largest source of error in our understanding of the ocean (Stewart, 2009), makingwhen and whereto sample the key problem when designing oceanographic exper-iments. A guiding rule of thumb provided by the the Nyquist theorem is to sample at least twice in time for the shortest significant record period, and twice in space for the short-est significant length (Nyquist, 1928) to either resolve or eliminate (by filtering) scales of oceanic variability shorter than those being studied. In practice, this means mapping at an adequate spatial resolution faster than significant changes – in the phenomena – occur.
2.1. Observing Earth’s Ocean
Summarizing the above, we can identify the following challenges:
The challenges in ocean sampling (thesampling conundrum):
• Sparseness:It is usually not possible or practical to observe the entire environment in detail both in terms of coverage (space) and resolution (space and time); usually only a quasi-synoptic coverage is possible.
• Space-Time dependent environment:The fundamental turbulent, heterogeneous, and episodic nature of the ocean makes observations time-dependent and sensitive to both location and scale (sampling bias); this also affects the ability to maintain up-to-date knowledge. Understanding and quantifying this influence is difficult.
• Proxy measurements:Sensor observations are rarely able to acquire direct mea-surements of the process or quantity we are interested in, introducing additional uncertainty. Certain instruments also affect the environment themselves (e.g. light and noise) that may cause instrument bias.
• Sensing scales:A multitude of sensors are used to fill observation gaps and to avoid undersampling, making cross-comparisons complex.
• Harsh Environment:Pressure, corrosion, and bio-fouling affect all equipment that goes into the ocean. Logistically, these instruments are expensive and complex to in-stall. Once in place, wave motion, current, and wind subject the observation systems to varying loads and forces.
Addressing the sampling conundrum in the ocean requires understanding of the fundamental environmental characteristics, as well as new technological solutions and sensing practices that enable unification and augmentation of data from a range of sources and scales. From a sampling perspective, the combination of synoptic data sources such as ocean models and remote sensing with robotic platforms will be key, and will thus be covered in more detail in the following sections.
2.1.1 Space and Time Variability in the Ocean
The ocean is fundamentally turbulent. A multitude of oceanic processes interact to create variability in space and time, spanning many orders of magnitude from large scale oscilla-tions exceeding 100 km, down to biogeochemical processes below 1 cm. This dynamical landscape is usually divided into the following scales: i)Mesoscale: 50-500 km, 10-100 daysand ii)Sub-mesoscale: 1-10 km, days-months. Fig. 2.2 depicts some of the prominent oceanic events that occur in this vast dimension.
The energy of mesoscale processes, such as eddies, generally exceeds that of the mean flow by an order of magnitude or more (National Research Council, 2010), having a strong impact on the ecosystem. In operational oceanography, traditional techniques, like ship-board and moored measurements, can be effective at large spatial (O(100 km)) and tempo-ral (O(week to months)) scales, but quickly become too sparse for sub-mesoscale variabil-ity (Graham et al., 2013). The introduction of satellite oceanography has also proved sig-nificant at these scales, capable of providing global coverage for variables such as sea level
Figure 2.2: Some of the prominent oceanic processes and events, shown with their spa-tiotemporal extent. Image credit: (Schofield et al., 2013) and Tom Dunne.
height. Satellites also provide an overlap towards sub-mesoscale dynamics, whose impor-tance is significant and directly influences events such as primary production (L´evy, 2003) or patch formation of biomass (Franks, 1992). For many years this variability was so un-dersampledthat its impact was greatly misunderstood (Munk, 1997). One example is the spatial distribution of phytoplankton. Its intensity, morphology, and scale dependence are substantially driven by sub-mesoscale processes such as turbulent advection, upwelling, and vertical mixing (Mackas et al., 1985; Van Gennip et al., 2016). Local processes (such as upwelling zones) bring deep water nutrients to the surface/photic zone and nurture phy-toplankton, creating regional hot spots (with high biomass concentration) at scales ranging from 5–10 km (Martin et al., 2002) or even≤1 km for complex coastal zones (Hedger et al., 2003). In the open ocean, the same aggregation can range from 70–140 km (spatial correlation) in the horizontal plane; vertically, persistent upper water column stratification may lead to a layered structure with different subsurface maxima, where, for example, phytoplankton is concentrated in the bottom of a pycnocline (a density gradient) (Silsbe and Malkin, 2016). A vital point to note about stratification effects is the consequence that vertical correlation is much weaker compared to the horizontal, where the increase can be as high as factor of 111×for temperature and 800×for Chlorophylla(Sahlin et al., 2014).
This is important to consider when dealing with spatial interpolation and data assimilation, or when formulating proxy models for processes in the water column.