So far in this thesis, the problem of CD in a time series of remote sensing images has been discussed without assuming any relationship between the images themselves. In the following, a clear distinction between the defini-tions of homogeneous and heterogeneous data is set, to show the limitadefini-tions imposed by using the former and the challenges faced when dealing with the latter.
2.3.1 Motivation
When describing the ideal scenario for CD, Campbell et al. [23] refer to the case in which the images are captured by the same or well intercalibrated sensors, at the same time of day, using the same field of view and look angle, and so on. Working under these assumptions assures that spurious and irrelevant discrepancies between the acquisition schemes are kept to the minimum and the change extraction is optimised to detect only what truly has changed within the area under investigation. Far from this ideal scenario, the reality is in fact much harder to face: even when the images are acquired by the same sensors, unpredictable bias and distortions might be too strong to be corrected, or the data might even be corrupted or missing due to instrument errors (or cloud coverage in the case of optical data). Also, being limited to the use of one sensor can be unpractical, if not problematic.
Imagine the timeline depicted in Figure 2.8: a particular area is covered by three satellites, each revisiting this same location every 12 days. A forest fire flares up at timet0, and the most logical thing to do would be to compare the two images from Sensor 3 at timet0−3 daysand Sensor 1 at timet0+3 days.
Instead, detecting this event with a homogeneous CD method requires the use of the image acquired at timet0−9 days. In the same way, one may think to monitor the development and the velocity of spread of this fire, however they would not be able to do so with images acquired every three days, but only by comparing data collected 12 days apart.
Undoubtedly, the limitations imposed by the assumptions of homogeneity are
17 2.3. Heterogeneous change detection
Figure 2.8: Combining heterogeneous data sources allows to increase the time resolution for detecting changes promptly and monitor their development more frequently.
too strict. The variety of available data and the methodological and compu-tational evolution of the last decade have eventually led the remote sensing community to develop CD algorithms that overcome these restrictions and are able to fully exploit all the available sources. These are called heteroge-neous CD methods, whose input data is also named multisource [21], mul-tisensor [10], cross-sensor [14], multimodal [15] and information unbalanced data [44]. The last two can also be seen as more general, since they cover both the multisensor case and the case when we have data from the same sensor, but with differences that can be attributed to sensor modes, sensor parametres and environmental parameters.
2.3.2 Challenges and solutions
When invalidating the assumptions of homogeneity, conventional homoge-neous CD techniques are unsuitable, and additional pre- or postprocessing steps are required [18, 20]. Indeed, heterogeneous data imply different do-mains, diverse statistical distributions, and inconsistent surface signatures across the images, especially when different sensors are involved that are not measuring the same physical quantities. Coping with these issues is much more complex than simply adding a preprocessing or cocalibration step to the CD pipeline described previously. In other words, a direct comparison is meaningless or even unfeasible without severe manipulations of the data [45].
Nonetheless, an assumption which must necessarily hold true is class sepa-rability, where the term class can refer to land covers, land uses, or single objects, depending on the specific applications and the spatial resolutions
Chapter 2. Heterogeneous change detection in remote sensing 18 used. If the representations of two or more classes of data produced by a sensor cannot be distinguished from one another, the resulting ambiguities cannot be coped with. Classes would mistakenly be thought as merging or splitting from one time to the next, and false or missed alarms could arise.
Therefore, there must be a one-to-one correspondence across domains for the class signatures involved in the changes. Moreover, the concept of class separability must be extended further. If a change alters a target’s physical property, which is not among the ones quantified by a specific sensor, then this change is inevitably invisible to the latter. Clearly, this requires that the correct sensor systems are used in order to detect a specific change process or change event [35, 46].
The taxonomy of heterogeneous CD methodologies is not trivial nor well-defined. The approaches to these problems are multiple and very diverse, and one can find several possible ways to categorise them [47, 48]. A first distinction can be made between supervised and unsupervised methods. Su-pervision in heterogeneous CD refers to the fact that training data is avail-able, where some pixels are labelled as changed and others as unchanged.
The labels can be obtained e.g. as a result of a visual inspection and a manual selection or of a ground campaign. These labels can be used as tar-gets during training of a change detector, or to exclude change pixels from the training set when learning an image regression function. Unsupervised methods do not have access to training data and cannot rely on any such labels.
This thesis uses the termself-supervised to mean that labels of changed and unchanged pixels have not been provided by an external source, but have been inferred from the data by the algorithm itself. This kind of automatic selec-tion of training data points has already been referred to as self-supervision in other research fields, such as as robotics [49, 50]. There are also a few examples of using this term in remote sensing [51, 52], although it has not taken root in the heterogeneous CD literature prior to this work. In any case, it should be made clear that a self-supervised method is unsupervised.
Another proposed classification of heterogeneous CD methods is the follow-ing: some are using similarity measures [10, 11, 53] or scale-invariant local descriptors [12, 54] with assumed invariant properties across the acquisitions.
Data transformation methods instead include those procedures based on the projection of the heterogeneous images into a common domain or feature
19 2.3. Heterogeneous change detection
Figure 2.9: Proposed taxonomy for the topic of heterogenous CD. The papers included in Chapters 7 to 9 are placed accordingly.
space, where they share the same statistics and for which classical CD meth-ods can be applied [13, 14, 19, 20, 21, 48, 53, 55]. In the same spirit, super-pixel segmentation [15], classification [16, 17, 31], or clustering [18] allow the mapping to a semantic space where it is easier to detect changes. Figure 2.9 depicts a combination of these classifications, and show where the presented papers fit in this overviewing picture.
An alternative subdivision into two groups sees parametric methods being contrasted against nonparametric ones. The former make use of a mixture of multivariate (or meta-Gaussian) distributions to model the dependencies between the two imaging modalities, or the joint statistics, or the different types of multisensor data [13, 56, 57, 58]. Instead, the latter come with the advantage of not explicitly assuming a specific parametric distribution for the data [19, 20, 21, 25, 44, 47, 59, 60, 61]. Among these, the most recently developed for heterogeneous CD are deep learning methodologies, which are also the most popular given the trend of the last few years, not only in remote sensing, but in many other fields of research in general.
Chapter 2. Heterogeneous change detection in remote sensing 20