Multi-sensor data fusion and feature extraction for forest applications

(1)

FACULTY OF SCIENCE AND TECHNOLOGY DEPARTMENT OF PHYSICS AND TECHNOLOGY

Multi-sensor Data Fusion and Feature Extraction for Forest Applications

Temesgen Gebrie Yitayew

FYS-3900 Master’s Thesis in Physics

May 2012

(2)

(3)

Abstract

This thesis focusses on extracting, evaluating and selecting features to study multi-sensor data fusion for forest applications. Due to the difference in the underlying sensor technology, data from synthetic aperture radar (SAR) and optical sensors refer to different properties of the observed scene and it is believed that when they are fused together, they complement each other to improve the performance of a particular application. Improved fusion results and useful physical interpretation of the observed scene can also be obtained by extracting and selecting sensor specific features. The two primary aims of this study are investigating the benefit of Polarimetric SAR and multi-spectral optical data fusion for forest applications by using different features extracted from each of these datasets, and identifying those few best Polarimetric SAR and multi- spectral optical features that jointly perform best for forest classification. Two secondary aims are comparing the potential of four different datasets for forest applications based on their individual classification performances, and comparing two feature selection algorithms in selecting the first few best features.

Multi-frequency fully Polarimetric SAR data at P-, L- and C-band and multi-spectral Landsat TM data acquired over the Nezer forest in France were used for demonstration. The scene is composed of homogeneous fields of either bare soil or maritime pine trees of different ages, and the application was discriminating the bare soil, and the trees in terms of their ages. On the one hand, different combinations of the polarimetric channels were used to extract simple polarimetric SAR features. On the other hand, different combinations of the available multi-spectral bands were used to extract different vegetation indices. A Supervised maximum likelihood Bayesian classification scheme was applied to evaluate and compare the classification performances of each of the four datasets and their different combinations. The classification accuracy (%) was used for a quantitative comparison. The two standard, sequential forward and sequential backward, feature selection algorithms were applied and compared in selecting the best features.

A literature review of data fusion methods found that feature level fusion is the best approach for our application. A total of twenty-six features; six from each of the three Polarimetric SAR datasets and eight from the optical dataset were extracted. A number of features from the extracted set were found useful to interpret the scene in terms of its physical parameters. In comparing the classification performances of the four datasets, it was found that P-band is the best whereas C-band is the poorest. The L-band and the Landsat TM datasets were found to have moderate performances. Therefore, P-band is potentially the best band for forest applications, and whenever it is available, priority should be given to the use of it. Significant classification accuracy improvement (up to 12%) was achieved by fusing the polarimetric SAR and the multi- spectral optical datasets. Therefore, attention should be given to the combined use of them whenever they are available.

The sequential forward feature selection approach gave slightly better results in selecting the few best features than the backward one. Therefore, whenever the objective is to choose the few best features, the forward approach should be used. Five features were found to jointly preserve 98.5% of the classification information of the available set. This shows the incredible advantage of feature selection in preserving most of the classification information and at the same time reducing the size of the data. Two of the best features, namely the mean radar backscatter and the cross-pol ratio, were identified from the polarimetric SAR features. The wetness and the soil brightness index were found to be the two best optical features in complementing the

(4)

polarimetric SAR features irrespective of the SAR frequency used. The normalized difference vegetation index was found specifically useful in complementing the P-band whereas the greenness was best in complementing the L-and C-band features. In addition to retaining most of the valuable information, these few identified features were found useful to interpret the scene in terms of the different forest scattering mechanisms. Therefore, they can be reasonably expected to be used for other forest applications too.

(5)

Acknowledgements

I am pleased to acknowledge the people who assisted me for completing this thesis. First and foremost, my deepest gratitude goes to my supervisors Camilla Brekke and Anthony Paul Doulgeris for tirelessly working with me from the beginning up to the end and sharing me their broad knowledge and experience. They have been following up the progress of my thesis with great care and commitment. Their discussions taught me a lot beyond the thesis and their ingenious suggestions are evident in every section of this thesis. They have spent a great deal of their valuable time reading and commenting each and every page. They earn my gratitude more than anybody else.

Thanks to Gokhan Kasapoglu for his help and valuable suggestions on certain issues in the thesis.

I would like to thank Robert Jenssen for being an excellent teacher and giving me the foundation of data analysis techniques

Thanks to Prof. Laurent Ferro-Famil at Univeristy of Rennes-1 for the Nezer forest dataset.

Special thanks to the Norwegian state educational loan fund-Lanekassen for the scholar- ship grant during my study at the university of Tromsø

Last but not least, I would like to thank my friends who helped and encouraged me in doing this project, specially Vidar and Thomas for helping me with Matlab related problems, Desalegn and Firehun for their wonderful help on L^ATEX .

(6)

(7)

List of Figures

2.1 Geometry of SAR. . . 10

2.2 Scattering mechanisms in a forest. . . 13

2.3 Spectral characteristics of healthy green vegetation [1]. . . 17

2.4 Spectral signature of some common land cover features. . . 18

3.1 Flow chart of pixel level fusion. . . 23

3.2 Flow chart of feature level fusion. . . 27

3.3 Decision level fusion. . . 28

4.1 Ground truth maps of the Nezer forest.. . . 34

4.2 Color composite P-, L-, and C-band images of the SAR data, HH(Red), HV(Green) and VV(blue). . . 35

4.3 Color composite image of a portion of the Landsat data, band-4(Red), band-2(Green) and band-3(Blue). . . 36

4.4 A schematic diagram showing the preprocessing applied to the datasets. . 37

4.5 Atmospheric correction of the Landsat TM dataset; left: uncorrected, right: corrected; band-4(Red), band-2(Green) and band-3(Blue). . . 38

4.6 Registered images at the SAR resolution; left: P band, HH (Red), HV (Green) and VV (blue). Multi-spectral TM data, band-4(Red), band- 2(Green) and band-3(Blue) (right). . . 39

4.7 Unltered P-band SLC image (left) and Speckle ltered image (right); color coding: |𝐻𝐻|(Red), |𝐻𝑉|(Green) and |𝑉 𝑉|(blue). . . 40

5.1 The 6 feature images of P band PolSAR data. . . 46

5.2 Histogram plots the 6 features of P band PolSAR data. . . 46

5.3 The 6 feature images of L band PolSAR data. . . 47

5.4 Histogram plots the 6 features of L band PolSAR data.. . . 47

5.5 The 6 feature images of C band PolSAR data. . . 48

5.6 Histogram plots the 6 features of C band PolSAR data. . . 48

5.7 Scatter plots of all possible pairwise combinations P-band features. . . 50

5.8 The soil line and distribution of pixels in a red and near/infrared space, [1]. 53 5.9 The 8 features of multi-spectral TM dataset.. . . 54

5.10 Histogram plots the 8 features of multi-spectral TM dataset.. . . 55

5.11 Scatter plots of all pair-wise combinations of the Landsat TM features. . . 56

(10)

6.1 Ground truth maps of the Nezer forest.. . . 61 6.2 The prepared ground truth map with its color-coding. . . 61 6.3 Training data points, shown as dotted (left) and testing data points (right). 62 6.4 Ground truth (left) and classication result (right) using all 26 features. . 66 6.5 Ground truth for the two-class case. . . 67 6.6 Ground truth (left) and classication result (right) using all 26 features in

discriminating between bare soil and forest. . . 68 6.7 Ground truth for the dierent tree age categories. . . 69 6.8 Ground truth (left) and classication result (right) using all 26 features in

discriminating among the tree age categories. . . 72 7.1 Ground truth map with equal number of samples. . . 76 7.2 Step-wise increase and decrease of the classication accuracies, SFFS (Left)

and SBFS (Right). . . 80 7.3 Ground truth (left) and seven class classication result (right) from the

selected four features (right) and ground truth map (left). . . 81 7.4 Color-coded scatter plots of possible pair-wise combinations of the four

selected features. . . 82 7.5 Ground truth map with equal number of samples for tree age categories. . 85 7.6 Ground truth (left) and tree age classication result (right) from the se-

lected ve features (right) and ground truth map (left). . . 87 7.7 Ground truth map with equal number of samples for bare soil versus forest. 88 7.8 Ground truth (left) and classication result (right) using the P-band. . . . 90 7.9 Ground truth map (left) and seven class classication result (right) from

the selected ve features.. . . 91 7.10 Color-coded scatter plots of possible pair-wise combinations of ve features. 96 7.11 Color-coded scatter plots of possible pair-wise combinations of the ve

features. . . 97

vi

(11)

List of Tables

2.1 Commonly used bands by SAR sensors for forest applications. . . 14

2.2 Landsat instrument bands.. . . 16

2.3 Comparison between SAR and optical Multi-spectral sensors. . . 19

3.1 Comparison of the three levels of fusion. . . 30

4.1 The proportion of each of the classes in the dataset. . . 34

5.1 List of extracted features. . . 57

6.1 Classication results from all individual and combined datasets. . . 64

6.2 Classication results from all individual and combined datasets in discriminating bare soil from forest. . . 67

6.3 Classication results from all individual and combined datasets in discriminating the six tree age categories. . . 69

6.4 Confusion matrix for tree age classication. . . 71

7.1 Feature labels. . . 78

7.2 Forward feature selection results. . . 78

7.3 Backward feature selection results. . . 79

7.4 Features selected using SFFS and SBFS for seven class discrimination. . . 80

7.5 Individual rankings of all the twenty-six features. . . 83

7.8 Features selected using SFFS and SBFS for tree age discrimination. . . 87

7.13 Comparing SFFS and SBFS results for P-band and Landsat TM features. 92 7.14 Forward feature selection results. . . 93

7.16 Comparing SFFS and SBFS results for L-band and Landsat TM features. 93 7.17 Forward feature selection results. . . 94

(12)

7.18 Backward feature selection results. . . 94 7.19 Comparing SFFS and SBFS results for C-band and Landsat TM features. 95 7.20 The selected features for P-band and Landsat TM. . . 99 7.21 The selected features for L-band and Landsat TM. . . 99 7.22 The selected features for C-band and Landsat TM. . . 99

viii

(13)

Abbreviations

SAR Synthetic aperture radar

PolSAR Polarimetric synthetic aperture radar RAR Real aperture radar

AIRSAR Air-borne synthetic aperture radar

TM Thematic Mapper

MSS Multi-spectral Scanner EMS Electromagnetic spectrum

IR Infrared

NIR Near infrared

SWIR Short wave infrared IHS Intensity hue saturation PCA Principal component analysis PCS Principal component substitution

PAN Panchromatic

HPF High pass lter

CPs Control points

ROI Region of interest

RST Rotation-scaling-translation QAC Quick atmospheric correction SLC Single-look complex

MLC Multi-look complex

VI Vegetation index

NDVI Normalized dierence vegetation index B Soil brightness index

G Greenness

W Wetness

PVI Perpendicular vegetation index TVI Triangular vegetation index SAVI Soil adjusted vegetation index

ARVI Atmospherically resistant vegetation index SFFS Sequential forward feature selection SBFS Sequential backward feature selection

(14)

(15)

Chapter 1

Introduction

These days, many remote sensing satellite sensors are acquiring information at dierent spatial, spectral and temporal resolutions, and hence a wide spectrum of data is available for the same observed site. However, the information provided by the individual sensors might be incomplete or imprecise for a given application [2, 3, 4]. Hence, a combined use of data from two or more sources may provide complementary information which could help to understand the observed scene better or improve the result of a particular application [5,6]. This is the main motivation for this study.

Data from synthetic aperture radar (SAR) and optical sensors refer to dierent characteristics of the observed scene, and it is believed that when combined, they oer complementary information that helps to distinguish the dierent classes of a particular observation. Optical data contains information on the reective and emissive characteristics of the targets in terms of spectral intensity. This spectral information is related the chemical composition and moisture content of the observed target. On the other hand, SAR data contains information on the geometric structure, surface roughness and dielectric properties of natural and man-made objects. As an example, spectral signature is the information inferred from optical data, which is used to characterize ground targets. However, some vegetation species may not be discriminated as they can have similar spectral responses. Therefore, radar images can help in discriminating these vegetation species as they contain additional information about the geometric structure and dielectric properties of the vegetation cover.

Data fusion refers to combining information from two or more sources together to improve the quality and interpretability of the source data. This can be achieved at any one of the three dierent processing levels of the image information: pixel, feature or decision levels.

Pixel level fusion is a low level fusion where dierent source images are combined to produce a single fused image

Feature level fusion is an intermediate level of fusion, which requires the merging

(16)

Chapter 1. Introduction

of extracted features

Decision level fusion is a high level fusion which is used to integrate separately processed image information using decision rules

Regardless of the fusion techniques used, the fused image is believed to contain greater information content for a given scene than any one of the individual image sources alone [3].

The process of transforming an input dataset into a set of representative features which accurately and concisely represent the original information is referred to as feature extraction. When dealing with data from multiple sensors, feature extraction is the rst process undertaken to preserve sensor specic information. When wisely implemented, the extracted features can be helpful to interpret the observed target in terms of its physical parameters. A related process, i.e., feature selection, refers to selecting a subset of features from the available set. It is an important process to identify those best features that contain most of the valuable information for a particular application. An additional benet of feature extraction, when it is accompanied by a systematic feature selection process, is that a smaller amount of memory and processing time will be required in the feature space because of the removal of redundant information in the process.

Many studies have been conducted to combine SAR and optical data for a number of applications [7,8,9,10,11]. To mention some of the applications, two SAR datasets from ERS-1/2 (C-band) and JERS-1 (L-band) are fused with a multi-spectral dataset from the SPOT satellite for the purpose of urban land cover classication [7]. In [8] a study was conducted to integrate images from ERS-1 satellite and Landsat thematic mapper sensors for geological study purposes. Snow cover mapping using these two dierent datasets was demonstrated in [9]. Another application example where SAR and optical data sets are integrated is land cover mapping, [10,12].

Even though they are not many, a number of studies have been conducted to integrate SAR and optical data for forest applications. However, most of these studies concentrated on estimating dierent forest parameters, mainly biomass and some related variables such as canopy height [13, 14, 15, 16]. Very recently, classication of forest in terms of tree species has been studied in [17]. In most of these studies, the raw data from the dierent sources is directly used for either forest variable estimation or classication.

However, only few studies have been conducted on extracting and combining features from SAR and optical datasets for forest applications [13,16]. In [16], statistical metrics derived from dierent features extracted from four data sources (LiDAR, SAR/InSAR, ETM+ and Quickbird) are used to compare the performance of the combined datasets with that of the individual ones in mapping forest biomass and canopy height. In [13], some selected polarimetric channels from multi-frequency polarimetric SAR, coherence amplitude from a single frequency interferometric SAR, and some selected bands from Landsat TM dataset are combined for the purpose of mapping foliage biomass. The selection among the channels and the bands was performed by looking at the corre-

2

(17)

lation information in a scatter plots. Apart from these two related eorts, no work is done to integrate dierent polarimetric SAR and multi-spectral optical features for forest classication.

This study focuses on extracting, evaluating and selecting dierent multi-frequency polarimetric SAR (PolSAR) and multi-spectral optical features to demonstrate the benet of SAR and optical data fusion for forest applications. The primary aim of the feature evaluation and selection process is to identify those features, from the dierent datasets considered, which are best for forest applications, with a secondary aim of comparing the potential of two feature selection algorithms in selecting a few best features. In addition, the study will look in to the individual classication performances of four dierent datasets with respect to forest classication. The main motivation of extracting and using dierent multi-frequency PolSAR and multi-spectral optical features is that sensor specic information can be retained in the fusion process and could be helpful to improve the fusion results by using dierent features that are unique to the sensors.

A data fusion approach, which takes in to account the sensor specic information in the fusion process and is suitable for the evaluation and selection of features, is chosen by an extensive literature review. The application is illustrated by using datasets acquired over the Nezer forest, France, under the objective of discriminating maritime pine trees of dierent ages and bare soil. Four datasets are considered; three air-borne fully polarimetric SAR datasets acquired with three dierent frequencies, C-, L- and P-bands, and one multi-spectral Landsat Thematic Mapper (TM) dataset. As a preprocessing step of the datasets, atmospheric correction is applied to the multi-spectral Landsat TM dataset, then all the bands of the Landsat TM dataset are co-registered with the Pol- SAR images by manual image tie-point selection and resampling, and the SAR images are multi-looked to reduce speckle and to adapt to the Landsat TM resolution. After processing the original single-look complex polarimetric SAR data, dierent combinations of the polarimetric channels are used to extract simple polarimetric features [18], and dierent combinations of the available multi-spectral bands are used to extract different vegetation indices [1]. A supervised maximum likelihood Bayesian classication scheme is applied to evaluate and compare the classication performance of each of the four datasets and their dierent combinations. Percentage classication accuracy is used for a quantitative comparison. Two standard, sequential forward and backward, feature selection algorithms [1] are applied and compared in selecting the few best features.

In total, twenty-six features; six from each of the three PolSAR datasets and eight from the Landsat TM dataset are extracted and used for this study. This number does not present all possible polarimetric and optical features, however, it includes the most commonly used features and is enough for our intended purposes, i.e., (a) to investigate the benet of extracting and selecting sensor specic features to identify those most representative ones and reduce the volume of the data, and (b) to investigate the complementariness of SAR and optical features. There are a lot more Polarimetric features, especially those can be derived from the polarimetric target decomposition theorems and other more multi-spectral features. They are not considered here as it is beyond the

(18)

scope of this study. The dierent fusion approaches are not quantitatively compared here. This is also beyond the scope of the project and the selection among the fusion approaches is simply made by reviewing the literature. For the classication and future selection part, a commonly used classier is applied, and the dierent classiers that can be used to analyse multivariate datasets will not be investigated here for the same reason.

As it is pointed out above, this study encounters discriminating among the dierent age groups of single-species pine trees. This task is actually harder than other forest applications that involve mixed-species trees. Therefore, the results of this study could potentially be used for other forest applications.

1.1 Structure of the thesis

The thesis consists of both theoretical concepts and experimental parts. The theoretical concepts are covered in chapters 2 and 3, while the experimental parts are covered in chapters 4, 5, 6 and 7.

The second chapter introduces two key concepts. The rst one is the sensor technology of SAR and its optical multi-spectral counterpart, and the second fundamental concept is the interaction of electromagnetic waves with the forest in the microwave and visible/infrared regions of the electromagnetic spectrum.

The third chapter looks into the concept of data fusion. Here, the basic concepts of data fusion, as it is applied to remote sensing purposes, are discussed and the most widely used fusion techniques are reviewed. In addition, by comparing the dierent data fusion approaches, an appropriate one for our application is selected.

The fourth chapter rst introduces the datasets used in this study, and then addresses the preprocessing of the datasets. Atmospheric correction, image to image registration and speckle ltering are the main preprocessing tasks considered.

Feature extraction from the datasets is presented in chapter ve. Here, dierent polarimetric SAR and multi-spectral features are extracted and discussed in terms of the physical parameters of the scene.

In chapter six, a maximum likelihood Bayesian classication scheme is applied on the extracted feature sets. Here, the benet of multi-source data fusion is demonstrated by combining the features from the dierent sources. The classication results from the individual and the combined datasets are quantitatively compared.

Feature evaluation and selection is covered in the seventh chapter. Here, the whole feature set from the dierent datasets are analysed and features, which jointly perform best with respect to classication accuracy, are selected. Two feature selection approaches are used and compared.

4

(19)

Lastly, chapter eight concludes the whole work.

(20)

(21)

Chapter 2

SAR and optical sensors and

electromagnetic interactions with forest

A lot of literatures and textbooks can be referred for a detailed description of polarimetric synthetic aperture radar [19,20,21,22] and optical multi-spectral sensors [1,23].

In this chapter, some of the relevant concepts about these sensors are addressed to give a background information about the PolSAR and optical multi-spectral datasets used in this project. In addition, the interaction of electromagnetic radiation with the forest is among the topics discussed. A few basic characteristics of the images acquired by these sensors are also described, to give some insight about the issues that should be ac- counted for in the preprocessing step of the datasets considered. Finally some important comparisons between the datasets acquired by these sensors will be given.

2.1 The potential of remote sensing for forest applications

Due to their extended area coverage and complex nature, information about forests is not easily accessible. Remote sensing is an important tool for forest applications as it is capable of acquiring information over a wide area in a repetitive manner. The ability of satellite remote sensing systems to image extensive areas makes them preferable for applications at the global scale, and air-born remote sensing systems can be used for forest monitoring at a local scale [24]. From their orbiting nature, space-born sensors are capable of repeat imaging every part of the earth's surface at a xed interval of time. Hence, they can provide a large amount of multi-temporal data for the same observed forest site, that can be used for dierent purposes. Even though they can oer greater exi- bility with respect to the time of data acquisition, air-born systems will cost much more to provide similar frequency of observation over extended areas. An important aspect

(22)

Chapter 2. SAR and optical sensors and electromagnetic interactions with forest

of radar systems is their ability to see through clouds and atmospheric moisture. This is particularly useful for forest applications, especially in the humid tropics. Moreover, the high resolution and polarimetric capabilities of recent SAR systems, makes remote sensing an important source of information for dierent forest applications.

Remote sensing information acquired over forest can be used for dierent applications such as [25]:

Forest/non-forest mapping

Forest type and species mapping

Forest age discrimination

Deforestation and forest regeneration mapping

Forest re detection

Carbon and biomass mapping

As it is pointed out in the introduction chapter, this study encounters forest age discrimination. Such an information can be used for dierent purposes such as timber production and forest inventory [26,27].

2.2 Synthetic aperture radar (SAR)

2.2.1 Basic concepts

Radar is an active sensor, which works by transmitting and receiving pulses of microwave energy. The sensor transmits a signal, which is directed toward the target area to be investigated. The radiation reected from that target is detected and measured by the radar receiver. For earth observation purposes, these sensors can be installed on either an air-borne or space-borne platforms and can operate at various frequencies. Real aperture radar (RAR) is a radar system where the resolution in the ight direction is controlled by the physical length of the antenna. For these systems, only the amplitude (and not the phase) of each return echo is measured and processed. To determine the spatial resolution at any point in a radar image, it is necessary to compute the resolution in two dimensions, in the range (across track) and azimuth (along track) directions. The range resolution of a RAR is dependent on the eective length of the pulse in the slant range direction and mathematically it is given by [28]:

△𝑅= 𝐶𝜏 2 = 𝐶

2𝛽

where△R is the slant range resolution, C is the speed of light,𝜏 is the pulse length and 𝛽 is the bandwidth. This resolution could be improved by decreasing the pulse length.

8

(23)

But if we decrease the pulse length, the system requires much input power (in order to get a detectable amount of signal) as the signal energy is given by

𝐸 ≡𝑃 𝜏

where P is the instantaneous peak power. However, the maximum power is severely limited by the sensor hardware, particularly in the case of space borne sensors. Thus, in order to have high detection ability (large E) and a high resolution (large𝛽), a pulse with the seemingly incompatible characteristics of large 𝜏 and large𝛽 is needed. This can be achieved by signal processing techniques, modulating and de-modulating the transmitted and received signals respectively [28].

The azimuth resolution △A is given by the beam width and is a function of the operating wavelength, position in range and the dimension of the antenna in that direction.

Mathematically, it is given by

△A=𝐿= 𝜆𝑅 𝐷𝑎

Where L is the beam width, 𝜆 is the wave length, 𝐷𝑎 is the dimension of the antenna in the azimuth direction and R is the slant range distance to the target. One way of achieving better azimuth resolution is to boost frequency; another is to increase along track antenna length; a third is to decrease the target range. None of these options is very eective from space. This is where the concept of Synthetic aperture radar (SAR) comes into play.

SAR is a coherent mostly airborne or space borne side-looking radar system which utilizes the motion of the platform to simulate an extremely large antenna using advanced signal processing, and that generates high-resolution remote sensing imagery. The geometry of SAR is shown in gure 2.1 below. The platform travels forward in the ight direction with the nadir directly beneath the platform. The radar beam is transmitted obliquely at right angles to the direction of ight illuminating a swath, which is oset from nadir.

That is the reason why it is also known as a side looking radar. The radial line of sight distance between the radar and each target on the surface is called the slant range distance.

As the radar moves, a pulse is transmitted at each position (shown by string of dots in the gure below). The amplitude and phase of the signals returned from objects are recorded and stored throughout the time period in which the objects are within the beam of the moving antenna. Advanced signal processing techniques are used to coherently combine the recorded information from each of the returned signals to achieve very high azimuth resolution. The nal output could be a single image with the highest azimuth resolution possible (single-look data) or a multi-look processed data composed of several images with reduced azimuthal resolution. Generally, depending on the processing, resolutions achieved by SAR sensors are of the order of 1-3 meters for air-borne SAR and 5-50 meters for space-borne SAR [20, 28]. SAR achieves ne range resolution the same way as RAR does, by pulse modulation. As an example, the SAR data used in this project

(24)

has 3 meters spatial resolution in the azimuth and 6.7 meters of spatial resolution in the range.

Figure 2.1: Geometry of SAR.

2.2.2 SAR Polarimetry

Polarization describes the orientation of the electric eld component of electromagnetic waves in a plane perpendicular to the direction of propagation. In general, radar systems can have one, two, or all four of the following transmit-receive linear polarization combinations; HH, VV, HV, VH. Conventional SAR systems operate within a single, xed-polarization antenna for both transmission and reception of microwave signals. In this way, a single radar reectivity is measured, for a specic transmit and receive polarization combination, for every resolution element of the image. As a result, additional information about the scattering process contained in the polarization properties of the scattered signal is lost. To ensure that all the information of the scattered wave is retained, the polarization of the scattered wave must be measured. Polarimetric SAR is a radar system, which transmits microwave signals in two orthogonal directions (H and V) and records the backscattered signal in two or more separate channels. Depending of the

10

(25)

number of receiving channels, polarimetric SAR systems can be semi- or fully polarimetric. Fully polarimetric channels record the received signal in four separate channels. The target response is given by a 2×2 scattering matrix, where the diagonal elements are the co-polarization (HH, VV) terms, while the o diagonal elements are known as cross- polarization (HV, VH) terms. Mathematically the scattering matrix S is represented by:

S=

[︃ 𝑆_ℎℎ 𝑆_ℎ𝑣 𝑆𝑣ℎ 𝑆𝑣𝑣

]︃

SAR polarimetry has great advantage over conventional single-channel radar systems as it is capable of measuring the complete scattering matrix for each resolution element of a scene [21,29]. This helps to obtain more information about the scattering mechanisms on surfaces or within volumes.

2.2.3 SAR image characteristics

A SAR image has several characteristics that make it unique. Images obtained from coherent sensors such as SAR systems are characterized by speckle. Speckle is a salt and pepper appearance of radar images which is caused by random constructive and destructive interferences from the multiple scattering returns that will occur within each resolution cell. In other words, speckle is a statistical uctuation associated with the radar reectivity (brightness) of each pixel in the image of a scene. It is a form of noise which degrades the quality of radar images and therefore reducing this eect could help for better discrimination of targets. One of the most common speckle suppression techniques is multi-looking. Single-look imaging uses all signal returns from a ground target to create a single image. Multi-looking is the dividing of the radar beam into several narrower sub- beams. Each sub-beam provides an independent look of the illuminated scene. Summing and averaging the images from the dierent looks will result an image with reduced speckle. In cases where only single-look processed images are available, the averaging can be done on the local neighbourhood of pixels. The later approach is used for this study, as the original datasets were supplied as single-look complex images. However, multi-looking in general is performed at the expense of the spatial resolution of radar images.

However, it is also important to note that speckle is not really noise in the classic sense because it is the radar signature of the target. It has some useful information that can be used to characterize the texture information in radar images. Texture in radar images refers to the natural variation of the average radar backscatter on a scale of several resolution cells. Therefore, its statistical characterization requires measurements from a nite sampling window rather than estimates at the individual pixel level [30].

Another property of radar images is that the position and proportions of objects in SAR images can appear distorted compared to a photograph. This distortion is a unique geometric characteristic of radar images resulted from the dierence in sample spacing

(26)

between the slant range plane and ground plane. This slant range distortion causes variation in the scale of radar images from near to far range, and targets in the near range appears compressed relative to the far range. There are also other geometric distortions of SAR images and a more complete summary of them can be found in [30].

2.2.4 Interaction of microwaves with forest

When electromagnetic radiation strikes a surface, it may be transmitted, absorbed and re emitted or reected. Transmission occurs when radiation passes through a target while reection occurs when radiation "bounces" o the target and is redirected. In the case of absorption, the incident electromagnetic radiation will be converted to other forms of energy after being absorbed by the target. Remote sensing relies on measuring the reected energy from targets. Depending on the surface roughness in comparison to the wavelength of the incoming radiation, earth surface features reect either specularly or diusely, or somewhere in between these two extremes. Usually the denition of smoothness or roughness for surface scattering is given by some criteria and is totally dependent on the wavelength of the electromagnetic radiation used, the incident angle and the surface standard deviation height. Such two criteria are the Rayleigh and the Fraunhofer criteria [22]. Specular reection occurs when almost all of the incident energy is directed away from the surface in a single dominant direction and it is a property of smooth surfaces. On the other hand diuse reection occurs when the incident energy is reected almost uniformly in all directions. If the wavelengths are much smaller than the surface variations or the particle sizes that make up the surface (rough surface), diuse reection will dominate [31].

SAR sensors record the backscatter signal (both the amplitude and the phase) from targets. The proportion of this backscattered signal as compared to the transmitted one is dependent on a number of factors including, surface roughness, slope of the surface, dielectric properties of the target, types of land cover (soil, vegetation, man-made objects), microwave frequency, polarization and incident angle. Let us closely examine each of these factors in the framework of land cover features in general and forest in particular.

In general, the backscatter intensity for rough surfaces is higher than smooth surfaces;

hence the SAR image will look brighter. Trees and other vegetation are usually moderately rough on the wavelength scale. Hence, they appear as moderately bright features in SAR images. When a transmitted wave strikes a target and returns directly back to the sensor, it is known as single scattering or single bounce. In a forest media, this could happen if the wave directly bounces back after hitting the dierent tree structures or the ground. Figure 2.2shows the dierent types of scatterings from a forest [32]. Single bounce is denoted by A and C in the gure. If the transmitted wave bounces o twice, it is termed as double bounce. This is a common property of a group of targets known as corner reectors that include built-up areas, ships on the sea, high-rise buildings, metal- lic objects such as cargo containers etc. As it is shown in the gure (letter B), double

12

(27)

bounce could be a result of interactions between ground and trunk or between trunk and twigs in a forest. Such targets have brighter appearances in radar images. This is due to the fact that while the transmitted signal bounces o twice, the specular reection component with its high energy will direct back to the sensor. The third type of scattering occurs when the transmitted wave bounces o more than twice, and it is known as multiple scattering or volume scattering. This is the most common type of scattering in environments such as dense forest canopies which occur due to interactions among leaves, branches, twigs and trunks. Therefore, in a forest, we usually have all these three scattering mechanisms, however the volume scattering is dominant. The proportion of each depends on the type, height and density of the trees.

Figure 2.2: Scattering mechanisms in a forest.

Dielectric constant is another parameter which inuences the interaction of radar waves with targets. It describes the response of a medium to the presence of electric eld [33], and therefore it is related to its conductivity. For trees and natural targets, this information is related to their moisture content. A medium with a higher dielectric constant has a higher reectivity. As an example, water has a dielectric constant of 80 whereas dry soil and rocks have 3-8 at radar wavelengths [33]. This direct relationship between water and dielectric constant and higher sensitivity of radar backscatter to dielectric constant can provide useful information when it comes to forest applications [34]. This is because information about moisture and volumetric water content of forest canopies can easily be inferred from radar signatures.

(28)

Another very important parameter to consider in microwave-target interactions is frequency (or wavelength). Because the surface roughness is dependent on the operating frequency of SAR, it will have a direct eect on the appearance of a particular target in radar images. Penetration is the key factor for the selection of a specic band for a particular application: the longer the wavelength (the smaller the frequency), the stronger the penetration into vegetation and soil. This higher penetration capability of relatively longer wavelengths has a two fold advantage for forest applications. One is that longer wavelengths are capable of penetrating through clouds and atmospheric moisture. This is especially useful for forest monitoring in the tropical regions which are frequently under cloud covers throughout the year. One of the main advantages of SAR over optical sensors, that are to be discussed in the section to follow, is this penetration capability which enables them to acquire information in almost all weather conditions. The second key advantage of those longer wavelengths for forest applications is that they can penetrate through the vegetation canopy and interact with the many structures of the forest [33].

This may give useful distinguishing ability that may not be present in surface scattering alone. Most widely used bands by SAR sensors for dierent forest applications are listed in table 2.1.

Table 2.1: Commonly used bands by SAR sensors for forest applications.

Band designation Wavelength Frequency P 30 - 130 cm 0.3 - 1 GHz

L 15 - 30 cm 1 - 2 GHz

C 3.75 - 7.5 cm 4 - 8 GHz X 2.4 - 3.75 cm 8 - 12.5 GHz

The type of polarization of the radiation in polarimetric SAR has a major role in deter- mining the type of signal interaction with the forest components. The probability that the like-polarised radiation interact with structures having similar orientation is very high, so vertical structures in a forest will interact strongly with VV polarization. Branches with horizontal orientation interact strongly with HH polarization. The cross-polarized backscatter (HV/VH) is highly sensitive to biomass as it is commonly originated from canopy volume scattering in a forested/vegetated media. This is a direct consequence of the higher depolarizing eect of the multiple scatterers in a forest. For at bare soil surfaces with no signicant moisture content, HH and VV have approximately similar responses [35].

Incident angle is another parameter which aects the radar backscatter. It refers to the angle between the incident radar beam and the direction perpendicular to the surface of the target at the point of contact. Its eect on the microwave-forest interaction is primarily due to its inuence on the microwave vertical penetration depth. In general, the radar backscatter decreases with increasing incident angle.

14

(29)

2.3 Optical Multi-spectral sensors

Unlike active sensors such as radar, optical sensors are passive sensors, which measure naturally available energy. It is well known that the sun is a very convenient source of naturally available energy for remote sensing. The sun's energy is either reected, as it is for visible and a portion of infrared (IR) wavelengths, or absorbed and then re-emitted, as it is for thermal infrared wavelengths. Therefore, reected energy can only be detected during the time when the sun is illuminating the Earth. Energy that is naturally emitted (such as thermal infrared) can be detected day or night, as long as the amount of energy is large enough to be recorded.

Optical wavelength regions in the electromagnetic spectrum (EMS) which are mainly applicable for passive remote sensing are:

The visible region (0.4-0.7𝜇m)

The infrared region (0.7𝜇m-1mm)

The visible region is a narrow band which is visible to the human eyes. It consists of the various color components of the EMS among which red (610−700𝑛𝑚), green (500−570𝑛𝑚) and blue (450−500𝑛𝑚) are the principal color components. The portion of the infrared band which is useful for passive remote sensing can be further divided into reected IR and thermal IR. Radiation in the reected IR region is used for remote sensing purposes in ways very similar to radiation in the visible portion. The thermal IR region is quite dierent from the visible and reected IR portions, as this energy is essentially the radiation that is emitted from the Earth's surface in the form of heat.

There is no clear distinction between these regions as there is radiation reected and emitted from some portion of the region simultaneously.

Multi-spectral remote sensing systems record reected or emitted energy from an object or region of interest in multiple bands of the electromagnetic spectrum. These sensors are implemented as either air-borne or space-borne systems. The Landsat Multi-spectral Scanner and Thematic Mappers are the two well known sensors, which acquire information about the Earth's surface from space [1]. The Multi-spectral Scanner is able to record the reected energy in the visible and IR portion of the spectrum in four dis- crete bands, whereas the TM records reected visible, reected IR and emitted (thermal IR) energy in seven separate bands. The wavelength ranges of both Landsat MSS and Landsat TM bands are listed in table2.2.

(30)

Table 2.2: Landsat instrument bands.

Band MSS Band TM

1 0.5-0.6 𝜇m GREEN 1 0.45-0.52 𝜇m BLUE 2 0.6-0.7 𝜇m RED 2 0.52-0.6 𝜇m GREEN

3 0.7-0.8 𝜇m IR 3 0.63-0.69 𝜇m RED

4 0.8-1.1 𝜇m IR 4 0.76-0.9 𝜇m NIR

5 1.55-1.75 𝜇m SWIR1 6 10.4-11.5 𝜇m TIR 7 2.08-2.35 𝜇m SWIR2

Where, IR = infrared; NIR = near infrared; SWIR = short wavelength infrared; TIR = thermal infrared; and 𝜇m = micron or micrometer.

The output of multi-spectral imaging systems is a stacks of images, each associated with the dierent bands. Images acquired by optical sensors like any other images are susceptible to geometric distortions caused by variations in platform stability including changes in their speed, altitude, and angular orientation with respect to the ground during data acquisition. It is assumed that all of the system corrections have been made for the datasets used in this study.

2.3.1 Interaction of visible and IR electromagnetic waves with the forest

The amount of reected visible and IR energy from land cover features is mainly inu- enced by the chemical composition and moisture content of the observed scene. This energy, which can be recorded by optical sensors, is usually expressed as a percentage of the amount of energy incident upon those features, and it is termed as reflectance.

Across any range of wavelengths, the percent reectance values for landscape features such as water, bare land, sand, vegetation, etc. can be plotted and compared. Such plots are called spectral signatures. Vegetation in general has a particular spectral signature form which enables it to be distinguished readily from other types of land cover features in an optical/near-infrared image. The spectral signature of a typical healthy green vegetation is shown in gure 2.3. It can be clearly seen from the gure that the reectance for vegetation is low in both the blue and red regions of the spectrum. This is due to the fact that chlorophyll and other pigments in plants absorb the incoming radiation at these specic wavelength ranges for the purpose of photosynthesis, a food making process in plants. On the other hand, these pigments reect the incident radiation at the green region, which gives rise to the green color of vegetation. In the near infrared (NIR) region, the reectance is much higher than that in the visible band due to the cellular structure in the leaves. This is evidenced by the peak reectance values in the gure. Therefore, the internal structure of healthy leaves act as excellent diuse

16

(31)

reectors of near-infrared wavelengths. Hence, vegetation can be identied by the high NIR but generally low visible reectance values [1,36,37].

Figure 2.3: Spectral characteristics of healthy green vegetation [1].

In most cases, dierent landscape features have dierent spectral signatures. As an example, the spectral signature of some common landscape features is given in gure2.4, [38].

This unique spectral absorption and reection characteristics of the dierent landscape features in visible and near infrared region is the basis for multi-spectral and hyperspectral remote sensing. In principle, a particular landscape feature can be identied from its spectral signature if the spectral resolution of the sensing system is high enough to distinguish its spectrum from other landscape features [38]. As it is pointed out in the introduction chapter, this study encounters discriminating among the dierent tree age categories of a forest. Therefore, a relevant point for our study that can be inferred from the gure is that the shape of the reectance spectrum can be used for identication of vegetation types. From the gure, vegetation types 1 and 2 can be easily distinguished

(32)

from one another as they have dierent responses to the visible and infrared regions.

For example, this could be used to discriminate between oak and pine trees in a forest of heterogeneous species or younger and older trees in a mono-species forest. Therefore, even with in the same vegetation type, it is possible to discriminate among the dierent classes of vegetation based on age, moisture content, health etc. as these conditions aect the way the radiation interacts with vegetation.

Figure 2.4: Spectral signature of some common land cover features.

Another point that has to be mentioned with respect to optical sensors is that they do not directly measure the spectral reectance of targets, rather they measure the spectral radiance (up-welling radiance) at the sensors. Therefore, in addition to surface reectance, the spectral radiance measured by these remote sensors depends on the interactions of input solar energy with the atmosphere (during its upward and downward passages).

This is one of the limitations of optical remote sensing systems compared to microwave remote sensing systems. Since these additional factors aect our ability to retrieve accu- rate spectral reectance values for ground features, an atmospheric correction has to be made before any further processing. This issue is addressed in section4.2.1.

18

(33)

2.4 Comparison and Complementariness of SAR and opti- cal datasets

Advantages of SAR sensors include the ability to obtain measurements under almost all weather and environmental conditions so that data can be collected at any time. In addition, their high resolution imaging and polarimetric capabilities are other important aspects of SAR sensors. Moreover, since they use their own source of energy, SAR sensors can be used to better control the way a target is illuminated. However, active remote sensing systems in general require the generation of a fairly large amount of energy to adequately illuminate targets. From the discussions of this chapter, some important points and comparisons of SAR and multi-spectral optical sensors are summarized in table 2.3.

Table 2.3: Comparison between SAR and optical Multi-spectral sensors.

Optical multi-spectral SAR

Platform airborne/space-borne airborne/space-borne

Radiation Use reected sunlight (passive) Use Its own radiation (active)

Spectrum visible/infrared microwave

Frequency multi-frequency multi-frequency

Polarimetry not available is possible

Acquisition time day time Day/night

Weather blocked by clouds Can see through clouds and light rain

As it is discussed above, information acquired by SAR sensors contain information on the geometric structures, surface roughness and dielectric properties of natural and man- made objects. On the other hand, optical data contains information on the reective and emissive characteristics of the Earth's surface. This information is highly dependent on the chemical composition and moisture content of the observed target. We have seen in the above discussions that this information can be eectively used to determine the type of feature that the imaged surface contains (water, vegetation, etc.). Even though it is aected by atmospheric attenuation, optical multi-spectral imagery take advantage of the lack of the speckle eect leading to images with a far better quality [39]. These dierent types of information retrieved from SAR and optical sensors are referring to dierent object qualities and when used together, they will complement each other. This will improve the result of a particular application such as classication. This is of course one of the motives of this study, taking advantage of this complementary information.

Data fusion is treated next.

(34)

(35)

Chapter 3

Multi-Source Data Fusion

This chapter is aiming at introducing the concept of data fusion applied in the context of remote sensing purposes. The commonly used fusion approaches in the data fusion community are reviewed and some practical examples are provided. Finally, some comparisons among the fusion approaches are made to choose a convenient approach for our study.

3.1 Introduction

Data fusion refers to combining data from two or more sources or data from a single source acquired at two dierent times to increase the quality and improve the interpretation performance of the source data. Fused images may provide increased interpretation capabilities and more reliable results since data with dierent characteristics and complementary information are combined [3]. Preprocessing such as image registration and geocoding are applied to the component datasets prior to fusion to bring them into alignment [40].

In remote sensing, fusion of multi-temporal and multi-sensor datasets is of considerable importance to earth and space observation applications, such as environmental, agricultural and maritime monitoring. It is applied to integrate the information acquired with dierent spatial, spectral and temporal resolution sensors mounted on satellites, aircraft and ground platforms to produce fused data that contains more detailed information than each of the component inputs alone. In particular it has been successfully applied for land cover classication [10, 41]; urban area surface feature enhancement and mapping [42]. Specic application examples include: sharpen images [43]; enhance certain features not visible in either of the component datasets [44]; complement data sources for improved classication [10]; change detection using multi-temporal data [45]. Some examples of the many studies conducted to integrate a pair of images where the rst is

(36)

Chapter 3. Multi-Source Data Fusion

acquired by a multi-spectral sensor of low spatial resolution while the second is acquired by a panchromatic sensor of high spatial resolution include [46,47,48,49].

The datasets to be fused could be from:

Two or more dierent sensors with dierent spatial or spectral resolutions, A good example in this regard is the fusion of images acquired by SAR sensors with optical data which is acquired by sensors sensitive to the visible/infrared portion of the electromagnetic spectrum.

A single sensor, where the data is acquired at dierent times (multi-temporal).

This helps to reveal the changes between datasets acquired at dierent times.

It is also possible to combine remotely sensed data with ancillary data, for example fusion of optical images with geographic information system (GIS) data.

The fusion of optical and SAR data has received a tremendous attention by the remote sensing community[7,8,9, 10, 11, 50]. This is because, as it is discussed in chapter 2, the two datasets contain complementary information about the observed scene. Some application examples where these two dierent datasets applied include; land cover mapping [10,12], geological study [8], snow cover mapping [9] and forest biomass estimation [14, 15]. As it is pointed out in the introduction chapter, this study investigates the fusion of multi-frequency Polarimetric SAR and multi-spectral optical datasets for the purpose of forest classication.

Dierent techniques have been proposed by many authors to integrate multi-source remote sensing data for the purpose of enhancing various features. The most commonly used data fusion techniques are discussed in the section to follow.

3.2 Data fusion techniques

In general remote sensing data fusion techniques can be classied into three dierent levels: pixel level, feature level and decision level [3]. This classication is according to the stage of processing at which the fusion takes place.

3.2.1 Pixel level fusion

This level of fusion refers to fusion at the lowest level, where multiple source images are combined to produce a single fused image. In many cases, this technique is applied to enhance the spatial resolution of one of the images while maintaining the spectral properties of the other images. A celebrated example in this regard is the fusion of a pair of images where the rst acquired by a multi-spectral sensor has a pixel size greater than the pixel size of the second image acquired by a panchromatic sensor. The results will be a new multi-spectral image with a spatial resolution equal to the panchromatic

22

(37)

one. In general, precise co-registration of the datasets is required (at a sub-pixel level of accuracy) for this type of fusion.

Figure 3.1: Flow chart of pixel level fusion.

Figure 3.1 shows the general structure of pixel level fusion of two images, where the component images undergo a registration process rst before being merged. There are dierent techniques of pixel level fusion. A detailed review of these techniques can be found in [3,51,52]. The basic principles and particular application examples of the some commonly used pixel level fusion techniques as reviewed from the literature are presented as follows:

1. Arithmetic fusion algorithms

These algorithms produce the fused image pixel by pixel, as an arithmetic combination of the corresponding pixels in the input images [51]. They are also the simplest and sometimes eective fusion methods. The arithmetic operations involved includes addition, multiplication, averaging, subtraction and division. Addition and multiplication of images is useful for contrast enhancement whereas dierence and ratio images are particularly suitable for change detection [3]. Examples of studies in which this fusion approach is applied to combine high-resolution panchromatic images with lower resolution multi-spectral data to obtain high-resolution multi-spectral imagery include [53,54, 55]. In their respective works, they used a combination of weighted addition and multiplication to combine the input images.

2. Color composite (RGB) methods

These methods assign three dierent monochrome inputs as the red, green and blue

(38)

color channels of the fused image. The intensity information (grey scale value) of the input images is converted into color information (pseudo color). Prior to the fusion process, dierent image enhancement techniques such as contrast stretching and histogram equalization are usually applied to the monochrome input images to improve the color contrast of the fused image [51]. The dierence in the grey scale values of each of the single channels results in variations in color which facilitates the interpretation of multi-channel image data. The RGB color composite technique is also commonly applied in combination with other fusion techniques such as intensity hue saturation (IHS) and principal component analysis (PCA), methods that will be discussed in the next subsections. This method has been applied to combine dierent datasets. As an example, it is applied to integrate images from SPOT and Seasat satelites for the purpose of improving the interpretability of geological features over temperate agricultural regions [56]. Some more examples where this method is applied to integrate optical and microwave data include [57,58].

3. Intensity hue saturation (IHS) transformation fusion techniques

This method is one of the most widely used pixel level fusion technique. It is a class of component substitution techniques where the intensity component image of the multi-spectral data (lower spatial resolution) is replaced by an image with higher spatial resolution to improve the spatial resolution of the multi-spectral image.

The method involves three steps. First, three bands of the lower spatial resolution dataset is transformed to IHS space. Second, the higher spatial resolution image replaces the intensity component. Third, backward transform from IHS space to the original space to construct the fused result. The higher spatial resolution image has a contrast stretching applied to it so that it approximately can have the same variance and mean value as the intensity component image, just to make sure that the two images are approximately equal spectrally [3, 43]. This method has been successfully applied to merge dierent datasets. The following are some of the examples. It has been used to merge Landsat TM and PAN data [59], hyperspectral and radar data to enhance urban surface features [42]. Another study was also carried out in [60] to merging IRS-1C multi-spectral data and panchromatic data using this method. In [61], the IHS fusion approach and other fusion techniques were applied to combine multi-spectral optical data with a panchromatic image for forest application.

4. Principal component analysis (PCA) fusion method

This is a method used to transform a multivariate data set of inter-correlated variables into a dataset of uncorrelated linear combination of the original variables [3].

The method generates a new set of orthogonal axes. It involves the computation of the eigenvalues/eigenvectors in decomposing the data into its principal components [62]. The principal components correspond to the dominant eigenvalues whose corresponding eigenvectors describe the direction that optimally retain most of the variance of the data. The uncorrelated data is the result of projecting the original

24

(39)

data on to the eigenvectors. There are two approaches of image fusion using PCA [3,51]. The rst involves performing PCA on a multichannel image and then re- placing the rst principal component by a dierent image (usually an image with higher spatial resolution like PAN). The procedure of this merging method is similar to the IHS method and it is also known as principal component substitution (PCS).

The second approach integrates the disparate natures of multi-sensor input data in one image. The image channels of the dierent sensors are combined into one image le and a PCA is calculated from all the channels [36]. In both cases, inverse PCA is used to transform the data back to the original image space. A number of studies use this technique to fuse dierent datasets. In [50] the method is used to combine a SPOT XS image and an ERS-2 SAR image. The integration of ERS-2 and four bands of IRS-1C datasets using this fusion technique is also demonstrated in [63] for geological information enhancement. In [64], the performance of the PCS fusion method is compared with two other pixel level fusion approaches, i.e., IHS and Brovey's transformation ([65]) for forest applications. It is reported in their results that PCS provides better information for the discrimination of forest stand types than the other two.

5. High pass ltering (HPF)

This method is commonly used to enhance the spatial resolution of a lower spatial resolution image by using high pass lters. In this method, the higher spatial resolution data have a small high pass lter applied. The results of the ltering operation contain the high frequency information that is mostly related to the spatial information. The HPF results are added, pixel by pixel, to the lower spatial resolution, but higher spectral resolution, dataset. The result will be a fused image with both higher spatial and spectral resolutions. As an example, this method has been successfully applied to merge the IRS-1C multi-spectral and panchromatic bands [60].

3.2.2 Feature level fusion

Feature level fusion requires the extraction of various features from multiple data sources and then combining them into a single feature vector that can be used instead of the original data for further processing. The features are an abstraction of the raw data intended to either highlight a particular characteristics of the observed target or provide a reduced form of the raw data, which accurately and concisely represents the original information. Representative features for imagery data includes [66]:

Geometrical characteristics of image segments, such as edges, lines, line length, line relationships (parallel, perpendicular), arcs, circles, conic shapes, size, area.

Structural features, such as surface area, relative orientation, orientation to vertical and horizontal ground plane.

(40)

Statistical features, such as number of surfaces, moments/mean, variance, kurtosis, skewness, Fourier descriptors, wavelet coecients, entropy.

Spectral features, such as color coecients, spectral peaks, spectral signature and vegetation indices.

Polarimetric features, such as co-pol and cross-pol ratios.

Contextual features, e.g., texture.

Transformation of the raw data into feature vectors is termed as feature extraction. Meth- ods applied to extract features usually depend on the characteristics of the individual source data and the application [52]. Segmentation procedures, region characterization, band combinations, polarimetric decomposition and principal component analysis are some of the methods that can be used for the feature extraction process. The output of this process is a list of feature vectors describing the main characteristics of the original data. These extracted features can use the separate sensor specic characteristics, which pixel level does not. Feature level fusion is then implemented on these feature sets. Fig- ure 3.2 shows a general scheme of fusion at feature level for two registered images. As can be seen from the gure, the extracted features from each of the images are fed to a feature level fusion algorithm to form a single feature vector. When the feature sets are homogeneous, e.g., features obtained from data of identical sensors, feature level fusion can be achieved by computing a single resultant feature vector from the weighted average of the individual feature vectors. However, in the case of non-homogeneous feature sets, e.g., features obtained from data of dierent sensors, this level of fusion can be attained by combining them into a single concatenated feature vector.

In cases where there are dierences in the range of values and distribution of the individual feature vectors, a feature normalization procedure that consists of modifying the scale of the values of the features is applied in order to map them into a common domain.

26

Multi-sensor data fusion and feature extraction for forest applications