Limitation of the dataset

Unfortunately the dataset is composed only of image-sets from 11 patients; however, only 10 patients were adopted during the training of the distinct models: patient with ID 1 was not included due to the peculiar structure of its rearranged CTP images which have a different number of groups compared to its manually annotated images. This problem leads to confusion during the association of the different rearranged groups with the manually annotated images; hence, a decision was made to exclude the patient 1 from the dataset.

The sample size is not adequate for any DNN because a model needs to be trained with a large and suitable number of elements. For this reason, an augmented dataset is also generated from the current one. A series of vectors create the dataset used during the training of the various methods. A vector is composed of 30 small portions of 16x16 pixels, called tile, plus an output. The 30 tiles refer to the first image in the time-series, plus the same part of the other 29 images in the time-series. The vectors dataset is created using a sliding window technique, Fig. 3.9 represents a visual explanation of the sliding window technique method to overcome the dataset limitation. Starting from position [0,0] in the image until position [512,512] with a stepping size of 4 pixels, with a sliding window technique, a red square, which represents the 16x16 tile, is extrapolated from each image plus the image containing the annotation region. The final result of this step is a training vector that contains (16x16×N+ 1), whereN is the number of brain sections per patient, the input, while the (+1) indicates the 16x16 tile extrapolated from the image with the annotated regions, which corresponds to the output.

Each vector created is labeled with an output, which is different based on the approach used. The outcome of the Tile Classification approach, described in Chap. 4, is an integer that corresponds to the region class with the most considerable number of pixels inside the tile; while the output of the pixel-by-pixel segmentation, defined in Chap. 5, is the complete vector that represents the extrapolated tile. The output integer for the Tile Classification approach is generated by summing up together the number of pixel-based on their different intensity: the pixel value with the highest number of elements is chosen as the labeled class. Classes have a selection priority, thus if two or more classes have the

Tomasetti Luca Chapter 3 Dataset & Image pre-processing

Figure 3.9: Example of the sliding window technique.

same number of pixels inside a tile, the choice is made based on the priority: first core, thenpenumbra,brain and finallybackground. The decision of this order is necessary to maintain simplicity among the process and to increase the number of core and penumbra classes. An improvement for future work could be to find another method to decide the specific class of a tile: a possible way could be to add new classes for those tiles that have a similar number of pixels for two or more classes.

At the end of the process, the number of vectors extrapolated using this technique is more suitable to train a model based on a DNN. The total number of vectors is 803453;

the number of vectors labeled with a background class is 70500, while the ones with a brain class are 596039. Vectors labeled with the penumbra class are 127692; vectors with a core label are 9222. The extraction of background tiles is limited because overweighting the dataset with too many images of the same class; thus, the number of background vectors was fixed to 500 per each brain section in the volume.

Tomasetti Luca 39 Data Augmentation

Considering that the number of core tiles is inadequate in the dataset (1.1%) compared to the other classes extrapolated, there is a chance that the trained models will not be able to detect that particular class. Besides the standard dataset, it was also created another dataset with the same method as the other one with the addition of some data augmentation techniques. For each image labeled with the core class, five different methods were applied to augment the sample size:

• Rotate the time-series images of 90, 180 and 270 degrees counterclockwise.

• The image tiles are flipped upside down.

• Flipped from left to right.

The augmented dataset is formed of 848849 number of vectors in total. 70500 are labeled with a background class, 596039 represents a brain class, 127692 belong to the penumbra class, while the remaining 55338 are vectors labeled with the core class, leading to 6.5%

of the vectors are marked with the core class.

To not create confusion for the next chapters, the augmented dataset will be called

“Dataset 2”, while the dataset without any data augmentation technique will be called

“Dataset 1”.

4

Tile Classification Approach

Figure 4.1: Focus of chapter four.

41

Tomasetti Luca Chapter 4 Tile Classification Approach

4.1 Introduction

The chapter explores three different architectures to test which one yields the best result during the classification. The structure of these methods is similar among each other except for smaller changes, explained in detail in corresponding sections. Fig.

4.2 displays an overview of the involved steps after the creation of the input until the generation of the predicted output, passing through the selection of the architecture and the post-processing steps. The three Tile Classification architectures are described in detail, highlighting the differences and the different results achieved, based on statistical information and confusion matrices. Additionally, visual examples are presented to show the predicted brain section of the patient with ID 2. All the other predicted images for all the patients, with their corresponding confusion matrices and statistical information, are presented in Appendix A.

Figure 4.2: Overview of the input and output section for the CNN architectures.

The Tile Classification approach is based on CNN architecture. The general idea is to use the dataset of 4D vector images, described in detail in Sec. 3.3.3, as the input;

then train one of the proposed models with a predefined number of epochs equal to 50 and, after that, receive as output a value that represents one of the four classes that characterize an image. Each output is transformed into a 16x16 image and merged with the other productions to create the final image. All predicted images in this chapter display pixelation due to the merging implementation of the various tiles.