Biomedical Image Processing in the Study of Living Cells

(1)

Biomedical Image Processing in the Study of Living Cells

Erlend Hodneland

Dissertation for the degree philosophiae doctor (PhD) at the University of Bergen

25.07.2008

(2)

(3)

Acknowledgement

I wish to thank my three supervisors for their support and encouragement during the PhD project. Their complementary background has been very useful in the process. My main supervisor, Hans-Hermann Gerdes, has always encouraged me and believed in my skills and also shown deep and honest fascination for the potential of image processing, which has been particularly important for my own motivation. Arvid Lundervold has during the PhD project been a constant source of encouragement by bridging the medical profession with image processing, and also through his enthusiasm for mathematics. I wish to thank my external supervisor, Xue-Cheng Tai, for numerous interesting discussions on mathematics and for keeping my mathematical skills alive in my daily medical environment. He has been very supportive, understanding and positive to my work at all stages. In the future I hope to continue the common work and further strengthen the relation with all three supervisors. I deeply appreciate the work of Roland Kauf- mann and his strong and painful efforts to keep my computer well functioning, and also Frode Meland for reading the thesis and for giving constructive and precise feedback promoting further development of the text. I also want to thank my collegues for a very positive collaboration and hopefully mutual benefits in professional discussions. I have enjoyed the close collaboration to Steffen Gurke and Nickolay Bukhoresthliev who have pushed the limits for applied image processing. I am also grateful to Knut Teigen and the sofa he bought. It has been frequently used. Furthermore, I want to thank the rest of my friends for changing my focus towards leisure and enjoyable activities outside work.

Last but not least, I would like to thank my family, and in particular my parents, for their everlasting and overwhelming support at all times.

Bergen, July 2008

i

(4)

(5)

List of publications

1.1. List of publications included in the PhD thesis

[I] Erlend Hodneland, Arvid Lundervold, Steffen Gurke, Xue-Cheng Tai, Amin Rustom and Hans-Hermann Gerdes. Automated detection of Tunnelling Nanotubes in 3D images. Cytometry Part A 69A:961-972 (2006).

[II] Xue-Cheng Tai, Erlend Hodneland, Joachim Weickert, Nickolay V. Bukoreshtliev, Arvid Lundervold and Hans-Hermann Gerdes. Level set methods for watershed image segmentation.

Scale Space and Variational Methods in Computer Vision, Proceedings of the SSVM07:178-190 (2007).

[III] Erlend Hodneland, Xue-Cheng Tai and Hans-Hermann Gerdes. Four-Color Theorem and Level Set Methods for Watershed Segmentation. Revised and resubmitted to International Journal of Computer Vision.

[IV] Erlend Hodneland, Nickolay V. Bukoreshtliev, Tilo Eichler, Steffen D. Gurke, Xue-Cheng Tai, Arvid Lundervold and Hans-Hermann Gerdes. A unified framework for automated 3D segmentation of surface-stained living cells and a comprehensive segmentation evaluation. Revised and resubmitted to IEEE Transactions of Medical Imaging.

[V] Nickolay V. Bukoreshtliev, Steffen Gurke, Erlend Hodneland and Hans-Hermann Gerdes.

Selective block of TNT formation inhibits intercellular organelle transfer between PC12 cells.

Manuscript in preparation.

[VI] Steffen Gurke, Joao F. Barroso, Erlend Hodneland, Nickolay V. Bukoreshtliev, Oliver Schlicker and Hans-Hermann Gerdes. Tunneling nanotube (TNT)-like structures facilitate a constitutive, ATP-dependent exchange of endocytic organelles between normal rat kidney cells.

Revised and resubmitted to Experimental Cell Research.

[VII] Nickolay V. Bukoreshtliev, Erlend Hodneland, Tilo Wolf Eichler, Patricia Eifart, Amin Rustom and Hans-Hermann Gerdes. Partitioning and exocytosis of secretory granules during division of PC12 cells. Manuscript in preparation.

1

(8)

2 1. LIST OF PUBLICATIONS

[VIII] Tanja K¨ogel, R¨udiger Rudolf, Andrea Hellwig, Sergei A. Kuznetsov, Thomas Sllner, Flo- rian Seiler, Erlend Hodneland, Joao Barroso and Hans-Hermann Gerdes. Distinct roles of myosin Va in maturation and exocytosis of secretory granules in PC12 cells. Revised and resubmitted to Traffic.

(9)

1.2. RELATED PUBLICATIONS NOT INCLUDED IN THE PHD THESIS 3

1.2. Related publications not included in the PhD thesis

[IX] Khanh Kim Dao, Knut Teigen, Reidun Kopperud, Erlend Hodneland, Frank Schwede, Anne E. Christensen, Aurora Martinez and Stein Ove Døskeland. Epac1 and cAMP-dependent Pro- tein Kinase Holoenzyme Have Similar cAMP Affinity, but Their cAMP Domains Have Distinct Structural Features and Cyclic Nucleotide Recognition. The Journal of Biological Chemistry.

281 (30):21500-21511 (2006).

[X] Erlend Hodneland and Knut Teigen. A simple method to calculate the accessible volume of protein-bound ligands: Application for ligand selectivity. J. Mol. Graph. Model. 26(2):429-433 (2007)

[XI] Ramadhan Oruch, Erlend Hodneland, Ian F. Pryme and Holm Holmsen. Interference by psychotropic drugs of the tight coupling of polyphosphoinositide cycle metabolites in human platelets: A result of receptor-independent drug intercalation in the plasma membrane? Ac- cepted for Biochemical Journal.

(10)

(11)

CHAPTER 2

Introduction

2.1. Medical imaging and image processing

Medical imaging has become increasingly important in bio-medical research and clinical practice, and it is the driving force in the development of modern volumetric imaging techniques [1, 2, 3, 4, 5, 6]. In medical clinical research and practice, imaging has become an essential part to diagnose and to study anatomy and function of the human body. MRI [7], X-Ray and ultra-sound are imaging techniques that noninvasively can reveal tumors and fractures with a minimal hazard to the living tissue. The data produced by the imaging techniques are challenging to quantify both due to the complexity (3D, multichannel, multimodal) but also due to the vast amount to be processed. Human resources are limited and are also impeded with inter-observer variability. Computers can perform image analysis tasks on a large amount of images with a higher degree of objectivity than humans. Especially segmentation, registration and temporal analysis are image processing tasks suitable for computer algorithms. On the single cell level, biologists generate 3D image stacks by fluorescence microscopy to better understand the dynamics of living cells. Fluorescence microscopy enables the simultaneous recording of different cellular structures down to the nanometer scale, and thus provides knowledge about the dynamic interactions in terms of cell division, cell motility and cell-to-cell communication.

2.2. Fluorochromes and fluorescence microscopy

This work is entirely dealing with images from fluorescence microscopy, and a short introduction to fluorochromes and image acquisition is therefore given in this section.

Fluorescence microscopy is based upon fluorescent reagents,fluorochromes, which upon exi- tation by specific wavelengths emit light [8, 9]. The fluorochromes are used to selectively label molecules like proteins, lipids, nucleic acids, ions and also cell populations and subcellular organelles. Normally, two types of fluorescent labeling are used in fluorescent microscopy. First, the fluorescent dyes are popular due to their relatively straightforward process of use, which involves immersing the sample in the dye or adding the dye to the specimen before imaging.

Second, the fluorescent proteins represent another type of labeling. The discovery of the green fluorescent protein (GFP) enables fusing of the GFP to a wide range of protein targets by cloning techniques. Thus, it became possible to study protein dynamics and function in living cells and localize previously uncharacterized proteins. Combined with modern light sensitive microscopy techniques, cells that express gene products tagged with GFP can be imaged over several hours to provide information on protein distribution and cell function. Several other fluorescent proteins and mutants have been discovered, ranging over the whole imaging spectrum, and fluorescent proteins can also be used in vivo to create fluorescently labeled organisms.

5

(12)

6 2. INTRODUCTION

xy plane z

time xy plane

z

Figure 2.1: 4D image acquisition. 2D focal planes are recorded and assembled into 3D image stacks taken over time, thus becoming an image sequence of 3D image stacks. Such 4D (3D + time) images represent huge amount of data.

In wide field microscopy, the whole sample is exposed to excitatory light of narrowed wave- length characteristics, and the focal planes are obtained by changing the focus. When fluorescent specimens are imaged using such a microscope, fluorescent light emitted above and below the focal plane interferes with the features in focus. Especially in 3D image analysis, requiring high accuracy, this kind of convolution problem can be destructive (see Fig. 3.7(a)) by inducing falsely detected signals that are only partially removed by deconvolution algorithms.

Point-scanning confocal microscopy reduces the convolution problem by using a point illu- mination in optical sections instead of exposing the whole sample to the excitation light. A laser beam is consecutively directed towards well-defined regions in 2D and the emitted light is passed through a physical pinhole, which also eliminates out-of-focus signals. In spinning-disc confocal microscopy, a single laser source is split into numerous light rays which illuminate the sample simultaneously. The emitted light is passed through an array of pinholes, which only allows light perpendicular to the disc to penetrate, thus eliminating out-of-focus signals. For all methods, photon sensitive sensors convert photons into electrical current as a result of the photo-electric effect. The current is enhanced in the electron multiplier, and the induced current is proportional to the light emitted from the specimen. The signal from the light emitting specimen has normally a signal to noise ratio of around 10 to 20. This can be enhanced by increasing the incident light flux intensity, but it would on the other hand lead to a higher rate of photodestruction of the fluorescent dye and also cause oxidation destructive reactions. Two- photon microscopy leads to smaller rates of photodestruction than other types of microscopy since the light excitation only occurs in the focal point where the two rays of photons meet.

The use of multiple dyes with dissimilar excitation wavelengths enables distinct recordings of the same specimen and thus simultaneously one can study different structures in the same cell.

Combinations of multichannel, 3D and temporal recordings represent a very flexible system with huge possibilities for imaging of living cells. As a consequence of this there has been a tremendous increase of data, see Fig. 2.1. Thus, the use of automated image processing for analysis is extremely useful since human resources are limited.

(13)

2.3. FUNDAMENTAL STEPS IN DIGITAL IMAGE PROCESSING 7

2.3. Fundamental steps in digital image processing

Modern, high-speed imaging facilities produce huge amount of data. Image compression is therefore used to reduce the quantity of data and at the same time preserve crucial information.

Image compression aims at removing information which is repetitive or irrelevant, and it should not significantly reduce the quality of the images [1]. Image restoration is required to improve the quality of the given image and thus restore the ”true” image as it would have appeared free from noise and artifacts. Most real imaging applications produce images with a certain amount of noise and artifacts. In fluorescence imaging, the artifacts may occur from endocytosed dyes or inhomogeneous staining, the latter is responsible for large intensity variations between the structures that are studied. Noise removal is an example of image restoration, as well as inpainting which is the task of filling in missing parts of an image. Depending on the scientific task, the aim is not restoration of the ”true” image, but rather image enhancement to obtain an image with specific qualities enabling a quantification of structures of interest. For these applications the human perception is frequently used as a gold standard since the needs are driven by human perception of quality. Edge enhancing- and ridge enhancement filters belong to this group, significantly enhancing edges or ridges compared to other structures.

Image restoration and enhancement techniques belong to low-level image processing [4].

These methods are crucial to obtain suitable images as input for high-level tasks like segmentation, classification and tracking, often combined in real applications to create algorithms for artificial intelligence and computer vision. High-level processing tries to mimic the excellent human ability for shape detection and recognition, and simultaneously takes advantage of the computers ability to process huge amount of data in an objective way. In particular, the advantage of the computer compared to human perception becomes clear in the analysis of large 3D data sets. On the other hand, the most challenging task for computerized image analysis is to combine local and global image information, simultaneously includingapriori information about the system. The computer starts with an array of numbers, and lacks the vast source of experience that humans have. Especially in complex 2D recognition and tracking tasks the advantage of humans becomes clear. Segmentation is the task of extracting regions or objects with common features. It is strictly user-defined, since the desired output depends on the demands of the user.

As an example, consider the image in Fig 2.2(a). Detection of the face (b) of the woman clearly returns another result than detection of the eyes (c). Segmentation is one of the most challenging image analysis steps due to the ill-posedness, the large variability of images and the different level of detail of the segmentation. Therefore, segmentation algorithms are often designed particularly to a special case or type of image, and the given segmentation protocol can be applied to such set of images if they share important common features like shape, intensity, viewangle, color-space, gradient and others. Segmentation can be done manually by humans, semiautomat- ically by combining the human perception and the computers computational power, or it can be performed fully automatically by computers. Commonly used segmentation algorithms are statistical methods, thresholding, active contours, region growing and watershed segmentation.

A segmentation is often followed by a classification step. The segmentation divides the image into disconnected regions, and the classification is required to assign the segmented regions

(14)

8 2. INTRODUCTION

(a) (b) (c)

Figure 2.2: Depending on the demands, a segmentation of the Lena picture (a) could be the extraction of any desired part, for instance the face (b) or the eyes (c).

into their respective classes. The classification normally relies on features extracted from the segmentation step to label every segmented object based on its descriptors. In most cases the classification requiresa priori knowledge. The knowledge base can be a set of thresholds which must be fulfilled for a given set of properties. Classification trees or neural networks represent more advanced knowledge bases, which are produced from a ground truth.

The output from segmentation and classification can be utilized for tracking, a process for identifying objects and following them in a sequence of images. Blind tracking with no directed movement of the objects is obviously more challenging than tracking of directed movements where the smoothness of the trajectory can be used as additional information. Also, tracking of deformable objects represents a more challenging task than tracking of rigid objects. Tracking is for instance used to follow the motion of clouds in meteorology, motion analysis for vehicles, in astronomy and for military applications and in biology for tracking of vesicles [4, 10] and drug delivery [11].

Tracing is a method which is associated to tracking. The tracing attempts to follow and trace thin and elongated structures. Road detection in satellite images is an example, as well as tracing of neurite extensions of neurons [12, 13]. Tracing represents a challenging task if there are large intensity variations along the structure of interest, as well as crossing and discontinuous segments. These challenges becomes clear in Fig 2.3 (left) and especially the magnified selection A (right) which contains discontinuous segments, as well as crossing-overs, bifurcation and merged neurite segments.

(15)

2.3. FUNDAMENTAL STEPS IN DIGITAL IMAGE PROCESSING 9

A

Figure 2.3: A neuron and its neurites. Tracing of neurites is a complex task due to discontinuous segments, cross-overs, bifurcation and merging of neurite segments.

(16)

(17)

CHAPTER 3

Methodological projects

The primary focus in this work has been to accomplish whole cell segmentation of surface- stained living cells in 3D, detection of organelle transfer betwen cells as well as detection of tunneling nanotubes (TNTs) connecting cells. Therefore, the image preparation has been directed at creating high-quality images enabling these tasks. In this chapter, a protocol is described for sample preparation, image acquisition, image reconstruction and enhancement, cell and TNT detection and segmentation evaluation.

3.1. Preparation and acquisition of images

3.1.1. Sample preparation. Two cell types were used in the experiments for the projects in Papers I-VIII, the spherical PC12 cells (rat pheochromocytoma cells, clone 251, Heumann et al. [14]) and the more flat NRK cells (normal rat kidney cells, Mrs. M. Freshney, Glasgow, UK). Three types of surface staining were applied to obtain images showing a strong and pronounced plasma membrane, suitable for whole cell segmentation (Chapter 3.2.4). Wheat germ agglutinin (WGA) conjugated to Alexa FluorR was most frequently used in our experiments (Papers I-V and VII) due to the simple application and the strong labeling of the plasma membrane which is obtained, see Fig. 3.2(a). WGA-Alexa Fluor^R is a dye, which is added to the LabTek^TMchambers (Nalge Nunc Int.,Wiesbaden, Germany) used for culturing of cells. The dye is a lectin that binds to a distinct sugar moiety of plasma membrane proteins. Due to constitutive internalization of small areas of the plasma membrane, a process termed endocytosis, the dye is quickly internalized in living cells. The process of endocytosis creates increasing signals from inside the cell, promoting the complexity of cell segmentation. To keep the endocytosed material at a minimum, it is therefore important that imaging occurs within the first 45 minutes after adding WGA-Alexa Fluor. The effect of endocytosed dye is shown in Fig. 3.1, where aR

PC12 cell was imaged shortly after administration of WGA-Alexa Fluor^R (a) and 3 hours later (b).

The second membrane marker which was used (Paper IV) is the PHD-YFP fusion protein which is a F-actin binding protein enriched at actin assembly sites, both on the plasma membrane and on endosomal vesicles. NRK cells were cultured in DMEM supplemented with 10% fetal calf serum. To obtain a strong plasma membrane labeling of NRK cells, they were transfected with the PHD-YFP cDNA construct as described in [15]. Transfection of NRK cells was accomplished by electroporation as described in [16]. An example of the obtained images is shown in Fig.

3.2(b), where the plasma membrane labeling is clearly visible.

In the experiments estimating phagocytosis in the presence of cytochalasine B in Paper IV, Bromphenol Blue was used to quench undesired signals from outside the cells (Chapter 4.1). The Bromphenol Blue excitation channel itself was also suitable for segmentation due to the strong

11

(18)

12 3. METHODOLOGICAL PROJECTS

(a) A PC12 cell shortly after administration of WGA.

(b) The cell in (a) 3 hours later.

Figure 3.1: Endocytosis of WGA-Alexa Fluor. The cell in (a) shows a pronounced plasma^R membrane staining. After 3 hours the same cell (b) displays strong signals from inside the cell due to endocytosed dye. The images are single focal planes from a 3D image stack recorded by spinning-disc confocal microscopy.

(a) PC12 cells stained with WGA-Alexa Fluor.^R

(b) NRK cells transfected with PHD- YFP cDNA.

(c) PC12 cells stained with Bromphenol Blue and DiI.

Figure 3.2: Different methods for plasma membrane labeling.

signal from the plasma membrane facing into the medium as well as facing towards adjacent cells, see Fig. 3.2(c).

To obtain the 3D arrangement of cells resembling tissue (Chapter 3.5 and Paper IV), PC12 cells were embedded in agarose. Briefly, low melt agarose (Carl Roth GmbH) was prepared in a 2% (w/v) solution in DMEM 10% FCS and melted at 70^◦ C. The solution was allowed to cool down to 45^◦C for 15 min. Pellets of approximately 4·10⁺⁷PC12 cells were loosesly resuspended in 500µl of growth medium supplemented with WGA-Alexa FluorR and mixed with 500µl of agarose stack solution, resulting in a 1% agarose end concentration. The latter mixture was plated in LabTek^TMchambers and allowed to cool at 4^◦C for 5 min. 3D imaging was performed immediately afterwards.

The NRK cells in Paper VI were stained with CellTracker^TMGreen CMFDA (10µM ) or Vybrant^R DiD (2−6µM) (Molecular Probes^TM, Invitrogen detection technologies, Carlsbad, CA, USA), performed as described in [17].

3.1.2. Image acquisition. For the image acquisition three different types of microscopy were applied, wide-field, point-scanning confocal and spinning-disc confocal. For experiments in

(19)

3.2. SEGMENTATION OF CELLS 13

Figure 3.3: Wide-field Olympus microscope.

Papers I-V, a wide-field Zeiss microscope was used (Carl Zeiss MicroImaging GmbH, Jena, Ger- many), equipped with a 63x Plan Apo /1.40 NA oilimmersion objective (Olympus Optical Co), a monochromator-based imaging system (T.I.L.L. Photonics GmbH, Martinsried, Germany), a tripleband filterset DAPI / FITC / TRITC F61-020 (AHF Analysetechnik AG, T¨ubingen, Germany) and a piezo z-stepper (Physik Instrumente GmbH & Co., Karlsruhe, Germany).

Point-scanning confocal microscopy was used for experiments in Paper IV, performed with a Leica TCS SP5 confocal microscope (Leica Microsystems, Mannheim, Germany). Spinning-disc confocal microscopy was applied in Paper IV, specifically a PerkinElmer spinning-disc confocal setup (PerkinElmer Ultra View TMRS Live Cell Imager, PerkinElmer Life and Analytical Sciences, Boston, MA, USA).

For both the wide-field and confocal imaging setups, the cells were analyzed in 3D by normally acquiring single focal planes 300 to 500nm apart from each other in the z-direction, spanning the whole cellular volume.

3.2. Segmentation of cells

Segmentation of cells is a major task in biological image processing. Proper cell segmentation enables quantification of biological processes on a single cell level, which is the basis for additional quantification tasks like estimation of cell volume and shape, sub-cellular distribution of signaling molecules, detection of transfer between cells, colocalization of vesicles and tracking of cells.

There are at least four different approaches for cell segmentation: segmentation of unstained cells, stained nuclei, cytoplasmically stained cells and segmentation of surface-stained cells. Each method has unique properties that can be useful in particular experimental setups. At the same time it is important to know their limiting factors in order to apply the optimal method for cell segmentation in a given experiment. In this work we have gained experience with the use of different stainings, types of microscopy and segmentation methods. In the following we describe and discuss advantages and disadvantages associated with the various approaches.

3.2.1. Segmentation of unstained cells. It is possible to perform segmentation of unstained cells [18, 19, 20, 21]. The images are then obtained from transmitted light microscopy, which has the advantage of being less harmful to the cells. However, transmitted light images often suffer from poor contrast and from strong background signals. Also, the image quality

(20)

varies significantly at different focal planes, both due to variations in the contrast but also due to the phenomenon that same cell structures can be reverted from bright to dark at different focal levels. Differential interference contrast (DIC) transmitted light images additionally suffer from a shadow effect since the exposed light is passed through a complex set of polarizers and optical filters. Due to these limiting factors it is a challenging task to obtain satisfying success rates from segmentation of transmitted light images.

3.2.2. Segmentation of stained nuclei. Segmentation of cell nuclei is well addressed in the literature [22, 23, 24, 25, 26, 27, 28, 29]. The nucleus of the cell is stained with a dye (Hoechst staining, see Chapter 3.2.5) accumulating in the nucleus, providing a strong and well defined signal. Segmentation of stained nuclei is capable of detecting the volume spanned by the nucleus, thus also the border around it. However, the nuclei segmentation is not capable of detecting the outer border of the cell, and can therefore not be regarded as a tool for cell segmentation. If the nuclei are densely packed, the segmentation often requires a post-processing step which splits under-segmented objects. This can for instance be obtained by a distance transform [27] or by controlled dilation [30]. Nucleus segmentation is particularly useful to detect the number of cells since every cell has only one nucleus. Thus, it can for instance be used to estimate the rate of cell division. Nucleus segmentation can also be used to address gene amplification [30], for evaluation of in situ hybridization signals (FISH) [23] and as initialization spots for whole cell segmentation.

3.2.3. Segmentation of cytoplasmically stained cells. The use of a fluorescent marker accumulating in the cytoplasmic regions of the cell enables a segmentation of the cell volume.

This approach is particularly useful when the aim is to provide a binary representation of the regions covered by cells. If this is achieved, it is possible to compare cell properties to those of background or to other conditions. However, similar to nuclei segmentation, the segmentation of cytoplasmically stained cells can not resolve the exact borders between adjacent cells in densely packed arrangements since there is only a very small decrease in signal intensities on the plasma membrane between attached cells with this kind of staining. This is demonstrated in Fig. 3.4(a) which shows three attached PC12 cells stained with CellTracker^TM. Clearly, the borders between the cells are not easily seen, and the segmentation in Fig. 3.4(b) was not able to distinguish between the three cells and a cell cluster is therefore returned. If the cells are only loosely connected with duplets and maybe triplets it is still possible to split the under- segmented cells by similar methods as for the nuclei segmentation, for instance by applying concavity requirements [31] or the distance transform [27]. However, if the cells are closely packed and in particular when concavity assumptions are not valid for the cell type in use, the splitting methods can not properly separate clustered cells. The borders facing outward towards the medium are still easily resolved.

3.2.4. Segmentation of surface-stained cells. A high-quality surface staining offers a wide range of opportunities for segmentation and analysis. The surface staining produces a pronounced signal from the plasma membrane of every stained cell, thus creating a closed contour with high intensities (Figs. 3.2(a) - 3.2(c)) which can be utilized for whole cell segmentation. A

(21)

3.2. SEGMENTATION OF CELLS 15

(a) Cytoplasmic staining.

(b) Segmentation of (a).

(c) Surface staining. (d) Segmentation of (c).

Figure 3.4: Different types of stainings and cell segmentation.

whole cell segmentation implies that each cell is segmented separately, disconnected from the other segmented cells. Figure 3.4(c) shows surface-stained cells with the corresponding whole cell segmentation in Fig. 3.4(d). The cell area has been resolved similar to the segmentation for the cytoplasmic staining in Fig. 3.4(b), and additionally, every cell in the triplet is detected and represented individually. To the best of our knowledge only very few studies address the task of segmentation of surface-stained cells. Notably, in these studies only fixed cells were used. In one study antibodies against integrin receptor subunits were used as surface markers to label the contour of fixed cells [24]. However, this method labels only the cell membrane attached to the substrate, which restricts the analysis to 2D. Furthermore, the initialization seeds inside cells had to be placed interactively for the commonly irregularly shaped cells. The study of Baggett et al. [32] also reported on whole cell segmentation of surface-stained fixed cells. They used Oregon Green 488 Phalloidin to label the peripheral F-actin cortex of cells and obtained a high success rate for 2D images by initiating the algorithm interactively. Dow et al. [33]

performed a 2D segmentation of cell boundaries from cells of fixed biopsy material of melanoma patients. They circumvented the problem of initiating markers for their snake algorithm by using the information provided by Hoechst-stained nuclei. To extend the existing methods toward a fully automated 3D segmentation of surface-stained living cells, we have developed a unified framework for whole cell segmentation in 3D of surface-stained living cells which applies to images showing fluorescently labeled cell borders (Paper IV). In particular, the use of living cells instead of fixed cells enables the study of cell dynamics. The algorithms have been applied to two distinct cell types, using different methods for cell surface labeling and various imaging techniques to demonstrate the versatility of the method.

3.2.5. Combining staining techniques. The marker construction, the whole cell segmentation and the classification step can all be performed on the surface-stained images. How- ever, other available stainings can additionally be used to improve the segmentation. An optimal solution can be a combination of a surface staining, a nucleus staining and a cytoplasmic staining. The nucleus staining labels the nuclei fluorescently with a one-to-one relationship to the cells. Thus, it offers an approach to compute binarized representations of the nuclei, where each binary region is inside one cell. This binarized representation can serve as a highly suitable marker image for further segmentation of the cells shown in the surface-stained image, either using watershed segmentation (Chapter 3.3.5) or active contours (Chapter 3.3.6). Background

(22)

Figure 3.5: Flow scheme for segmentation of surface-stained cells. The marker image can be created either from the surface-stained image itself or from the nucleus staining. A background marker is advantageously constructed from the cytoplasmic staining. Based on the markers, a segmentation of the surface-stained cells is accomplished. Finally, the watershed regions can be classified with a trained classifier using properties of the segmented regions from the surface-stained image. To obtain further reliability of the cell classification, information from the nucleus image or a cytoplasmically stained image labeling the cell regions with strong signals can additionally be used to improve the classification.

markers also need to be constructed, which for instance can be accomplished by thresholding of the cytoplasmic image or the surface-stained image. Furthermore, the cytoplasmic staining is especially useful in cell classification of the regions obtained from the segmentation since the cells are labeled brighter than the background in the cytoplasmic image channel. Different combinations of stainings for segmentation are displayed in Fig. 3.5. Normally, there are less than three available image channels for segmentation since the biological applications require at least one channel, but also combinations of two channels can improve the segmentation compared to only using the surface staining. Figure 3.6 demonstrates the use of three different combinations of stainings, a surface, a nucleus and a cytoplasmic staining. The given examples show that it is possible to only use the surface-stained image for whole cell segmentation, but the use of two or even three image channels can improve the segmentation (Figs. 3.6(d) - 3.6(f)).

3.3. A unified framework for segmentation of surface-stained cells

Due to the restricted availability of image channels in fluorescent microscopy, the following section contains a description of whole cell segmentation solely based on the surface-stained images. A complementary description of the process is given in Paper IV.

3.3.1. Choosing the optimal fluorescent marker and microscopy technique. To obtain a reliable and robust scheme of processing steps for segmentation of surface-stained cells, it is important to plan the experiments carefully in close collaboration with the biologists performing the imaging. As described in Chapter 3.1.1, we have used three different types of fluorescent surface markers facilitating a segmentation of the plasma membrane, WGA-Alexa Fluor^R (dye, see Fig. 3.2(a)), PHP-YFP cDNA (transfection, see Fig. 3.2(b)) and Bromphenol Blue (dye, see Fig. 3.2(c)). WGA-Alexa FluorR is quickly endocytosed into the cells, creating strong and undesired signals from inside the cell (Fig. 3.1). The imaging should therefore be accomplished within 45min after adding the dye. PHD-YFP is a fusion protein which is

(23)

3.3. A UNIFIED FRAMEWORK FOR SEGMENTATION OF SURFACE-STAINED CELLS 17

(a) Surface staining. (b) Nucleus staining. (c) cytoplasmic staining.

(d) Segmentation only using the surface-stained image in (a).

(e) Segmentation of (a), using (b) to create markers.

(f) Segmentation of (a), classification of cells using (c).

Figure 3.6: Combining staining techniques for segmentation. All segmentation was performed on the surface-stained image (a). In (d), the markers were constructed and cell classification was performed based on information from (a). Note the asterisk which denotes a region incor- rectly classified as a cell. In (e), the nucleus staining (b) was used to create markers for the segmentation. For (f), the cytoplasmic staining (c) was used for cell classification. Both (e) and (f) are improved compared to (c) due to the additional information available from the nucleus and cytoplasmic staining.

expressed steadily over long time. Thus, the limited available recording time of WGA-Alexa FluorR is therefore not a problem in the case of PHD-YFP. However, it is more preparatory work to transfect the cells, not all cells are transfected and the expression of the PHD-YFP is uneven and also stressful to the cells. The Bromphenol Blue is toxic to the cells, and the dye is therefore not appropriate for experiments where the cells are required to survive for a longer time during and after imaging.

Choosing the right microscopy technique is also important. The unlike properties of wide- field and confocal microscopy can be used deliberately to improve the segmentation results of cells with different morphology. The flat NRK cells are preferably recorded using confocal microscopy due to its ability to achieve images from very thin optical sections. Imaging of NRK cells using wide-field microscopy results in blurry and unsharp cell boundaries since the plasma membrane of the NRK cells runs almost parallel to the optical plane. The phenomenon is displayed in Fig.

3.7(a) showing NRK cells imaged by wide-field microscopy. The plasma membrane is clearly fuzzier than those obtained by confocal microscopy in Fig. 3.7(b). Therefore, the NRK cells in Paper IV were imaged confocally. On the other hand, the spherical and higher PC12 cells are easily imaged using a wide-field setup since the plasma membrane of the PC12 cells runs perpendicular to the optical sections in large portions of the cell. In fact, imaging of PC12 cells using wide-field microscopy provides smoother images than those obtained from confocal microscopy.

(24)

(a) NRK cells imaged by wide-field microscopy.

(b) NRK cells imaged by confocal microscopy.

Figure 3.7: Comparison of wide-field and confocal microscopy for imaging of the flat NRK cells. Note that the plasma membrane is fuzzier for wide-field microscopy (a) than for confocal microscopy (b).

(a)

0 20 40 60 80 100 120 140 160

0 0.2 0.4 0.6 0.8 1

(b)

Figure 3.8: The plasma membrane is expressed as ridges in surface-stained images (a). The line profile along the dashed line in (a) has been plotted in (b), where the ridges are indicated by arrows.

3.3.2. Filtering of the raw data. Prior to any further processing steps, it is advised to conduct a filtering of the raw data as an image restoration. A suitable filtering will smooth the images and reduce the impact of artifacts that occur as noise from the technical and optical setup. More important than noise are undesired signals occuring from real biological phenomena like endocytosis (see Fig. 3.1 and also white spots inside cells in Fig. 3.8(a)) and inhomogenious staining. Normally, a qualified filtering method is designed to enhance edges and corners, thus preserving the boundaries of desired objects [34]. However, the objects in the plasma membrane- stained images of Paper IV have other target properties than sharp edges, rather strong ridges or crests high-lighting the stained plasma membrane. Thus, the method of choice must aim at preserving ridges more than edges. In 3D, the structures of interest are surfaces. Figure 3.8(a) shows two PC12 cells where the plasma membrane is clearly visible as ridges. The intensity profile along the dashed line has been plotted in Fig. 3.8(b) to emphasize the ridge property of the plasma membrane, and the peaks of the ridges along the line profile are indicated by arrows.

A set of three direct spatial filters and three PDE based filter are presented and compared to each other in Paper IV. In this thesis, these filter techniques are explained in more detail. The

(25)

image in Fig 3.8(a) was chosen as an example to demonstrate the effect of the filtering techniques that we compare.

Gaussian smoothing. Gaussian smoothing is a fast and predictable method. Mathematically, it is a convolution with the Gaussian convolution kernel

(3.3.1) gσ(x) = 1

(2πσ²)^m/2e⁻^|x|

2 2σ2

where |x|² =x² +y²+z² is the squared blur radius in 3D, σ is the standard deviation of the distribution and m is the number of variables. To obtain the Gaussian blurred image ug, the convolution kernelg_σ is embedded into an integral and applied to the image function u(x) in a small neighborhood n_γ(x),

(3.3.2) u_g(x) = (u∗g_σ)(x) = Z

nγ(x)

u(y)g_σ(x−y)dy.

In the discrete image space, a discrete Gaussian filter mask must be computed to obtain an approximation of the convolution integral. Definew(x) to be a square filter centered inx with dimensions γ >0 and letn_γ(x) be the set of coordinates within the filter. An approximation of the convolution integral can be obtained by computing [35]

(3.3.3) u_g(x)≈ X

y∈nγ(x)

u(y)w_σ(x−y).

The discrete filter w_σ(x) is computed from (3.3.1) and (3.3.2). However, more precise results can be obtained using interpolation methods for the numerical approximation of the continuous convolution integral [36]. Figure 3.9(a) demonstrates the Gaussian smoothing.

Median filtering. Median filtering is an order-statistic filter method preserving edges effi- ciently [37]. In contrast with the Gaussian filtering, it is a non-linear operation not based on convolution. Define the filter responseR(x,y) =u(y)w(x−y). In the median filter, each output pixel contains the median of the image values within the unity filterw(x−y) = 1 ∀y∈n_γ(x), (3.3.4) u_m(x) = median _y∈n_γ_(x)R(x,y).

The median filter has excellent noise-reduction capacities for salt-and-pepper noise [35]. Figure 3.9(b) shows the results from median filtering.

Directional coherence enhancement. Directional coherence enhancement [30] is a directional filter performing a texture analysis to remove noise. The method has a high resistance against noise due to a special averaging process called semi-Olympic averaging. In semi-Olympic averaging, the largest element is removed before averaging, thus removing the outliers and reducing the noise significantly. The semi-Olympic average operator s_w applied to u(x) is defined as

(3.3.5) s_wu(x) = 1

W

X

y∈nγ(x),y6=y^∗

R(x,y)

where W is the sum of the elements in a directional filter w_i, except from y^∗ providing the largest filter response, y^∗ = {y:R(x,y^∗)≥R(x,y)∀ y∈n_γ(x)}. (3.3.5) is applied to u(x) with four filters along different directions, i.e. horizontal, vertical and the two diagonals. For

(26)

(a) Gaussian filtering,γ= 11 andσ= 3.

(b) Median filtering, γ = 11.

(c) Directional coherence enhancement filter, γ= 11.

(d) Edge enhancing diffusion, k = 0.01, ∆t = 0.1 and 100 iterations.

(e) Coherence enhancing diffusion, k = 0.0001,

∆t= 0.2 and 100 iterations.

(f) Inverse diffusion, ∆t= 0.2, λ2 = −0.5 and 5 iterations.

Figure 3.9: Filtering of the image in 3.8(a) using three direct methods (a-c) and three iterative methods (d-f). The ridges are better preserved using directional coherence enhancement filter (c), coherence enhancement filter(e) or inverse diffusion (f) than using the other methods.

γ = 3 the filters are given as

(3.3.6) w₁ =







0 1 0 0 1 0 0 1 0





, w₂=







0 0 0 1 1 1 0 0 0





, w₃=







0 0 1 0 1 0 1 0 0





, w₄=







1 0 0 0 1 0 0 0 1





.

Every pixel in the smoothed image is computed as the maximum value of the semi-Olympic average in the four directions, i.e. us(x) = maxi{swiu(x)}, i= 1, . . . ,4. Note, the directional coherence enhancement operator can be applied to the gradient map when the aim is to enhance edges. In our practical applications, γ ≈11. An example of directional coherence enhancement is shown in Fig. 3.9(c).

Edge enhancement smoothing. Now, we turn to the iterative PDE based anisotropic diffusion operators [38, 39]. In regions with steep gradients, they perform a stronger smoothing along than across the edges. In flat parts, the system reduces to isotropic diffusion with nearly equal diffusion in all directions. To obtain a local estimate of the edge direction, the gradient is computed. Edge enhancing diffusion is described with the PDE

(3.3.7) ∂u

∂t =∇ ·(D∇u)

(27)

whereDis a positive semi-definit and symmetric diffusion tensor. To obtain a more homogeneous gradient map, the Gaussian smoothed u_g is used to calculateD, which in 2D is constructed as

(3.3.8) D=R^T c₁ 0

0 c₂

! R

where R is the rotation matrix

(3.3.9) R= 1

|∇ug|

(ug)x −(ug)y

(u_g)_y (u_g)_x

!

and |∇ug| = q

(ug)²_x+ (ug)²_y. The columns of R are constructed from the gradient vector [(u_g)_x (u_g)_y]^T and an ortogonal component. Equations 3.3.8 and 3.3.9 lead to the diffusion tensor D

(3.3.10) D= 1

|∇u|²

c1(ug)²_x+c2(ug)²_y (c2−c1)(ug)x(ug)y

(c₂−c₁)(u_g)_x(u_g)_y c₁(u_g)²_y +c₂(u_g)²_x

!

The conductivity parametersc₁ andc₂ control the amount of smoothing along the gradient and the orthogonal direction, respectively. Preferably, c₂ > c₁ which leads to higher diffusion along the edges than across them. If c₁ = c₂ there will be isotropic diffusion with equal amount of smoothing in all directions. One possible choice for c2 and c1 could be [40]

c₂(|∇ug|) =e⁻^|∇ug|

2

k2 and c₁(|∇ug|) =αc₂(|∇ug|), (3.3.11)

where α ∈ [0,1] and k is a tuneable diffusion parameter depending on the magnitude of the gradient. Define

(3.3.12) D= d₁₁ d₁₂

d₁₂ d₂₂

! .

Then, the equation to solve becomes

(3.3.13) ∂_tu=∂_x(d₁₁∂_xu) +∂_x(d₁₂∂_yu) +∂_y(d₁₂∂_xu) +∂_y(d₂₂∂_yu).

To stay within the 3×3 neighborhood, central differences should be used for the mixed terms, and for the same reason a backward difference followed by a forward is advised to use in the Laplacian terms. Figure 3.9(d) is an example of edge enhancing diffusion.

Coherence enhancing diffusion. The coherence enhancing diffusion has proved to be success- ful on fingerprint images [41, 39]. The pictures presented in this work and fingerprint images have the common property that important structures are thin lines or ridges. The aim for both types of pictures is to smooth along the ridges and not across them. To estimate the local orientation of the ridges, thestructure tensor is used. In 2D, it is given as

(3.3.14) S= s₁₁ s₁₂

s12 s22

!

= g∗(u²_x) g∗(u_xu_y) g∗(uxuy) g∗(u²_y)

! .

It is important to choose a suitable scale of integration and standard deviation in the convolution to preserve the desired structures. The integration scale in the Gaussian convolution should

(28)

reflect the size of the structures of interest. S is a symmetric positive semidefinite matrix, and there exists an orthonormal set of eigenvectors v₁ and v₂ with eigenvalues µ₁ > µ₂ > 0.

The eigenvector v2 gives the local orientation perpendicular to the structures, and v1 gives the orientation along the structures. The diffusion tensor D is designed as in (3.3.8), but here the columns ofR are given as the eigenvectors ofS. They can be calculated analytically, leading to the diffusion tensor

d₁₁ = 1 2

c₁+c₂+(c₂−c₁)(s₁₁−s₂₂) α

d₁₂ = (c₂−c₁)s₁₂ α d22 = 1

2

c1+c2+(c2−c1)(s11−s22) α

where α=p

(s11−s22)²+ 4s²₁₂. The eigenvalues are given as

(3.3.15) µ_1,2 = 1

2(s₁₁+s₂₂±α). The diffusion coefficients c₁ and c₂ can be chosen as [40]

(3.3.16) c1= max

β,1−e^−(µ¹^−µ²⁾²^/k²

and c2 =β

where β ∈[0,1]. The difference (µ₁−µ₂)² is a measure of the difference of gray-value changes between the eigendirections, also called the local coherence. Figure 3.9(e) shows the image in Fig 3.8(a) after coherence diffusion filtering.

Inverse diffusion. Inverse diffusion is a diffusion method where the diffusion across the edge is reversed. This is an inherently unstable process since the diffusion equation has no backward compatibility. However, if the diffusion is limited to only a few iterations, the method can sharpen the ridges or edges of interest. The flow equations are similar to the scheme for the coherence enhancement diffusion and edge enhancing diffusion. A diffusion tensorDis constructed as in (3.3.8), using the eigenvectors of (3.3.14) as column basis ofR, similar to the approach in coherence enhancement diffusion,

(3.3.17) D=R^T λ₁ 0

0 λ₂

! R.

However, the eigenvalues of D are chosen asλ₁ =−α⁻¹ and λ₂ =α for some scalar α∈ h0,1i.

The important difference to the coherence diffusion is the modification of the largest eigenvalue to become negative. This inverts the diffusion across the principal edge direction and will enhance the edges. Figure 3.9(f) displays the result after inverse diffusion of Fig. 3.8(a).

Comparison of the smoothing operators. The choice of filtering method depends on the data available. If the images are of high quality, a traditional Gaussian filter is adequate for the purpose. However, if there exists a considerable amount of noise and if the image structures are characterized by discontinuities, the directional coherence enhancement, the coherence diffusion filter or inverse diffusion are preferred methods. These filters are able to close minor gaps in the structures along the principal flow direction. On the other hand, they can due to this useful property also create artifacts and induce wrongly connected structures, which is the reason why

(29)

the Gaussian filter is preferred for images of high quality. However, in large data sets there will always exist images with varying quality. In the long run the more advanced filters are therefore to be preferred. The median filter and the edge enhancing filters are more edge preserving than the already mentioned filters, and are not well suited for structures that manifest as ridges.

These filters enhance and even sharpen edges, thus edges that were hardly visible before filtering will become sharper. Therefore, applied to images showing cells with cytoplasmic or nucleus staining, the median filter or the edge enhancing filter are highly suitable since the aim then is to enhance the edges in the transition zone between cells and background.

3.3.3. Ridge enhancement. As an image enhancement step, a ridge enhancement is rec- ommended, especially for wide-field images since light from surrounding pixels will blur the signal. The ridge enhancement increases the contrast of ridges compared to other structures, and thus creates an image better suited for cell segmentation, and in particular better suited for automated marker generation. In Paper IV we compare two methods for ridge enhancement, the Hessian approach and our proposed method, ridge enhancement by curvature.

Hessian ridge enhancement. The Hessian method for ridge enhancement is the classical approach. A convoluted Hessian matrix is given as

(3.3.18) H_g =

"

g∗(u_g)_xx g∗(u_g)_xy g∗(ug)yx g∗(ug)yy

# .

The eigenvalues are computed and every pixel is assigned a geometrical class according to the sign of the eigenvalues [42]. Across the cell boundary there is a large intensity variation but the variation is small along the tangent plane of the plasma membrane. Therefore, the 3D plasma membrane has the characteristic geometrical property of λ₁ < 0, λ₂ ≈ 0, λ₃ ≈ 0 where λ₁, λ₂ and λ3 are the eigenvalues of the Hessian matrix in decreasing absolute values. To highlight the plasma membrane, a transfer function was defined as u_H(x) =−λ1−λ²₂−λ²₃. Due to this definition, the ridge enhanced image u_H will take the largest values on the ridges compared to other geometrical classes, see Fig. 3.10(a).

Ridge enhancement by curvature. As an extension to the Hessian ridge enhancement we have introduced ridge enhancement by curvature. In ridge enhancement by curvature, the ridge enhanced image is given as the curvature in the principal direction of variation of the second derivative. Similar to the Hessian ridge enhancement, the eigenvalue decomposition of the Hessian matrix is first computed. The purpose is to obtain the eigenvector with the largest modulus of eigenvalues, pointing perpendicular to the ridge. Name this eigenvector v₁. A twice differentiable function can be expressed using the canonical parametrization r = ˜xi+f(˜x)j.

Consider the vector formula for the curvature of a parameterized curve in 1D, [43]

(3.3.19) κ= |v×a|

|v|³ = |f⁰⁰(˜x)|

(1 +f⁰(˜x)²)³²

wherevis speed,ais acceleration and ˜xis the argument alongv₁. The curvature for every pixel in the image along v₁ is computed. Thus, for any 2D or 3D image the problem is transformed into finding the local curvature of the image intensities along a 1D line parallel to v₁ passing

(30)

(a) Hessian ridge enhancement.

(b) Ridge enhancement by curvature.

(c) Directional coherence enhancement followed by ridge enhancement by curvature.

Figure 3.10: Comparing methods for ridge enhancement, the Hessian method (a) and ridge enhancement by curvature (b), applied to the image in Fig. 3.8(a). In (c), coherence enhancement diffusion was first applied to the image, followed by a ridge enhancement by curvature.

Obviously, (c) has the highest quality.

through x. A three-point derivative was used to computef⁰(˜x) and f⁰⁰(˜x) alongv₁, (3.3.20)

f⁰(˜x) = ug(x+hv1)−ug(x−hv1)

2h andf⁰⁰(˜x) =ug(x+hv1)−2ug(x) +ug(x−hv1)

h² .

for a stepsizehapproximately equal to half the width of the ridge. For our data a value ofh= 3 pixels was appropriate. A linear interpolation scheme was applied to extract the derivatives from u_g. The final ridge enhanced image by curvature is taken as u_κ(x) = κ, an example is shown in Fig. 3.10(b).

Clearly, both methods for ridge enhancement (Fig. 3.10(a) and Fig. 3.10(b)) produce strong ridge patterns, but the ridge enhancement by curvature creates even more distinct ridges. The filtering methods described in Chapter 3.3.2 produce smooth images with better properties for segmentation than the raw images, but they can not substitute the ridge enhancement.

Preferably, the best results are obtained by combining the two steps, a filtering followed by a ridge enhancement, see Fig. 3.10(c).

3.3.4. Automated marker generation. The marker-controlled watershed segmentation, watershed by level set and the active contour model require a set of markers for initialization. For the watershed segmentation, the markers are binary regions with a one-to-one correspondence to the segmented regions. This is not the case for the active contours where the segmentation around a marker can disappear. Region-markers are normally a better choice than point-markers since region-markers are closer to the desired boundaries, which improves the segmentation.

Assigning the markers manually in high-throughput experiments is extensively time consuming.

Therefore, such experiments require automated methods for marker generation as well as cell segmentation. Therefore, we have developed and applied methods for automated generation of markers on a large number of images.

In Paper I we use morphological filling to detect markers. This method fills all holes in a grayscale image, where a hole is defined as an area that cannot be reached by filling in the

(31)

background from the edge of the image. The sensitivity for filling can be increased by raising the border values of the image. The values of the pixels belonging to a hole are replaced by the mean of the respective holes (Fig. 3.11, D1). Thus, a set of constant valued regions are obtained that can be easily detected from the zero gradient (Fig. 3.11, D2). Each connected marker is closed and filled for small holes to obtain the final marker image, where each region is suitable as markers since a majority of them have a one-to-one relationship to the cells (Fig. 3.11, D3).

In Papers II-VII we use another method to detect markers automatically. An adaptive thresholding (Chapter 3.6.2) is applied to the surface-stained image, which creates a set of binary regions labeling the high-intensity structures, mainly the cell borders (Fig. 3.11, C1). Small objects are removed (Fig. 3.11, C2). Thereupon, an iterative closing is conducted to connect interrupted binary structures originating from inhomogeneous staining in the raw image. After each closing step, every hole in the binary image is detected and taken as a marker (Fig. 3.11, C3) provided that it has no intersection with previously detected markers. This last requirement improves the quality of the markers significantly by creating markers that are closer to the desired boundaries. The process of this marker construction is described in more detail in Papers II,III and IV.

3.3.5. Cell segmentation with watersheds. The watershed algorithm is briefly described in Chapter A.2 and comprehensively documented in the literature. It is normally implemented either as a flooding process [44, 45, 46, 47, 48] or by image integration [49, 50, 51, 52]

(Chapter A.2.9). We have in Paper V used the watershed algorithm for the segmentation of surface-stained cells in our projects, where real biological questions have been resolved. This is due to the high reliability of the watershed algorithm, its 3D implementation and the computational speed. The watershed segmentation has successfully been used in previous works for segmentation of stained cytoplasm [53, 54, 31] or stained nuclei [23, 30, 25, 33]. An example of watershed segmentation for whole cell segmentation of the image in Fig. 3.12(a) is given in Fig. 3.12(c) where the segmentation is represented as a piecewise constant imageu_p(x), uniquely labeling every region.

The watershed algorithm has previously been combined with the snake model or active models for segmentation of subcellular structures or whole cells [30, 23]. The watershed was then often used to create an approximation of the solution, which was then refined with an active contour. There are at least two reasons for this approach. First, the watershed is more computational efficient than the active contours and is therefore suitable to create a rough approximation to the solution. Second, the standard watershed has no smoothing term and the solution is therefore often oscillating. The active contour model can easily be implemented with a suitable smoothing operator and is therefore appropriate to fine-tune a smooth solution.

However, recent works on watershed approaches have included a smoothing term into their models [50, 55].

3.3.6. Cell segmentation with active contours. For the active contours, there are at lest two different types of models, parametrized and geometric active contours, the latter also named implicit snakes. The first type synthesizes parameterized connected curves [28] and the second type is implemented implicitly as a level curve of a higher dimensional function [24],

(32)

1

A B

C1 D1

C2 D2

C3 D3

Figure 3.11: Creating markers. The image in (A) was filtered by coherence enhancing filtering and further enhanced by ridge enhancement to obtain an image (B) well suited for automated marker generation. This image was used as input for either of the two methods for marker generation, adaptive thresholding (C1-C3) or morphological filling (D1-D3).

often called the level set. The parameterized active contours have normally faster convergence than the geometric, but the latter can better handle topology changes. In particular, they can merge or disappear.

In Paper IV we have implemented the active contours described in Solorzano et. al [24].

They report segmentation of stained nuclei and whole lamin-stained cells using a geometric snake. Consider the level set functions ψ_i, i= 1. . . n where n is the number of markers. The interface

(3.3.21) Γ_i ={x:ψ_i(x) = 0}

of the level set should overlap with the cell boundaries at convergence. Every marker is assigned a level set function. Thus, a high number of markers require many level set functions, becoming memory consuming and computationally costly to the computer. The governing equation for the evolution is a parabolic PDE

(3.3.22) ψ_t−I(1−H)· |∇ψ| −β∇G· ∇ψ= 0

(33)

(a) Raw image. (b) Marker image.

(c)up(x) from watershed.

(d)up(x) from active contours.

(e)up(x) from ridge following.

(f)up(x) from watershed level set.

(g) After cell classification of (c).

(h) After cell classification of (d).

(i) After cell classification of (e).

(j) After cell classification of (f).

Figure 3.12: Algorithms for whole cell segmentation. The image in (a) was used to create a set of markers (b), which was used as initialization regions for the segmentation methods in (c), (d) and (f). In (e), initialization points (asterices) on the ridges were used to seed the ridge following. The piecewise constant segmentation image and the classified cells are shown using watershed (c,g), active contours (d,h), ridge following (e,i) and watershed level set (f,j), respectively.

which holds for every ψ=ψi. The second term is a variant of the mean curvature flow H|∇ψ|

where H is the curvature of the level curve (A.3.13).

The marker image normally contains more than one marker, and it is therefore necessary to adapt the algorithm to handle the conflict when the interfaces of two or more level set functions meet. If more than one level set function is larger than zero in a point, the solution is not unique. This situation is undesireable, and to overcome the problem the conflict measure (3.3.23) ψ_m+1ⁱ = minn

ψ_m+1(trial)ⁱ ,−ψ^j_mo

,1≤j ≤n, i6=j

was implemented where ψⁱ_m+1(trial) is a trial version of ψ_m+1ⁱ before the conflict measure has been applied. Thus, only one ψ_i will remain larger than zero in every point. Figure 3.13 shows how the situation with no conflict (a) can develop into a situation where both level sets ψⁱ and ψ^jhave regions where they are larger than zero (b, solid and dashed-dot lines). After the conflict measure has been applied, the last updated level set function is modified to ensure there is no overlap between ψⁱ and ψ^j were both of them are larger than zero (b, solid lines). The final

Biomedical Image Processing in the Study of Living Cells