Adaptive Depth Sharpening

Unsharp masking using the depth map of a scene strongly enhances cognition of the spatial distribution of objects [Luft et al. 2006]. We obtain a similar enhancement using

(b) contrast equalization tone mapping (a) global tone mapping

tone mapped image countershading result countershading profiles

tone mapped image countershading result countershading profiles

Figure 7.11: An HDR image tone mapped with two different techniques: global op-erator [Reinhard and Devlin 2005] and contrast equalization [Mantiuk et al. 2006].

Adaptive countershading automatically restores the visibility of texture details in the globally tone mapped image. The shadow boundaries, which became weak after the contrast equalization tone mapping, are automatically enhanced by adaptive counter-shading so that the brightness relations between the image areas can be well recognized.

Note that the particular style of the tone mapping operator remains unchanged.

the adaptive countershading by measuring the relation of a depth map of an image to a uniform map in place of the contrast ratios R in equation (7.3) and by using the depth map instead of the reference luminance in equation (7.5). In our approach, Figure7.12, the intensity of countershading does not only depend on the depth relations of objects, but is also guided by the visual model which prevents the appearance of unnaturally looking dark outlines over objects further behind in the scene. The visual model limits the countershading strength based on the actual image contents and prevents visible degradations.

7.7 Conclusions

Based on findings from psychophysics, we have explained how to enhance contrast in images using the Craik-O’Brien-Cornsweet illusion in a controlled way by employing the multi-resolution local contrast metric to guide the strength of enhancement and the visual detection model to prevent the appearance of objectionable artifacts. Counter-shading in most cases cannot be expected to restore the original contrast of the refer-ence, however the enhancement is well visible when profiles are well adjusted and are

7.7. CONCLUSIONS 101

(a) original image (b) depth map

(d) unsharp masking of depth (c) countershading of depth

Figure 7.12: Poor design of illumination in the scene results in a “flat” look of the image (a). Countershading using depth information (b) enhances cognition of the spa-tial distribution of objects in the scene (c). The visual models limits the appearance of countershading as halo artifacts. Unnaturally looking dark outlines may appear over objects further behind in the scene if only depth relations are considered, image (d) from [Luft et al. 2006]. Image and depth data from [Scharstein and Szeliski 2003].

masked by image contents.

We have presented an image processing tool to create countershading profiles which are individually and automatically adjusted to enhance selected image features that require such correction when compared to the reference. The same framework is also able to reintroduce lost contrast information. We have demonstrated how it can be used to enhance images using their HDR originals or the depth information as the reference.

Comparing to the results of traditional unsharp masking, the enhanced images better communicate information through contrast while the overall appearance is not distorted and the enhancement is achieved within the available dynamic range.

This research direction can be furthered by evaluating the achieved corrections in a per-ceptual experiment. Such an experiment could measure the actually perceived strength of the countershading enhancement in complex images for stimuli of different scales and given a variety of contrast references.

Chapter 8

Summary

In the following we summarize the contributions of this thesis, draw conclusions, and we end with an outlook on future work.

8.1 Conclusions

The continuing interest of this dissertation was to approach various aspects of tone mapping with a strong emphasis on human visual perception. Through this interdisci-plinary point of view, the several methods presented in this thesis successfully improve and evaluate the fidelity of tone mapping.

By approaching the human visual system as a black box we have identified the per-ceptual effects which significantly contribute to the appearance of scenes and included them in real-time tone mapping with a minimal overhead. These effects are simulated according to known behavior of human visual system with respect to the absolute lu-minance levels in a scene. This leads to an increased level of realism in the depiction of dynamic HDR contents particularly in applications for HDR video playback or real-time realistic image synthesis. Such effects convey the subjective impression of ap-pearance of night scenes and bright light sources which normally is not communicated on standard displays.

The appearance of natural images is influenced by both sensory and cognitive pro-cesses. The knowledge acquired from perception theories lets us design a computa-tional model of the anchoring theory to obtain accurate reproduction of HDR image appearance in terms of lightness. We demonstrated the application of the model to tone mapping, including difficult examples that are not well handled by other algo-rithms, and we validated the fidelity of its reproduction by successfully simulating the appearance of known perceptual illusions.

Psychophysical models of contrast perception let us investigate the quality of tone map-ping in terms of communicating original HDR contents. Such an objective evaluation gives a perceptually meaningful ranking without the burden involved in evaluations with human subjects, and furthermore permits the study of underlying reasons for bet-ter visual performance of some algorithms over others. The output of our metric can

103

be further used to guide the restoration processes.

The perceptual illusions of contrast inspired us to exploit the possibility of strong per-ceived contrast enhancement within the available dynamic range. Our adaptive coun-tershading can automatically fix any imperfections of an arbitrary tone mapping result through the use of such illusions even if the numerical result is well optimized. Our technique generalizes unsharp masking, a common enhancement tool in photography, by enabling a selective enhancement of image features of various sizes with no manual intervention. Finally, the known characteristics of human perception of contrasts let us build a supra-threshold visual detection model which assures that our enhancements do not introduce objectionable visual artifacts.

The results of presented methods show that the merge of image processing with knowl-edge of human visual perception can deliver an improved fidelity when depicting HDR contents on displays with limited capabilities. Although the contrast and luminance range of consumer displays grows rapidly, their match to the real-world seem to be still distant and a certain degree of tone mapping is necessary. In that sense, meth-ods presented in this thesis may have long lasting application in the fields of computer graphics, digital photography, video, and cinema. While many aspects of perception have been addressed in this thesis, our work motivates further research in the area. To-gether with this dissertation, we provide an Open Source software for working with HDR images and video, and we hope it will promote the HDR techniques and facilitate further developments.