Natural Image Statistics: Foundations and Applications
90
0
0
Fulltekst
(2) Syllabus Introduction - Tania (5 mins) Foundations - Tania (25 mins) Fourier statistics - Douglas (30 mins) Wavelets - Erik (10 mins) Color statistics - Erik (15 mins) Discussion - All (5 mins). Friday, 8 February 13.
(3) Introduction. Friday, 8 February 13.
(4) Introduction. Friday, 8 February 13.
(5) Friday, 8 February 13.
(6) Friday, 8 February 13.
(7) Friday, 8 February 13.
(8) Friday, 8 February 13.
(9) Statistics in Graphics. Friday, 8 February 13.
(10) Types of Statistics First order Each pixel viewed independently. Second Order Relations between pairs of pixels. Higher Order How does a pixel relate to more than one other pixel in the image? Friday, 8 February 13.
(11) Foundations Tania Pouli. Friday, 8 February 13.
(12) First Order Each pixel considered independently Location invariant Easy to compute & interpret. First order statistics: histogram moments contrast* Friday, 8 February 13.
(13) Intensity Histograms How often does each intensity value occur? Histograms can be very important in data analysis, image analysis, and visualization. Individual frequency of occurrence Friday, 8 February 13. Cumulative frequency of occurrence.
(14) Histogram Moments They give information about the shape of the distribution A measure of Gaussianity. mk =. N X (I(p) p=1. Friday, 8 February 13. N. c)k.
(15) Histogram Moments 1st moment: mean 2nd moment: variance. Friday, 8 February 13. 3rd moment: relates to skewness S =. m3. 4th moment: relates to kurtosis. m4. =. 3. 4.
(16) Intensity Histograms Probability. Linear Log. Pixel Intensity. Local contrast in natural images: normalization and coding efficiency - Brady & Field (2000) Friday, 8 February 13.
(17) Intensity Histograms LDR. Density. HDR. Log Intensity. Log Luminance. LDR vs HDR histograms for the same scenes Pouli et al. (2010) Friday, 8 February 13.
(18) Histograms relate to surface properties:. 1. –2 0 1 10 100 Diffuse reflectance (%). Skewness. 2. 4 3 2. 0. –2 0.1. 1 1 10 Specular intensity. Glossiness rating. 3 0. 3. 4. Lightness rating. Skewness. 3. 0 100. Image statistics and the perception of surface qualities - Motoyoshi et al. (2007) Friday, 8 February 13.
(19) Histogram Equalization Images do not use intensity values evenly Equalizing the values in the cumulative histogram can improve contrast Friday, 8 February 13.
(20) Histogram Matching. L a b. Source. Source. Friday, 8 February 13. Target. Target. Result. Result.
(21) Is 1st Order Enough? Simple to compute and interpret BUT... No spatial information No information on relations between pixels. We need 2nd/higher order statistics for that! Friday, 8 February 13.
(22) Fourier Statistics Douglas Cunningham. Friday, 8 February 13.
(23) Spectral Slope Fourier Transform Basics Real Space. Fourier Space. ⇔ ν. x Real Space. Fourier Space. ⇔ x Friday, 8 February 13. ⇔ x. ν.
(24) Spectral Slope Fourier Transform Basics Real Space. ⇔ x. Friday, 8 February 13. x.
(25) Edge Effects. Cosine Friday, 8 February 13. Two spikes.
(26) Sum of Sines.... I(x,y) = sin (2 (nx/2 - x). 1 sine. 2 sines. I(x,y) = sin (2 (nx/2 - x) + sin ( (nx/2 - x). I(x,y) = sin (2 (nx/2 - x) + sin ( (nx/2 - x) + sin ( Friday, 8 February 13. (nx/2 - x). 3 sines.
(27) ...Approximates an Edge. Friday, 8 February 13.
(28) log Power. Power Spectrum Power spectrum. 2. 1. 0. Friday, 8 February 13. 0. 1 2 3 Spatial Frequency (log cycles/images).
(29) Spectral Slope Spectra of individual images varies AVERAGE spectra follow power law: 1 A⇡ f. Humans most sensitive to slopes between 2.8 and 3.2 Friday, 8 February 13.
(30) Spectral Slope Study Burton G. J., Moorhead I. R. 1987 Dong D., Atick J. 1995 Dror R. O., Adelson E. H., Willsky A. S. 2001 Field D. J. 1987 Field D. J. 1993 Field D. J., Brady N. 1997 van Hateren J. 1992 Huang J., Mumford D. 1999 Pàrraga C. A., Brelstaff G., Troscianko T. 1998 Reinhard E., Shirley P., Troscianko T. 2001 Ruderman D., Bialec W. 1994 van der Schaaf A., van Hateren J. 1996 Thomson M., Foster D. 1997 Tolhurst D. J., Tadmory Y., Chiao T. 1992 Torralba A., Oliva A. 2003 Webster M., Miharaya E. 1997 Friday, 8 February 13. # of Images. β±1sd. 19 320 95 6 85 20 117 216 29 133 45 276 82 135 12,000 48. 2.1±0.24 2.30 2.29 2.0 2.20 2.20±0.28 2.13±0.36 1.96 2.22±0.26 1.88±0.42 1.81 1.88±0.42 2.38 2.4±0.26 2.08 2.26.
(31) Spectral Slope Spectral slope is related to autocorrelation. β=0.8. β=1.6. β=2.4. β=3.2. Increasing slope increases coarseness Self similar (fractal). Friday, 8 February 13.
(32) Edge Effects. Cosine Friday, 8 February 13. Edge Effects.
(33) Windowing. Friday, 8 February 13.
(34) Angular Power Spectrum Angular power spectrum. 0.8 0.4 0.0 -0.4 -0.8 0. Friday, 8 February 13. 50. 100 150 200 250 300 350 Spatial Orientation (Degrees).
(35) Applications: Scene Classification Spectral Slope differs by scene. Forest (2.15). Close-up (2.23). distant meadow (2.4). Webster & Miyahara (1997) Friday, 8 February 13.
(36) Applications: Scene Classification and orientation Horizontal. 1.98. Oblique. Vertical. 2.02. 2.22. Natural. Torralba & Oliva (2003) Friday, 8 February 13.
(37) Applications: Scene Classification and orientation and orientation by scene Horizontal. 1.98. 1.83. Oblique. Vertical. 2.02. 2.22. Natural. 2.37. Man Made. 2.07 Torralba & Oliva (2003). Friday, 8 February 13.
(38) Applications: Scene Classification and orientation by scene. Torralba & Oliva (2003) Friday, 8 February 13.
(39) Applications: Scene Synthesis. © Ken Musgrave Friday, 8 February 13.
(40) Applications: Scene Synthesis Height map for terrain. Deussen Friday, 8 February 13.
(41) Applications: Scene Synthesis Plant modeling. Weber Friday, 8 February 13.
(42) Applications: Deblurring Resulting photograph. Camera shakes. Friday, 8 February 13. Real World.
(43) Applications: Deblurring Resulting photograph. ? Camera shakes. Friday, 8 February 13. ? Real World.
(44) Applications: Blind motion Deconvolution Constraint: Real world image must follow power law (Caron et al, 2002; Jalobeanu et al, 2002; Neelamani et al, 2004). Constraint: Estimate Blur by optimizing to match real gradient distributions Before. Friday, 8 February 13. After.
(45) Applications: Image Inpainting. Remove an unwanted object from an image Fill in hole by copying from elsewhere in image Match based on spectra (and other) information (Hirani & Totsuka, 1996) Noisy. After 1 iteration. a After 10 iterations. Noisy. Friday, 8 February 13. After 2 iterations. After 10 iterations.
(46) Wavelets Erik Reinhard. Friday, 8 February 13.
(47) Phase Structure. Friday, 8 February 13.
(48) Wavelets Phase spectra are computed over entire images What about spatially localized analysis? Wavelets do this They are also selective to specific orientations and scales. Friday, 8 February 13.
(49) Gabor Filters. Image filtered with with Gabor filters with different wavelength parameters: Left to right: 16, 32 and 64. Top: Filtered results. Bottom: Gabor filter.. Image filtered with with Gabor filters with different wavelength parameters: Left to right: 16, 32 and 64. Top: Filtered results. Bottom: Gabor filter.. Sinusoids weighted by Gaussians. Friday, 8 February 13.
(50) Haar Decomposition. Friday, 8 February 13.
(51) Haar Decomposition. Friday, 8 February 13.
(52) Haar Decomposition. Friday, 8 February 13.
(53) Coefficient Histogram 10 7. HDR Haar Wavelet Coefficient Histogram. 10 6 10 5 10 4 10 3 10 2 10 1 10 0. Friday, 8 February 13. -0.8. -0.4. 0. 0.4. 0.8.
(54) Wavelet Analysis. Distributions of histograms of wavelet coefficients have high kurtosis, i.e. long tails Can be modeled with a Laplacian. Friday, 8 February 13.
(55) Meaning of High Kurtosis Many natural image statistics end up showing high kurtosis This means that lots of values are small and some are large Effectively sparse coding. Friday, 8 February 13.
(56) Sparse Coding In human vision, sparse coding is an important feature: Variability of input is explained by fewer neurons Metabolic efficiency Minimizes wiring length Increases capacity in associative memory. Friday, 8 February 13.
(57) Wavelet Analysis Both phase and amplitude can be measured and correlated in a wavelet decomposition Surprising result: natural images are scaleinvariant in both phase and amplitude. Friday, 8 February 13.
(58) Correlation. Complex Wavelet Amplitude Amplitude correlations. 1. Friday, 8 February 13. 2. 3. 4. 5. 6. 7 8 9 Wavelet Scale.
(59) Correlation. Complex Wavelet Amplitude. Phase-scrambled image. Friday, 8 February 13. Amplitude correlations. 1. 2. 3. 4. 5. 6. 7 8 9 Wavelet Scale.
(60) Applications of Wavelets Image denoising Image compression Object detection Image retrieval. Friday, 8 February 13.
(61) Image Denoising. Friday, 8 February 13.
(62) Face Detection. Viola & Jones use a small set of waveletlike features to detect faces. Friday, 8 February 13.
(63) Wavelet Reconstruction. Friday, 8 February 13.
(64) Color Statistics Erik Reinhard. Friday, 8 February 13.
(65) Light Transduction Lo ( ) L M S. Friday, 8 February 13. =. Z. Le ( ) + Z = Lo ( ) Z = Lo ( ) Z = Lo ( ). Li ( ) fr ( ) cos(⇥)d! ⌦. ¯l( ) d m̄( ) d s̄( ) d.
(66) Implications Metamerism: different spectra integrate to the same cone responses, and are therefore perceived identically This allows us to build color displays, for instance Color statistics can be collected on tristimulus values, rather than color spectra Friday, 8 February 13.
(67) Color Constancy. Humans can discount the color of the illumination Friday, 8 February 13.
(68) Color Constancy. Cannot be computed analytically from retinal input; it is an under-constrained problem Human vision makes statistical assumptions. Friday, 8 February 13.
(69) Statistical Assumptions Grey world: Spectrum of light sources usually off-white Average BRDF of a scene often close to grey Average color of an image yields estimate of dominant illuminant. Friday, 8 February 13.
(70) Grey World. Friday, 8 February 13.
(71) Grey World - Failure. Friday, 8 February 13.
(72) Possible Fixes Exclude most saturated pixels from average Optionally: convert to CIELAB Compute 2D histogram on a* and b* channels Spread of histogram and distance to origin determine if color cast is likely due to illumination or reflectance. Friday, 8 February 13.
(73) White Patch Algorithm. Assume that lightest patches in the scene are neutral in color Their color therefore represents the illuminant. Friday, 8 February 13.
(74) Grey-Edge Assumption. The difference between two colored pixels tends to evaluate to grey. Friday, 8 February 13.
(75) Algorithm Selection Different white balancing algorithms tend to work best on specific types of images Can therefore collect statistics on the image pixels and select an appropriate algorithm based on the outcome Weibull distribution is shown to be indicative Friday, 8 February 13.
(76) A Further Implication r e b em. Rem For grey values, in RGB (as well as LMS and similar color spaces) we have R=G=B If values average to grey, then in RGB-like color spaces strong correlations exist between channels. Friday, 8 February 13.
(77) Statistical Decorrelation Tania Pouli, Douglas W. Cunningham and Erik Reinhard / Image Statistics and their Applications in Computer Graphics. -4 -6. Red - Green Red - Blue Green - Blue. Friday, 8 February 13. 0 -2. Red - Green Red - Blue Green - Blue. econd channel. -2. econd channel. econd channel. Figure 18: Examples images used to demonstrate the correlation between channels. The first two images are reasonable ex amples of natural images, whereas the third image is an example of an image taken in a built-up area. Built environments ten to have somewhat different natural image statistics compared with natural scenes [ZL97, ZL98]. Figure taken from [RKAJ08] courtesy AK Peters, Ltd. 0 -1 -2. Red - Green Red - Blue Green - Blue.
(78) Tania Pouli, Douglas W. Cunningham and Erik Reinhard / Image Statistics and their Applications in Computer Graphics. Correlations in RGB/LMS. -6. Red - Green Red - Blue Green - Blue. -4 -6. -12. -8. -10 -9 -8 -7 -6 -5 -4 -3 -2 First channel 1.6 ll1.2 0.8. Red - Green Red - Blue Green - Blue. -2. -10. Second channel. Second channel. -8. 0. Second channel. -4. 0 Red - Green Red - Blue Green - Blue. -1 -2 -3. -6. -5. -4. -3. -2. 1.6. ll-. 1.2. -4. -1 0 1 First channel. 0.8. Second channel. -2. Second channel. Second channel. Figure 18: Examples images used to demonstrate the correlation between channels. The first two images are reasonable examples of natural images, whereas the third image is an example of an image taken in a built-up area. Built environments tend to have somewhat different natural image statistics compared with natural scenes [ZL97, ZL98]. Figure taken from [RKAJ08], courtesy AK Peters, Ltd.. -3.5 -3. 0.8. 0.4 0.2. 0.0. 0.0. 0.0 -8. -6. -4. -2 0 2 First channel. ll-. 0.4. -10. -1.5 -1 -0.5 0 First channel. 0.6. 0.4. -16 -14 -12 -10 -8 -6 -4 -2 0 2 First channel Friday, 8 February 13. -2.5 -2. -6. -5. -4. -3. -2. -1 0 1 First channel.
(79) -6. -4 -6. -12. -8. -10 -9 -8 -7 -6 -5 -4 -3 -2 First channel 1.6 ll1.2 0.8. Red - Green Red - Blue Green - Blue. -2. -10. Second channel. Second channel. -8. 0. 0. Red - Green Red - Blue Green - Blue. -1 -2 -3. -6. -5. -4. -3. -2. 1.6. ll-. 1.2. -4. -1 0 1 First channel Second channel. Red - Green Red - Blue Green - Blue. -4. Second channel. Color Opponent Space. -2. Second channel. Second channel. Figure 18: Examples images used to demonstrate the correlation between channels. The first two images are reasonable examples of natural images, whereas the third image is an example of an image taken in a built-up area. Built environments tend to have somewhat different natural image statistics compared with natural scenes [ZL97, ZL98]. Figure taken from [RKAJ08], courtesy AK Peters, Ltd.. 0.8. -3.5 -3. 0.8. 0.4 0.2. 0.0. 0.0. 0.0 -8. -6. -4. ll-. 0.4. -10. -1.5 -1 -0.5 0 First channel. 0.6. 0.4. -16 -14 -12 -10 -8 -6 -4 -2 0 2 First channel. -2.5 -2. -2 0 2 First channel. -6. -5. -4. -3. -2. -1 0 1 First channel. Figure 19: Random samples plotted in RGB color space (top) and L ⇥ color space (bottom). The top to bottom order of the plots is the same as the order of the images in Figure 18. Figure taken from [RKAJ08], courtesy AK Peters, Ltd.. The measure D⇤ = D/⇤ can be used to assess the strength of the cast. If the spread of the histogram is small, and lies far away from the origin, the image is likely to be dominated by strong reflectances rather than illumination. Friday, 8 February 6.1.2. 13 Generalised Grey-World and White Patch. cial instances of the Minkowski norm [FT04]: Lp =. f p (x)dx dx. ⇥1/p. = ke. (44).
(80) Statistical Decorrelation. Tania Pouli, Douglas W. Cunningham and Erik Reinhard / Image Statistics. For 200 shown in The point space wh showing t most com case for th. Lαβ color space. The co in Figure rate chann ⇥ channel minance v image sho setting bo. decorrelated - values of pixels in one channel do not predict the values in another. The fa decorrela compositi a beautifu. It also to the fac lar, we hi to transfe basis of it ture of co cated thre a colour decorrela histogram Section 6 used here. L - luminance α,β - opponent channels Friday, 8 February 13. Figure 20: The top-left image is decomposed into the L channel of the L ⇥ color space, as well as L + and L + ⇥. 6.2.1. Co.
(81) Histogram Matching. r e d in. Rem. L a b. Source. Source. Friday, 8 February 13. Target. Target. Result. Result.
(82) Color Transfer. Color transfer between images (Reinhard et al.2001). Friday, 8 February 13.
(83) Histogram Reshaping. Friday, 8 February 13.
(84) Histogram Reshaping. Friday, 8 February 13.
(85) Histogram Reshaping. Friday, 8 February 13.
(86) Conclusions. Friday, 8 February 13.
(87) Statistics There are many ways to transform images, after which we can compute statistics When we transform images according to how we think the human visual system operates, we end up with highly kurtotic and sometimes independent representations Sparse coding is good for human vision, and probably good for solving engineering problems Friday, 8 February 13.
(88) Applications Many applications already known Object detection Compression Deblurring Inpainting Color transfer etc. Friday, 8 February 13.
(89) Applications Hopefully, as our knowledge of our environment increases, there will be many more to come Graphics, computer vision and image processing are prime areas of research that we think may benefit from natural image statistics. Friday, 8 February 13.
(90) Questions?. Friday, 8 February 13.
(91)
RELATERTE DOKUMENTER
Her current research focus within image analysis is on deep learning for applications with complex image data, such as medical images, seismic data, and marine observation data,
Software applications are developed for virtual bronchoscopy, involving usually segmentation of the tracheobronchial tree from the medical image scan, which is a
Using Principal Components Analysis, we find the principal axis of each cluster. The length of the line is limited by the range of the cluster. The foreground f and background b
Jaume Rigau, Miquel Feixas, and Mateu Sbert / Applications of Information Theory to Computer Graphics test image in Fig.. Visual comparison is
Since we only use the local decision of neighbors in our la- beling scheme, we can handle the inhomogeneous texture in images, and the result is shown in Figure 15(c) Moreover, we
We present an approach to object detection and recognition in a digital image using a classification method that is based on the application of a set of features that include
Here we aim to identify a set of image measurements that can capture this “naïve visual impression of art”, and use these features to automatically cluster a database of images
While the biquadratic B-spline filtering is the best alternative for the synthesis if the maxification filter of drawImage is a bilinear interpolation (e.g. in the case of Chrome
