[Ward and Simmons 2004]
HDR JPEG HDR JPEG
[Ward and Simmons 2004]
[Ward and Simmons 2004]
HDRimage = TM * RI
• Backward compatible JPEG encoding
–Tone mapped original (TM) accompanied by a ratio image (RI) needed to recover full HDRimage
–RIcarried in a subband of a standard 24-bit RGB format
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
• The ratio image RIis often downsampled to RId
• Distortion reduction –Pre-correction of foreground
image
–Post-correction of ratio image TM RI RId
[Ward and Simmons 2004]
HDR JPEG HDR JPEG
[Ward and Simmons 2004]
[Ward and Simmons 2004]
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
HDR MPEG Encoding [Mantiuk et al. 2004]
HDR
HDR MPEG MPEG EncodingEncoding [[MantiukMantiuket al. 2004]et al. 2004]
• Detail level 1: Input & Output
• White: MPEG
• Orange: HDR Encoder HDR
LDR bitstream
Video encoder
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
HDR MPEG Encoding [Mantiuk et al. 2004]
HDR
HDR MPEG MPEG EncodingEncoding [Mantiuk[Mantiuket al. 2004]et al. 2004]
• Detail level 2: Color Transform
bitstream
• White: MPEG
• Orange: HDR Encoder
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
HDR MPEG Encoding [Mantiuk et al. 2004]
HDR HDR MPEG MPEG EncodingEncoding [[MantiukMantiuket al. 2004]et al. 2004]
• Detail level 3: Edge Coding
Color
• White: MPEG
• Orange: HDR Encoder
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
Encoding of ColorEncoding of Color Encoding of Color
Color
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
Encoding of Color Encoding of Color Encoding of Color
• How to represent color data?
– Floating Points – ineffective compression – Integers – ok, but require quantization
• How to quantize color data?
–Quantization errors < threshold of perception –Use uniform color space (L*u*v*, L*a*b*) [Ward98]
–Find minimum number of bits
• Color (u*v*) – 8 bits are enough
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
Encoding of LuminanceEncoding of Encoding of LuminanceLuminance
• How to quantize luminance?
–Gamma correction? Logarithm?
Integer representation
log Luminance Y
-4
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
Threshold Versus Intensity Threshold Versus Intensity Threshold Versus Intensity
• Psychophysical detection measurements –The smallest perceivable difference ∆Y for a
certain adaptation level YA –tvi[Ferwerda96, CIE 12/2.1]
YA- Adaptation Luminance
log Adaptation Luminance YA
log Threshold ∆Y
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
Luminance QuantizationLuminance Quantization Luminance Quantization
• Align quantization errors with thresholds of perceivable contrast using tvi(YA)
f
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
Luminance Quantization
Integers Lp
log Luminance Y
f Just below threshold
of perception
• Capacity function [Ashkihmin02]
• Grayscale Standard Display Function [DICOM03]
• 10 – 11 bits are enough
Maximum
quantization error
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
Luminance Quantizations
log Contrast Threshold
log Adapting Luminance
-4 -2 0 2
32-bit LogLuv cvi[Ferwerda96]
11-bit percep. quant.
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
Edge Coding Edge Coding Edge Coding
Color
Tran. Comp.
Motion Coding DCT
Run-length Edge
Coding LDR
HDR
bitstream Variable
length
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
Edge Coding: MotivationEdge Coding Edge Coding: : MotivationMotivation
• HDR video can contain sharp contrast edges –Light sources, shadows
• DCT coding of sharp contrast may cause high frequency artifacts
DCT coding Edge coding
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
Edge Coding: Solution Edge Coding
Edge Coding: : SolutionSolution
• Solution: Encode sharp edges in spatial domain, the rest in frequency domain
Original Signal
Sharp edge signal
Smoothed signal
ÎRun-length encoding ÎDCT
encoding
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
Edge Coding: AlgorithmEdge Coding Edge Coding: : AlgorithmAlgorithm
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
Results Results Results
• 2x size of tone-mapped MPEG-4 video
• 20-30x saving compared to intra-frame compression (OpenEXR)
27 1
0.55
OpenEXR HDR Enc.
MPEG-4
Bit-stream Size
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
Example Application:
HDR Video Player Example Application:
Example Application:
HDR Video Player HDR Video Player
• Tone mapping adjusted to display device
• Inspecting various luminance ranges with a linear luminance mapping
• Physically based post-processing effects:
blooming, motion blur, night vision –Require HDR information
–Computed on-the-fly by graphics hardware –Can be scripted through annotations in
video stream
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
HDR Video Player: Glare HDR Video Player: Glare HDR Video Player: Glare
Glare switched off
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
HDR Video Player:
Motion Blur
HDR Video Player:
HDR Video Player:
Motion Blur Motion Blur
LDR HDR
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
HDR Video Player:
Night Vision
HDR Video Player:
HDR Video Player:
Night Vision Night Vision
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
Summary: HDR Video Compression
Summary:
Summary: HDR HDR VVideo ideo CompressionCompression
• Modest changes to MPEG-4 –Lpu’v’ color space
ÎLuminance quantization (10-11 bits) –Edge coding
• Backward compatible HDR MPEG-4?
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
Visible Difference Metric Visible Difference Metric Visible Difference Metric
• Can the human eye see the differences between two images?
Metric
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
• Lossy image compression and broadcasting
• Design of image input/output devices –scanners, cameras, monitors, printers
• Watermarking
• Computer graphics, medical visualization
Application ExamplesApplication Examples Application Examples
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
Subjective vs. Objective Methods
Subjective vs. Objective Subjective vs. Objective Methods
Methods
• Subjective methods involving human subjects –Very costly
• Simple objective metrics e.g. RMS Error –Not reliable
• Basic characteristics of the Human Visual System (HVS) must be modeled to improve difference prediction reliability:
+Luminance
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
74
Michelis max ≈
= +
Photoreceptor ResponsePhotoreceptor Response Photoreceptor Response
Luminance of light falling on receptive field’s center Luminance of light falling on receptive field’s center
Neural response Neural response (impulses per second)(impulses per second)
spontaneous
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
74
Michelis max ≈
= +
Luminance of light falling on receptive field’s center Luminance of light falling on receptive field’s center
Neural response Neural response (impulses per second)(impulses per second)
spontaneous new adaptation level new adaptation level
½ Rmax
new response
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski J.G. Robson CSF chart
Contrast Sensitivity Function Contrast Sensitivity Function
EG 2005 Tutorial 7: HDR Techniques in Graphics Karol Myszkowski
• Strong masking:
similar spatial frequencies
• Weak masking:
different orientations
• Weak masking:
different spatial frequencies