Dynamic 2D/3D Registration

(1)

Dynamic 2D/3D Registration

Sofien Bouaziz Andrea Tagliasacchi Mark Pauly

(2)

Sofien Bouaziz Andrea Tagliasacchi Mark Pauly Dynamic 2D/3D Registration For the Kinect

Presenters

2

Computer Graphics and Geometry

Mark Pauly

Sofien Bouaziz Andrea Tagliasacchi

(3)

RGB-D Sensors

3

(4)

RGB-D Sensors

4

2008

(5)

RGB-D Sensors

4

2008

(6)

RGB-D Sensors

4

2008 2010

(7)

RGB-D Sensors

4

2008 2010

(8)

RGB-D Sensors

4

2008 2010 2013

(9)

RGB-D Sensors

4

...

2008 2010 2013

(10)

RGB-D Sensors

4

...

2008 2010

size/cost

2013

(11)

Sofien Bouaziz Andrea Tagliasacchi Mark Pauly

Dynamic 2D/3D Registration For the Kinect 5

(12)

Overview

• Registration

• 3D Geometry

• 2D Images

• Combined 2D / 3D

6

• Applications

• Rigid Scanning

• Non-rigid Modeling

• Realtime Face Tracking

Introduction

Conclusions

(13)

RGB-D Sensors

7

http://www.openni.org

http://www.openni.org/openni-programmers-guide

(14)

• Scan to scan

Registration - Examples

8

(15)

• Scan to scan

Registration - Examples

8

(16)

• Scan to scan

Registration - Examples

8

(17)

• Scan to scan

Registration - Examples

9

Newcombe, Izadi, Hilliges, Molyneaux, Kim, Davison, Kohli, Shotton, Hodges, Fitzgibbon:

KinectFusion: Real-Time Dense Surface Mapping and Tracking, ISMAR 2011

(18)

• Parameterized template to scan

Registration - Examples

10

(19)

• Parameterized template to scan

Registration - Examples

10

(20)

• Parameterized template to scan

Registration - Examples

10

(21)

Pairwise Registration

• Align a source model onto a target model

11

X Y

(22)

Pairwise Registration

• Align a source model onto a target model

• find a transformation that brings into alignment with

11

X Y

T ( X )

Y

Z = T ( X )

(23)

Pairwise Registration

• Align a source model onto a target model

• find a transformation that brings into alignment with

12

X Y

T ( X )

Z = T ( X )

(24)

Pairwise Registration

• Align a source model onto a target model

• find a transformation that brings into alignment with

• Two main questions:

13

X Y

T ( X )

X Y

(25)

Pairwise Registration

• Align a source model onto a target model

• find a transformation that brings into alignment with

• Two main questions:

• How do we measure the quality of the alignment?

13

X Y

T ( X )

X Y

(26)

Pairwise Registration

• Align a source model onto a target model

• find a transformation that brings into alignment with

• Two main questions:

• How do we measure the quality of the alignment?

• What transformations are allowed / good?

13

X Y

T ( X )

X Y

(27)

• Issues

Pairwise Registration

14

Noise

(28)

• Issues

Pairwise Registration

14

Partial matching

missing data

Noise

(29)

• Issues

Pairwise Registration

14

Partial matching

missing data

Noise

Ambiguity

(30)

• Issues

Pairwise Registration

14

Partial matching

missing data

Noise

Illumination changes

Ambiguity

(31)

• Registration

• 3D Geometry

• 2D Images

• Combined 2D / 3D

Overview

15

• Applications

• Rigid Scanning

• Non-rigid Modeling

• Realtime Face Tracking

Introduction

Conclusions

(32)

Registration

• Registration as energy minimization

16

arg min

T

E _reg (T, X , Y )

source

target

transformation

(33)

Registration

• Registration as energy minimization

17

Alignment Error

How do we measure the quality of the alignment?

arg min

T

E _reg (T, X , Y )

E _reg (T, X , Y ) = E _match (T, X , Y ) + E _prior (T )

(34)

Registration

• Registration as energy minimization

18

Alignment Error

How do we measure the quality of the alignment?

Transformation Error

What transformations are allowed / good?

arg min

T

E _reg (T, X , Y )

E _reg (T, X , Y ) = E _match (T, X , Y ) + E _prior (T )

(35)

Registration

19

• Alignment Error E _reg = E _match + E _prior

E _match (T, X , Y )

(36)

distance function

=

Z

X

'(T (x), Y )dx

Registration

19

• Alignment Error E _reg = E _match + E _prior

E _match (T, X , Y )

(37)

distance function

=

Z

X

'(T (x), Y )dx

Registration

19

• Alignment Error E _reg = E _match + E _prior

X

E _match (T, X , Y )

(38)

distance function

=

Z

X

'(T (x), Y )dx

Registration

19

• Alignment Error E _reg = E _match + E _prior

Y X

E _match (T, X , Y )

(39)

distance function

=

Z

X

'(T (x), Y )dx

Registration

19

• Alignment Error E _reg = E _match + E _prior

Y X

T ( X )

E _match (T, X , Y )

(40)

distance function

=

Z

X

'(T (x), Y )dx

Registration

19

• Alignment Error E _reg = E _match + E _prior

E _match Y

X

T ( X )

E _match (T, X , Y )

(41)

distance function

=

Z

X

'(T (x), Y )dx

Registration

19

• Alignment Error E _reg = E _match + E _prior

E _match Y

X

T ( X )

E _match (T, X , Y )

(42)

distance function

=

Z

X

'(T (x), Y )dx

Registration

19

• Alignment Error E _reg = E _match + E _prior

E _match Y

X

T ( X )

E _match (T, X , Y )

(43)

Registration

20

E _reg = E _match + E _prior

(44)

Registration

20

E _reg = E _match + E _prior

rigid

(45)

Registration

20

E _reg = E _match + E _prior

elastic

rigid

(46)

Registration

20

E _reg = E _match + E _prior

elastic

articulated

rigid

(47)

Registration

20

E _reg = E _match + E _prior

elastic

articulated rigid

composite

(48)

Registration

20

E _reg = E _match + E _prior

elastic

articulated rigid

fluid

composite

(49)

Registration

21

E _reg = E _match + E _prior

elastic

articulated rigid

fluid

composite

rigid motion

prior

(50)

Registration

21

E _reg = E _match + E _prior

elastic

articulated rigid

fluid

composite

(pseudo-) elastic deformation prior rigid motion

prior

(51)

Registration

21

E _reg = E _match + E _prior

elastic

articulated rigid

fluid

composite

(pseudo-) elastic deformation prior

skeletal model prior

rigid motion

prior

(52)

Registration

21

E _reg = E _match + E _prior

elastic

articulated rigid

fluid

composite

(pseudo-) elastic deformation prior

skeletal model prior

data-driven prior

rigid motion

prior

(53)

Registration

21

E _reg = E _match + E _prior

elastic

articulated rigid

fluid

composite

(pseudo-) elastic deformation prior

skeletal model prior

data-driven prior

rigid motion prior

?

(54)

Registration

21

E _reg = E _match + E _prior

elastic

articulated rigid

fluid

composite

(pseudo-) elastic deformation prior

skeletal model prior

data-driven prior

rigid motion prior

?

(55)

Registration

22

arg min

T

E _reg (T, X , Y )

source

target transformation

• Registration as energy minimization

E _reg = E _match + E _prior

(56)

Registration

22

arg min

T

E _reg (T, X , Y )

source

target transformation

• Registration as energy minimization

• typically non-linear

E _reg = E _match + E _prior

(57)

Registration

22

arg min

T

E _reg (T, X , Y )

source

target transformation

• Registration as energy minimization

• typically non-linear

• common solution: linearize, iterate

E _reg = E _match + E _prior

(58)

Registration

22

arg min

T

E _reg (T, X , Y )

source

target transformation

• Registration as energy minimization

• typically non-linear

• common solution: linearize, iterate

E _reg = E _match + E _prior

( X , Y )

Input

(59)

Registration

22

arg min

T

E _reg (T, X , Y )

source

target transformation

• Registration as energy minimization

• typically non-linear

• common solution: linearize, iterate

E _reg = E _match + E _prior

( X , Y )

Input

E _reg

Z ¹ = T ₁ ( X )

(60)

Registration

22

arg min

T

E _reg (T, X , Y )

source

target transformation

• Registration as energy minimization

• typically non-linear

• common solution: linearize, iterate

E _reg = E _match + E _prior

( X , Y )

Input

E _reg Z ¹ = T ₁ ( X )

Z ² = T ₂ ( X )

(61)

Registration

22

arg min

T

E _reg (T, X , Y )

source

target transformation

• Registration as energy minimization

• typically non-linear

• common solution: linearize, iterate

E _reg = E _match + E _prior

( X , Y )

Input

E _reg Z ¹ = T ₁ ( X )

Z ² = T ₂ ( X ) Z ⁿ = T _n ( X )

. . .

(62)

Registration

22

arg min

T

E _reg (T, X , Y )

source

target transformation

• Registration as energy minimization

• typically non-linear

• common solution: linearize, iterate

E _reg = E _match + E _prior

( X , Y )

Input

T = T _n

Output E _reg

Z ¹ = T ₁ ( X ) Z ² = T ₂ ( X ) Z ⁿ = T _n ( X )

. . .

(63)

• Registration

• 3D Geometry

• 2D Images

• Combined 2D / 3D

Overview

23

• Applications

• Rigid Scanning

• Non-rigid Modeling

• Realtime Face Tracking

Introduction

Conclusions

(64)

3D Registration

24

(65)

3D Registration

25

source

target

(66)

3D Registration

26

source

target

transformed source

(67)

3D Registration

27

source

target

transformed source

(68)

3D Registration - Matching

28

E _match ( Z ) =

Z

'(z, Y )dz

E _reg = E _match + E _prior

(69)

3D Registration - Matching

29

E _match ( Z ) =

Z

'(z, Y )dz

E _reg = E _match + E _prior

(70)

3D Registration - Matching

30

E _match ( Z ) =

Z

'(z, Y )dz

E _reg = E _match + E _prior

(71)

E _match ( Z ) =

X n

i=1

w _i k z _i P _Y (z _i ) k ² ₂

3D Registration - Matching

31

E _match ( Z ) =

Z

'(z, Y )dz E _reg = E _match + E _prior

discretized matching cost based

on point correspondences

(72)

3D Registration - Matching

32

E _match ( Z ) =

X n

i=1

w _i k z _i P _Y (z _i ) k ² ₂

(73)

3D Registration - Matching

32

transformed point on source

E _match ( Z ) =

X n

i=1

w _i k z _i P _Y (z _i ) k ² ₂

(74)

3D Registration - Matching

32

transformed point on source

corresponding point on target

E _match ( Z ) =

X n

i=1

w _i k z _i P _Y (z _i ) k ² ₂

(75)

3D Registration - Matching

32

transformed point on source

corresponding point on target correspondence

weight

E _match ( Z ) =

X n

i=1

w _i k z _i P _Y (z _i ) k ² ₂

(76)

3D Registration - Example

• Iterative Closest Point (ICP) Algorithm

33

(77)

3D Registration - Example

• Iterative Closest Point (ICP) Algorithm

33

• Step 1: find correspondences

using closest points for fixed

transformation

(78)

3D Registration - Example

• Iterative Closest Point (ICP) Algorithm

33

• Step 1: find correspondences using closest points for fixed transformation

• Step 2: find best rigid

transformation for fixed

correspondences

(79)

3D Registration - Example

• Iterative Closest Point (ICP) Algorithm

33

• Step 1: find correspondences using closest points for fixed transformation

• Step 2: find best rigid transformation for fixed correspondences

Iterate until

convergence

(80)

3D Registration - Example

• Iterative Closest Point (ICP) Algorithm

33

• Step 1: find correspondences using closest points for fixed transformation

→ efficient data structures

• Step 2: find best rigid transformation for fixed correspondences

Iterate until

convergence

(81)

3D Registration - Example

• Iterative Closest Point (ICP) Algorithm

33

• Step 1: find correspondences using closest points for fixed transformation

→ efficient data structures

• Step 2: find best rigid transformation for fixed correspondences

→ closed form solution Iterate until

convergence

(82)

3D Registration - Example

• Iterative Closest Point (ICP) Algorithm

33

• Step 1: find correspondences using closest points for fixed transformation

→ efficient data structures

• Step 2: find best rigid transformation for fixed correspondences

→ closed form solution Iterate until

convergence

DEMO

(83)

3D Registration - Example

• Iterative Closest Point (ICP) Algorithm

• (approximate) closest points→ (more) efficient data structures

34

(84)

3D Registration - Example

• Iterative Closest Point (ICP) Algorithm

• (approximate) closest points→ (more) efficient data structures

• weight accounts for importance (area measure) and confidence

34

(85)

3D Registration - Example

• Iterative Closest Point (ICP) Algorithm

• (approximate) closest points→ (more) efficient data structures

• weight accounts for importance (area measure) and confidence

• heuristics to prune or down-weigh bad correspondences

34

(86)

3D Registration - Example

• Iterative Closest Point (ICP) Algorithm

• (approximate) closest points→ (more) efficient data structures

• weight accounts for importance (area measure) and confidence

• heuristics to prune or down-weigh bad correspondences

34

Rusinkiewicz, Levoy: Efficient Variants of the ICP Algorithm, 3D Digital Imaging and Modeling, 2001

(87)

3D Registration - Example

• Iterative Closest Point (ICP) Algorithm

35

E _match ( Z ) =

X n

i=1

w _i k z _i P _Y (z _i ) k ² ₂

(88)

3D Registration - Example

• Iterative Closest Point (ICP) Algorithm

• Side Remark: Error norm

35

E _match ( Z ) =

X n

i=1

w _i k z _i P _Y (z _i ) k ² ₂

(89)

3D Registration - Example

• Iterative Closest Point (ICP) Algorithm

• Side Remark: Error norm

• squared Euclidean distance is sensitive to outliers

35

E _match ( Z ) =

X n

i=1

w _i k z _i P _Y (z _i ) k ² ₂

(90)

3D Registration - Example

• Iterative Closest Point (ICP) Algorithm

• Side Remark: Error norm

• squared Euclidean distance is sensitive to outliers

• robust norms reduce this sensitivity

35

E _match ( Z ) =

X n

i=1

w _i k z _i P _Y (z _i ) k ² ₂

(91)

3D Registration - Example

• Iterative Closest Point (ICP) Algorithm

• Side Remark: Error norm

• squared Euclidean distance is sensitive to outliers

• robust norms reduce this sensitivity

35

Bouaziz, Tagliasacchi, Pauly: Sparse Iterative Closest Point, SGP 2013

E _match ( Z ) =

X n

i=1

w _i k z _i P _Y (z _i ) k ² ₂

(92)

3D Registration - Prior

36

elastic

articulated rigid

fluid

composite

E _reg = E _match + E _prior

(93)

3D Registration - Prior

• Global Rigidity

37

E _reg = E _match + E _prior

source transformed source

(94)

3D Registration - Prior

• Global Rigidity

37

E _prior (Z, R, t) =

X n

i=1

k z _i (Rx _i + t) k ² ₂ E _reg = E _match + E _prior

source transformed source

(95)

3D Registration - Prior

• Local Rigidity

38

E _reg = E _match + E _prior

(96)

3D Registration - Prior

• Local Rigidity

38

E _reg = E _match + E _prior

source

transformed source

(97)

3D Registration - Prior

• Local Rigidity

38

E _prior (Z, R _i | ⁿ _i=1 ) =

X n

i=1

X

j 2N ⁱ

k (z _j z _i ) R _i (x _j x _i ) k ² ₂ E _reg = E _match + E _prior

source

transformed source

(98)

3D Registration - Prior

• Local Rigidity

38

E _prior (Z, R _i | ⁿ _i=1 ) =

X n

i=1

X

j 2N ⁱ

k (z _j z _i ) R _i (x _j x _i ) k ² ₂ E _reg = E _match + E _prior

Sorkine, Alexa: As-Rigid-As-Possible Surface Modeling, SGP 2007

source

transformed source

(99)

3D Registration - Prior

• Local Rigidity

39

E _reg = E _match + E _prior

DEMO source

transformed source

(100)

3D Registration - Prior

• Local Rigidity

40

Sumner, Schmid, Pauly:

Embedded Deformation for Shape Manipulation ACM SIGGRAPH 2007

E _reg = E _match + E _prior

Bouaziz, Deuss, Schwartzburg, Weise, Pauly:

Shape-Up: Shaping Discrete Geometry With Projection

SGP 2012

(101)

3D Registration - Prior

• Linear Model

41

E _reg = E _match + E _prior

(102)

3D Registration - Prior

• Linear Model

41

E _reg = E _match + E _prior

m ^p ¹ ^p ² ^p ³ ^p ⁴

(103)

3D Registration - Prior

• Linear Model

41

E _reg = E _match + E _prior

s = Pd + m,

m ^p ¹ ^p ² ^p ³ ^p ⁴

(104)

3D Registration - Prior

• Linear Model

41

E _reg = E _match + E _prior

s = Pd + m,

m ^p ¹ ^p ² ^p ³ ^p ⁴ _{mean shape}

(105)

3D Registration - Prior

• Linear Model

41

E _reg = E _match + E _prior

s = Pd + m,

m ^p ¹ ^p ² ^p ³ ^p ⁴ _basis _{mean shape}

(106)

3D Registration - Prior

• Linear Model

41

E _reg = E _match + E _prior

s = Pd + m,

m ^p ¹ ^p ² ^p ³ ^p ⁴ _basis _{mean shape}

coefficients

(107)

3D Registration - Prior

• Linear Model

41

E _reg = E _match + E _prior

s = Pd + m,

m ^p ¹ ^p ² ^p ³ ^p ⁴

d ₄ d ₃

d ₂

+ ^d 1 + + + =

basis mean shape

coefficients

(108)

3D Registration - Prior

• Linear Model

42

E _reg = E _match + E _prior

s = Pd + m,

E _prior (Z, d) =

X n

i=1

k z _i (P _i d + m _i ) k ² ₂

(109)

3D Registration - Prior

• Linear Model

42

E _reg = E _match + E _prior

s = Pd + m,

E _prior (Z, d) =

X n

i=1

k z _i (P _i d + m _i ) k ² ₂

Blanz, Vetter: A Morphable Model for the Synthesis of 3D Faces

ACM SIGGRAPH 1999

(110)

3D Registration - Prior

• Linear Model

42

E _reg = E _match + E _prior

s = Pd + m,

E _prior (Z, d) =

X n

i=1

k z _i (P _i d + m _i ) k ² ₂

Blanz, Vetter: A Morphable Model for the Synthesis of 3D Faces

ACM SIGGRAPH 1999

Weise, Bouaziz, Li, Pauly: Realtime

Performance-based Facial Animation

ACM SIGGRAPH 2011

(111)

• Registration

• 3D Geometry

• 2D Images

• Combined 2D / 3D

Overview

43

• Applications

• Rigid Scanning

• Non-rigid Modeling

• Realtime Face Tracking

Introduction

Conclusions

(112)

2D Registration

44

? Source

Target

(113)

2D Registration

45

T

(114)

2D Registration

45

T

(115)

2D Registration

45

T

(116)

2D Registration

45

T

(117)

2D Registration

45

T

DEMO

(118)

2D Registration - Matching

46

E _match (Z ) =

X n

i=1

k I(x _i ) J(z _i ) k ² ₂

E _reg = E _match + E _prior

color value at of image

x _i I

color value at of image

z _i J

(119)

• Lucas-Kanade

• Constant displacement field

2D Registration - Prior

47

E _prior (Z ) =

X n

i=1

X

j 2N ⁱ

k (z _j x _j ) (z _i x _i ) k ² ₂

E _reg = E _match + E _prior

Lucas, Kanade: An Iterative Image Registration Technique with an Application to Stereo Vision, IJCAI 1981

(120)

2D Registration - Prior

48

• Lucas-Kanade

• Constant displacement field

E _prior (Z ) =

X n

i=1

X

j 2N ⁱ

k (z _j x _j ) (z _i x _i ) k ² ₂

E _reg = E _match + E _prior

Lucas, Kanade: An Iterative Image Registration Technique with an Application to Stereo Vision, IJCAI 1981

(121)

2D Registration - Prior

• Lucas-Kanade

• Constant displacement field

E _prior (Z ) =

X n

i=1

X

j 2N ⁱ

k (z _j x _j ) (z _i x _i ) k ² ₂

E _reg = E _match + E _prior

|| - ||

|| - || + || - || + || - || + || - || + + || - || + || - | | + | | - | |

Lucas, Kanade: An Iterative Image Registration Technique with an Application to Stereo Vision, IJCAI 1981

(122)

2D Registration - Prior

50

• Horn-Schunck

• Smooth displacement field

E _prior (Z ) =

X n

i=1

k (z _i x _i ) 1

|N ⁱ |

X

j 2N ⁱ

(z _j x _j ) k ² ₂

E _reg = E _match + E _prior

Horn, Schunk: “Determining Optical Flow”, Artificial Intelligence Journal 1981

(123)

2D Registration - Prior

51

• Horn-Schunck

• Smooth displacement field

E _prior (Z ) =

X n

i=1

k (z _i x _i ) 1

|N ⁱ |

X

j 2N ⁱ

(z _j x _j ) k ² ₂

E _reg = E _match + E _prior

Horn, Schunk: “Determining Optical Flow”, Artificial Intelligence Journal 1981

(124)

2D Registration - Prior

52

• Horn-Schunck

• Smooth displacement field

E _prior (Z ) =

X n

i=1

k (z _i x _i ) 1

|N ⁱ |

X

j 2N ⁱ

(z _j x _j ) k ² ₂

E _reg = E _match + E _prior

+ + + + +

+

( + )

- ||

|| 1/8

Horn, Schunk: “Determining Optical Flow”, Artificial Intelligence Journal 1981

(125)

• Registration

• 3D Geometry

• 2D Images

• Combined 2D / 3D

Overview

53

• Applications

• Rigid Scanning

• Non-rigid Modeling

• Realtime Face Tracking

Introduction

Conclusions

(126)

2D/3D Registration

54

?

(127)

2D/3D Registration

54

?

(128)

2D/3D Registration

55

T

Weise, Bouaziz, Li, Pauly: Realtime

Performance-based Facial Animation,

ACM SIGGRAPH 2011

(129)

2D/3D - Matching

56

E _match (Z ) = w ₁

X n

i=1

k z _i P _Y (z _i ) k ² ₂ + w ₂

X n

i=1

k I(x _i ) J(f (z _i )) k ² ₂

E _reg = E _match + E _prior

(130)

2D/3D - Matching

56

E _match (Z ) = w ₁

X n

i=1

k z _i P _Y (z _i ) k ² ₂ + w ₂

X n

i=1

k I(x _i ) J(f (z _i )) k ² ₂ E _reg = E _match + E _prior

projection of the 3D vertex

in the 2D image

(131)

2D/3D - Matching

57

f(z _i ) = h _f _z

i,x

z _i,z

f z _i,y z _i,z

i ^T

Perspective projection

f(z _i ) = ⇥

z _i,x z _i,y ⇤ ^T

Orthographic projection

E _match (Z ) = w ₁

X n

i=1

k z _i P _Y (z _i ) k ² ₂ + w ₂

X n

i=1

k I(x _i ) J(f (z _i )) k ² ₂

(132)

• Applications

• Rigid Scanning

• Non-rigid Modeling

• Realtime Face Tracking

• Registration

• 3D Geometry

• 2D Images

• Combined 2D / 3D

Overview

58

Introduction

Conclusions

(133)

Rigid Scanning

• Popular Kinect application

59

(134)

Rigid Scanning

60

(135)

Rigid Scanning

60

(136)

Rigid Scanning

60

(137)

Rigid Scanning

60

(138)

Rigid Scanning

60

(139)

Rigid Scanning

60

(140)

Rigid Scanning - Pipeline

61

(141)

Rigid Scanning - Pipeline

61

Calibration matrix K

Depth map

(142)

Rigid Scanning - Pipeline

61

Calibration matrix K

Depth map

Smoothing

(143)

Rigid Scanning - Pipeline

61

Calibration matrix K

Depth map

Smoothing Mesh & Normals

(144)

Rigid Scanning - Pipeline

61

Alignment &

Accumulation Calibration matrix K

Depth map

Smoothing Mesh & Normals

(145)

Rigid Scanning - Pipeline

61

Alignment &

Accumulation Calibration matrix K

Depth map

Smoothing Mesh & Normals

(146)

Rigid Scanning - Optimization

62

arg min

Z,R,t

w ₁

X n

i=1

k z _i P _Y (z _i ) k ² ₂ + w ₂

X n

i=1

k z _i (Rx _i + t) k ² ₂

(147)

Rigid Scanning - Optimization

62

arg min

Z,R,t

w ₁

X n

i=1

k z _i P _Y (z _i ) k ² ₂ + w ₂

X n

i=1

k z _i (Rx _i + t) k ² ₂

matching term

(148)

Rigid Scanning - Optimization

62

arg min

Z,R,t

w ₁

X n

i=1

k z _i P _Y (z _i ) k ² ₂ + w ₂

X n

i=1

k z _i (Rx _i + t) k ² ₂

matching term rigid prior

(149)

Rigid Scanning - Optimization

62

arg min

Z,R,t

w ₁

X n

i=1

k z _i P _Y (z _i ) k ² ₂ + w ₂

X n

i=1

k z _i (Rx _i + t) k ² ₂

matching term rigid prior

(150)

Rigid Scanning - Optimization

63

arg min

Z ^t+1 , R, ˜ ˜ t

w ₁

X n

i=1

k z ^t+1 _i P _Y (z ^t _i ) k ² ₂ + w ₂

X n

i=1

|| z ^t+1 _i ( ˜ R(R ^t x _i + t ^t ) + ˜ t) || ² ₂

arg min

Z,R,t

w ₁

X n

i=1

k z _i P _Y (z _i ) k ² ₂ + w ₂

X n

i=1

k z _i (Rx _i + t) k ² ₂

(151)

Rigid Scanning - Optimization

63

arg min

Z ^t+1 , R, ˜ ˜ t

w ₁

X n

i=1

k z ^t+1 _i P _Y (z ^t _i ) k ² ₂ + w ₂

X n

i=1

|| z ^t+1 _i ( ˜ R(R ^t x _i + t ^t ) + ˜ t) || ² ₂

arg min

Z,R,t

w ₁

X n

i=1

k z _i P _Y (z _i ) k ² ₂ + w ₂

X n

i=1

k z _i (Rx _i + t) k ² ₂

use previous estimate

= closest point

(152)

Rigid Scanning - Optimization

63

arg min

Z ^t+1 , R, ˜ ˜ t

w ₁

X n

i=1

k z ^t+1 _i P _Y (z ^t _i ) k ² ₂ + w ₂

X n

i=1

|| z ^t+1 _i ( ˜ R(R ^t x _i + t ^t ) + ˜ t) || ² ₂

arg min

Z,R,t

w ₁

X n

i=1

k z _i P _Y (z _i ) k ² ₂ + w ₂

X n

i=1

k z _i (Rx _i + t) k ² ₂

use previous estimate

= closest point linearize rotation matrix

R ^t+1 = ˜ RR ^t

t ^t+1 = ˜ Rt ^t + ˜ t

(153)

Rigid Scanning - Optimization

64

R ⇡ R ˜ =

2 4 1

1 ↵

↵ 1

3 5

Rotation matrix linearization

cos(✓ ) ⇡ 1

sin(✓ ) ⇡ ✓

(154)

Rigid Scanning - Optimization

65

k z ^t+1 _i P _Y (z ^t _i ) k ² ₂

z ^t _i

Y

P _Y (z ^t _i )

(155)

Rigid Scanning - Optimization

65

k z ^t+1 _i P _Y (z ^t _i ) k ² ₂

z ^t _i

Y

P _Y (z ^t _i )

z ^t _i

Y

P _Y (z ^t _i )

n _i

(156)

Rigid Scanning - Optimization

65