The novel stereo camera model for use in bundle adjustment presented in this chapter has the generality to accommodate a wide range of the stereo cameras used today, and can be incorporated efficiently into the conventional sparse bundle adjustment algorithms. The conducted tests show that the use of the new stereoscopic camera model significantly increases the accuracy of the estimation in presence of noise and outliers. The reduction in the number of parameters used to describe the model enables significant reductions in the computation time required.
Limitations The formulation presented assumes fixed translations between the centers of the left and right camera. This assumption is required in order not to overparametrize the problem in respect to the conventional formulation. This could prove problematic for camera configurations where this assumption is violated.
Future Work For future work, it would be interesting to further investigate the implications of stereoscopic input data for the traditional processing pipeline. Though the creation of correspondences works satisfactory with the employed image interleaving
46
3.7 Discussion
Figure 3.7: Real-world experiments:The frames on the left depict garden furniture and were recorded using a consumer HD camcorder with a 3D conversion lens. The frames on the right depict a scene at a train station and were recorded using professional equipment. Both sequences were augmented by a cuboid to demonstrate the quality of the estimated camera parameters.
scheme, a dedicated stereo detection and outlier elimination algorithm may further improve the results.
Future work might also include the investigation of the effects of different types of parametrizations on the quality of the results. To this end, extensive tests with differ-ent stereo cameras and differdiffer-ent parametrizations have to be conducted. Alternative parametrizations could for example be based on a single vergence angle only.
CHAPTER 4
A Generalized Framework
for Constrained Bundle Adjustment
This chapter introduces hierarchies of Euclidean transformations as a means for con-strained bundle adjustment. The Euclidean transformations provide a framework able to handle many types of camera and scene constraints simultaneously in an intuitive and flexible way. It can be seen as a generalization of the stereoscopic camera model for bundle adjustment described in the previous chapter.
4.1 Introduction
In the previous chapter, a stereoscopic camera model for bundle adjustment has been introduced. This specialized model serves to reduce the number of parameters of the overall estimation process while still representing the real camera geometry, thus reducing overparametrization and providing enhanced reconstruction precision.
Overparametrization may not only be encountered in the description of the camera geometry, but also in the description of the scene. Reconstructed points may be collinear, coplanar, or share angular relations, for example. A result of overparameterization may be an unsatisfactory reconstruction in spite of a low reprojection error.
The constraints arising from the stereoscopic camera model of the previous chapter were introduced into bundle adjustment via additional rotational and translational components, or more generally: transformations. Using this approach as inspiration,
Chapter 4 A Generalized Framework for Constrained Bundle Adjustment
this chapter will seek to improve on the stereo camera model to provide an elegant and intuitive framework for constraints in bundle adjustment based on hierarchies of Euclidean transformations. Hierarchies of Euclidean transformations can be used to represent dependencies between constraints, and allow efficient incorporation into existing bundle adjustment procedures.
The space of constraints addressed is coplanarity, collinearity, angular relations, dis-tances, and parallelism, which can be conveniently expressed in terms of hierarchies of Euclidean transformations and therefore handled in a common mathematical frame-work.
Previous methods for constrained bundle adjustment, which will be reviewed in the next section, lack the ability to model constraints on the scene structure and on the camera geometry simultaneously, and are typically not able to describe all constraints in a consistent, homogenous way.
The novel approach for constrained bundle adjustment is flexible and applicable in many different scenarios, including stereo camera and moving object modeling.
As this approach makes explicit use of the properties of Euclidean space, for the remainder of this chapter it will be assumed that a perspective reconstruction and metric upgrade of the input data has been performed, as described in Chapter 2.
Outline This chapter continues with a review of related work in the next section, before methods for constrained bundle adjustment are reviewed in Section 4.3. Section 4.4 describes the new approach for constrained bundle adjustment based on hierarchies of Euclidean transformations, and Section 4.5 provides an example of how this approach might be used in order to construct a constrained parallelepiped. Section 4.6 presents application examples. This chapter is concluded by a discussion in Section 4.8.
4.2 Related Work
Lagrange Multipliers The method of Lagrange multipliers is commonly used to solve many constrained numerical optimization problems in mathematics. Triggs et al. [153]
discuss the application of the Lagrange multiplier method to bundle adjustment in general. The matter is also described by McLauchlan et al. [102] in detail. They em-ploy recursive partitioning in a variable state dimension filter formulation of SfM for efficiency. Meidow et al. [103] show how the method of Lagrange multipliers can be ap-plied to fundamental matrix estimation and the constrained estimation of homogenous entities in general.
Weighting Schemes McGlone [101] and Hrabáček and van den Heuvel [71] present systems for bundle adjustment that introduce geometric constraints as additional pseudo-observations. Szeliski and Torr [143] include weighted coplanarity constraints in
50