An Approach to Minimising Estimated Pincushion Camera Distortions

Minimizing camera distortion has been a focus of recent photorealism researches. Of the several types of camera distortions, researchers concentrated on the radial distortion being the most severe part of the total lens aberrations. Often, polynomial approximations by introducing distortion-specific parameters into the camera model have been methods of evaluating distortions. Of the two types of radial distortions, barrel, is the most discussed probably because of its dominance in cheap wide-angle lenses. In this paper, an approach to estimating and minimizing pincushion distortion by using fundamental surface area properties of the distorted image is proposed. This paper is significant as it reveals the importance of the topic and its implications on the camera. Demonstrations using four camera lenses show the robustness of this technique under different focal lengths conditions. The reliability of the approach is justified by comparing the results with the physical observations using (two tailed) t-test statistic.


Introduction
An augmented reality system must be structured to combine both the real and the virtual world, provide interactivity in real time and registration in 3D [1], [2].For effective 3D registration, distortions estimation and minimization are crucial factors [3].The radial distortions have been viewed as inherent errors of any lens [4], although some scholars including [5] have attributed this to design and or manufacturing errors.
Pincushion distortions are generally overlooked for their insignificance in total lens aberrations.They do not affect the image quality [1], [5], but, they could become significant for automated applications where time is a factor [5], [6], [7].Like any other types of distortions, researchers often attribute them to inherent error of the lens [1], [4].Pincushion distortion therefore could be viewed as "lens defect which makes images pinched at the centre" that typically occurs at the tele end of any lens.There are two possibilities, when the image is displaced closer to the optical axis of the lens, the result is the negative pincushion effect and when place otherwise, positive.Unfortunately, only one type (the negative displacement pincushion distortion) is often talked about [5], [8], [9].However in both cases, the surface area of the image is affected by either an increase or a decrease.The visual effect of pincushion distortions and barrel distortions on image geometry is illustrated in Figures 1 and 2.  Modern digital cameras with zoom lenses are often the victims of this type of lens aberrations.In some researches including [1], [6], [7], pincushion distortion has been estimated as the amount a reference line is bent to the percentage of the image height.Most consumer digital cameras retain a pincushion distortion much lower than that of the barrel distortion with 0.62% being a typical value [5].
Applications areas such as the augmented reality (AR) rely heavily on the correctness of camera's positioning (both real and virtual).For obvious reasons of misalignments, repeated calibration has often been an approach to reducing overall camera aberrations.This approach has been found daunting due to much human-involvement which is disadvantageous for automated applications where time is a factor [10].In literatures, two methods are common for calibrating cameras lenses more efficiently: (1) the camera self-calibration method by [10] and (2) the Dynamic camera self-calibration from controlled motion sequences by [11].Camera calibration is performed by observing an object whose geometry in 3D space is known.The calibration objects are orthogonally placed planar objects.
A fundamental issue in camera calibration is the computation of the transformational matrix.Often, for ease of computation, the matrix is reduced.The reduction is then used to reproject the viewing angle of the camera and thereby bringing minimising distortion.In estimating radial distortions, the fundamental matrix is determined by assuming that the distorted image is corrected offline [3] or by integrating the distortion correction during the estimation of the epipolar geometry [1], [9].These techniques all rely on the fact that either translational or rotational motion of the camera takes place.For many applications [15] therefore, distortion estimation and reduction are crucial issues which has generated keen research concerns [1], [3], [9].

ISSN: 1693-6930
An Approach to Minimising Estimated Pincushion Camera …. (Folorunso Olufemi Ayinde) 557 Like any other types of distortions, pincushion distortions are quite difficult to notice in photography except when the camera is used to capture a building or a horizon with straight lines or edges as shown in Figure 2. Some image manipulating software like the Photo-Brush is often employed to handle pincushion distortions using high quality bi-cubic interpolations integrated into its program logic.The output is often a high-quality image without "any" traces of manipulation as shown in Figure 11.The limitation of this however is the rigidity of its application and inapplicability to all types of pincushion distortions.In addition, estimation of pincushion distortion is a neglected issue in those applications and recent researches [3], [9], [11].Therefore, the understanding of the digital camera pincushion distortion estimation and minimization is a significant topic in photography and photorealistic researches.This paper is significant as it reveals the importance of the topic and its implications on the camera as a device.

Research Method
We propose a mathematical model for pincushion distortions based on the geometric view area (of the image) as captured by the camera lens.The model does not incorporate the use of either the intrinsic or the extrinsic camera parameters.The approach is most useful where minimisation of pincushion effects is desired especially in viewing of horizons and buildings that have straight boundaries and lines.A basic assumption in this paper is that the planar object is squared.This paper is organised thus: First, a model for estimating the total area of an image and the total area affected by pincushion distortions as a way of estimating them is presented.This is followed by the generalised cameral model which was expanded to encompass the geometrical based pincushion distortion.A mathematical approach for estimating pincushion distortion using the derived function is developed lastly some examples to validate the robustness of the proposed technique are presented.

Pincushion Distortion Model
Basic Assumption: Assuming that the image under consideration is planar and squared.The modelling of pincushion distortion is achieved by the understanding of the area covered by a minor segment AB of a circle centre O radius r and angle θ as shown in Figure 3.By definition, the area of segment AB denoted henceforth as (sAB) equals the area of sector AOB minus the area of triangle AOB, that is, or ‫ܤܣݏ‬ = ‫ݎ‬ ଶ (0.00873θ − 0.5‫݊݅ݏ‬θ) Now, the pincushion distortion affects the four edges of the image, therefore the total pincushion (TsAB) distortion is: Since the image is assumed planar on the x-y plane, (say): Assuming coordinates A(x 1 ,y 1 ), B(x 2 ,y 1 ), (x 2 ,y 2 ), D(x 1 ,y 2 ), for the image vertices as shown in Figure 4.The area covered by the distorted image (dIA) is thus:

The Camera Model
The modelling follows the conventional relationship between a 3D point Pi and its projection image pj described by [4] as: Where α is non-zero arbitrary scale factor, I and E are the intrinsic matrix and the extrinsic parameters respectively.Camera intrinsic matrix I is given as: TELKOMNIKA ISSN: 1693-6930 An Approach to Minimising Estimated Pincushion Camera …. (Folorunso Olufemi Ayinde) 559 where x and y are coordinates of the image's principal point, a and b are the scale factors in image x and y axes, and λ is the skewness of the two image axes.Camera extrinsic parameter E is the rotation and translation matrices which relate the world coordinate system to the camera coordinate system often denoted by: ‫ܧ‬ = ܴ| − ‫ݐܴ‬ In all, six parameters must be estimated, 3 intrinsic and 3 extrinsic.The extrinsic parameters could be estimated using method described by [3].This method provided solution through minimizing an algebraic distance which is described as "physically meaningless" but was refined by using the maximum likelihood inference.The maximum likelihood estimate was obtained by minimizing a nonlinear minimization problem, which was solved with the Levenberg-Marquardt Algorithm.The algorithm is well described and implemented by [1], [6], [12], [14].
During calibration, either the camera or the planar object is rotated, or translated or both which creates the distortion.For most practical applications λthe skewness, is zero, simply because the planar object is placed on (say) a table.In this research, a slightly different view from this is upheld by assuming that an arbitrary point J 0 (u 0 ,v 0 ) in the planar image is shifted from z=0 to z=dx in 3D (object space) to J 1 (u 1 ,v 1 ), sweeping the lens' view through angle θ as shown in Figure 7.Where f and f ∞ are the focal lengths of the camera lens and image, c is a parameter local to the camera mapping function, and ψ is the zooming angle.By applying the geometric relation assumption proposed by [5], it follows that: Figure 7. Showing geometric relationships based on the assumptions For ease of computations, c could be approximated to be equivalent to the focal length of the camera.
. Since log ట→±ట (ܿ − ݂) = 0 and any arbitrary point P i (x i ,y i ) in the planar coordinates and the corresponding point p j (x j ,y j ) in the virtual image must satisfy the model described in equation ( 4), and because the camera and the planar object are no longer perpendicular to each other relative to the z axis, it follows that the skewness of the two image axes λ = λ+cα, where α is a constant of displacement.By placing the camera lens at a chosen distance far away from the image plane such that f ∞ >>> f, It follows therefore from (4) that: are the rotation and the translation matrices associated with the cameral model. or

Modelling the Pincushion Distortions
Polynomial approximation has been a preferred method for modelling radial distortions.It is often expressed as [8]: Where u and u l are the distorted and the undistorted image coordinates.k is a vector parameter of distortion, and r l is the distance between u and u l .The order of the distortion function is often determined by the length of k.Most authors agreed that for barrel-radial distortions, second or forth order d is sufficient [1], [6], [8].The proposal for pincushion distortions therefore is: Where θ is the angle (in degrees) swept by the camera lens from point J 0 (u 0 ,v 0 ) to J 1 (u 1 ,v 1 ) through distance dx as shown in Figure 6.r is the image distortion radius.

Estimating Pincushion Distortions
Since ψ is the zooming angle of the camera, it is obvious that the zoomed camera image plane, the normal object plane and the new object plane (from Figure 7) are parallel.By using the frequently used model of camera perspective it follows that: Due to the increased zooming angle from (2), it follows that the pincushion distortions in the image is: This yield: TELKOMNIKA ISSN: 1693-6930 An Approach to Minimising Estimated Pincushion Camera …. (Folorunso Olufemi Ayinde)

561
Now, since the desired image should be pincushion-distortion free, it follows that: Or the Total Pincushion Distortions (tPD) In the neighbourhood of r r++ (or when r is 'reasonably large') as a result of increased zooming angle, the image is distortion free.The same results can be obtained by reducing the zooming angle.Since dx is a constant and f is known, this makes the computation (of the tPD) very easy.When compared to a similar research by [1], [5], [16] this pincushion distortion estimate is entirely a function of only the focal length of the camera.

Results and Discussion
To validate the propositions above, the null hypothesis is stated thus: There is no significant difference between the pincushion distortion obtained from practical and theoretical estimates using (14).Pincushion distortions were estimated by using four cameras lenses with focal lengths (6.5mm, 7.5mm, 8.5mm, 10.5mm) and six objects with dimension (30x30)mm .Each of the cameras lenses are used to capture images of the objects by moving the lens through distances dx= {40mm, 50mm, 60mm, 70mm and 80mm} corresponding to the length of each object (as illustrated in Figure 6).The images are printed out using HP DeskJet F4280 printer and placed on pre-printed square grids (of 1mm-squared) dimensions.The pincushion distortions on the images were estimated using the square grids denoted by grid(tPD).Theoretically the tPD described in this paper was also calculated for each view of the objects by each of the cameras using their focal lengths described in (14).The pincushion distortions estimates obtained from both the grid-based and the theoretical (with ( 14)) are compared using a repeated-measure t-test statistic as shown in Table 1.The obtained t-values are not extremely high enough to fall in the critical region (±2.571).The t-distribution table is consulted for a two-tailed test with significance level α = 0.05, df = 5.
Therefore, we fail to reject the null hypothesis and conclude that there is no significant difference between the two sets of data.The theoretically determined results using ( 14) are easier to compute, it is therefore suggested for future use in estimation of pincushion distortions.A bar chart for distortions obtained from both methods with Camera1 lens is presented in Figure 8, similar charts could be obtained for lenses of Cameras 2-4.The progression in the tvalues obtained for each successive pair of readings and the cameras' focal lengths is shown in Figure 9.The grid-based estimation is achieved through the following process: Each of the captured and printed images is superimposed on a uniformly ordered 1mm 2 tiles grid as shown in Figure 10.In an example with camera lens focal point f = 6.5,only 0.32 cells are within the distortion on each side of the superimposed squared image for the (30x30) mm    For ease of computation, only one side is determined and multiplied by 4 since the pincushion distortion obviously affects all the four edges.Pincushion distortion at one of the edges is sufficient to estimate the total for the whole image; visual variances observed at any other edges are largely due to ignorable human errors that gives pointers to more researches in the near future.

Conclusion
This paper presents an approach to estimating and minimizing pincushion distortion by making using fundamental surface area properties of the distorted images.It reveals the importance of the topic and its implications on the camera.The experimental results obtained from four camera lenses shows the robustness of this technique under different focal lengths conditions.The reliability of the approach is justified by comparing the results with the physical observations using (two tailed) t-test statistic.

Figure 1 .
Figure 1.(a) Showing effect of negative and (b) positive pincushion distortion on images.The dashed line represents the actual image without any distortion.

Figure 3 .
Figure 3. Showing circle centre O radius r and angle θ.

Figure 4 .
Figure 4. Showing distorted image ABCD with pincushion distortion represented by the shaded regions.

Figure 5 .
Figure 5. Showing changing distortion areas δsAB as r and θ changes.

Figure 6 .
Figure 6.Showing the distorted image and the distortion areas generated by varying θ and r.

Figure 9 .
Figure 9. Showing t-value progression with changing and increasing camera focal lengths.

Figure 10 . 2 .
Figure 10.Showing the grid of tiles and the superimposed squared image.

Table 1 .
Pincushion distortions estimates in (mm