Multiple Moving Obstacles Avoidance of Service Robot using Stereo Vision

In this paper, we propose a multiple moving obstacles avoidance using stereo vision for service robots in indoor environments. We assume that this model of service robot is used to deliver a cup to the recognized customer from the starting point to the destination. The contribution of this research is a new method for multiple moving obstacle avoidance with Bayesian approach using stereo camera. We have developed and introduced 3 main modules to recognize faces, to identify multiple moving obstacles and to maneuver of robot. A group of people who is walking will be tracked as a multiple moving obstacle, and the speed, direction, and distance of the moving obstacles is estimated by a stereo camera in order that the robot can maneuver to avoid the collision. To overcome the inaccuracies of vision sensor, Bayesian approach is used for estimate the absense and direction of obstacles. We present the results of the experiment of the service robot called Srikandi III which uses our proposed method and we also evaluate its performance. Experiments shown that our proposed method working well, and Bayesian approach proved increasing the estimation perform for absence and direction of moving obstacle.


Introduction
In recent years, service robots developed for various applications such as the personal, medical and welfare robots.Technologies and methods used for service robots increased drastically to make it more intelligent, and resulting these kind of robots available comercially, such as iRomba as autonomous vacum cleaner robot and very usefull for household.Among the indoor service robots, those that are able to operate in environments with humans, and especially those that are able to interact with the customer have gained high interest in recent years.The major task routinely performed by a service robot (for example deliver a cup, picking a cup and human robot interaction) are based on visually perceived information.In order a service robot perform such tasks, they must also have the ability to perceive and act upon visual information.Computer Vision is a an important tools for robotics systems since it mimics the human sense of vision and allows for non-contact measurement of the environment.A good program using vision sensor will make a service robot have the ability to detects and identifies detailed object around it (such as face recognition, distance measurement of obstacle, and free area for path planning).The main concern when develop a service robot is obstacle avoidance system and the implementation of stereo camera as an important vision sensor.Although vision-based service robot become the chalenging and complex task for research and engineer, models of the multiple moving obstacle avoidance system for vision-based service robot using Bayesian approach for state estimation is not explored extensively.
The development of an obstacle avoidance system for service robot to accurately detect moving obstacles in indoors is challenging task.The navigation and obstacle avoidance strategy are the important aspects in a vision-based service robot.Bayesian techniques provide a powerful statistical tool to help manage measurement uncertainty and perform multisensor fusion and identity estimation.The advantages of probabilistic robotics are able to accommodate imperfect sensors (such as camera with noises), robust in real world applications and best known approach to many hard robotics problem.Using sensory information to locate the robot in its environment is the most fundamental problem to providing a service robot with autonomous capabilities.By using Bayesian approach in robot system, the state of presence of obstacles and other properties can be estimated.
Based on literatures obtained by the authors, many research in development of service robot such as [1], [2], whereas task of the service robot is the setting and clearing of tables in a controlled environment without stereo camera.However, there is no multiple moving obstacles avoidance method for service robot in indoor environment exposed especially using stereo camera.The contribution of this paper is the introduction of a new method of multiple moving obstacles avoidance for service robots using a stereo camera in indoor environment.In this paper, we solved the drawbacks of common obstacle avoidance methods such as Potential Field Method (PFM) [3] and Vector Field Histogram (VFH) [4] by proposed a complete mechanism for multiple moving obstacles avoidance of vision-based service robot.

Research Method 2.1 Architecture for Service Robot
A mobile robot involving two actuator wheels is considered as a system subject to nonholonomic constraints and usually using fuzzy logic to control the motor [5].Consider an autonomous wheeled mobile robot and position in the Cartesian frame of coordinates shown in Figure 1, where ‫ݔ‬ ோ and ‫ݕ‬ ோ are the two coordinates of the origin P of the moving frame and ߠ ோ is the robot orientation angle with respect to the positive x-axis.The rotation angle of the right and left wheel denoted as ߮ and ߮ and radius of the wheel by R thus the configuration of the mobile robot q R can be described by five generalized coordinates such as: Based on Figure 1, ‫ݒ‬ ோ is the linear velocity, ߱ ோ is the angular velocity, ‫ݎ‬ ோ and ߣ ோ are radial and angular coordinate of the robot [6].The kinematics equations of motion for the robot given by: The angular velocity of the right and left wheel can be obtained by: Based on the model in Figure 1, we propose the model of a mobile robot using a stereo camera with a moving obstacle as shown in Figure 2. A camera as a vision sensor has limitations in view angle to capture an object, so we define Cam θ as a maximum angle that moving obstacles can be detected by a camera used in this research.We have developed a vision-based service robot called Srikandi III with the ability to face recognition and avoid people as moving obstacles, this wheeled robot is next generation from Srikandi II [7], [8].The prototype of service Robot Srikandi III that utilized a low cost stereo camera, 1 ultrasonic sensor, compass and Propeller 32 bit microcontroller in order the cost of development can be reduces is shown in Figure 3.

Improved Face Recognition System using PCA 2.2.1 Stereo Imaging Model
We have developed a system for face detection using Haar cascade classifier and depth estimation for measuring distance of peoples as moving obstacles using stereo vision.In the stereo imaging model, the tree-dimensional points in stereo camera frame are projected in the left and the right image frame.On the contrary, using the projection of the points onto the left and right image frame, the three-dimensional points positions in stereo camera frame can be located.Figure 4 shows the stereo imaging model using the left front image frame LF and right front image frame RF [9].To estimate the direction θ direction of moving obstacle using stereo vision, we calculate using the figure and formula below:

Improved Face Recognition System
The face is our primary focus of attention in developing a serves peoples.Unfortunatelly, developing a computational model of face recognition is quite difficult, because faces are complex, meaningful visual stimuli and multidimensional.Modelling of face images can be based on stat [10] and Linear Discriminat analysis (LDA) assumption of certain surface reflectance properties, such as Lamber Discriminant Analysis (LDA) is a method of finding such a linear combination of variables which best separates two or more classes system and faces database Indian face database.The advantages of our framework is able to store ordered item from customer in .xmlfile and displayed construct images under different illumination conditions by generate a rando brightness level for ITS face database.face.Each set of ITS face database consists of 3 poses (front, left, right ) and varied with illumination [13].Figure 6.ITS, Indian and ATT face database used as compa

A Moving Obstacles Avoidance 2.3.1 Probabilistic robotics for Multiple Obstacle Avoidance Method
Camera as vision sensor sometimes have distortion, so Bayesian decision theory used to state estimation and determine the sensor data.Bayesian decision rule probabilistically estimate a dynamic system state from noisy observations.Examples of measurement data include camera images and range scan.If x is a quantity that we would like to infer from y, the probability probability distribution.The Bayesian update formula is applied to determine the new posterior p(x, y) whenever a new observation is obtained

ቁ ace Recognition System
The face is our primary focus of attention in developing a vision based service robot to serves peoples.Unfortunatelly, developing a computational model of face recognition is quite difficult, because faces are complex, meaningful visual stimuli and multidimensional.Modelling of face images can be based on statistical model such as Principal Component Analysis (PCA) and Linear Discriminat analysis (LDA) [11][12] and physical modelling based on the assumption of certain surface reflectance properties, such as Lambertian surface (LDA) is a method of finding such a linear combination of variables which best separates two or more classes [9].We have developed a framework for face recognition faces database called ITS face database and will be compared with ATT and The advantages of our framework is able to store ordered item from customer in .xmlfile and displayed on the screen as shown in Figure 10

A Moving Obstacles Avoidance Method using Bayesian Approach Probabilistic robotics for Multiple Obstacle Avoidance Method
Camera as vision sensor sometimes have distortion, so Bayesian decision theory used to state estimation and determine the optimal response for the robot based on inaccurate sensor data.Bayesian decision rule probabilistically estimate a dynamic system state from noisy observations.Examples of measurement data include camera images and range scan.If we would like to infer from y, the probability p(x) will be referred to as prior probability distribution.The Bayesian update formula is applied to determine the new posterior whenever a new observation is obtained: Then, we restrict the action space denoted as A as:

= maneuver to right, maneuver to left, stop
We define a loss function L(a, θ) which gives a measure of the loss incurred in taking action a when the state is θ.The robot should chooses an action a from the set A of possible actions based on the observation z of the current state of the path θ.This gives the posterior distribution of θ as:

Proposed Multiple Moving Obstacles Avoidance Method and Algorithm
We have proposed a method and algorithm of obstacles avoidance for service robot that run from start to goal position, giving a cup to customer and going back to home.This method will identify a customer, checking moving obstacles and its distance and take action for maneuver to avoid the collision.Stereo camera used has limitation such as angle view, this camera only able to capture object infront of it about 30°.So, when the robot starts to maneuver, the moving obstacle could be out of view area of camera.So for this experiment, we have proposes a predefined motion for maneuver based on the estimation speed and direction of moving obstacle.Figure below shows the proposed model of maneuvering on the service robot, pL which is the probability of moving obstacle leads to the left, and pR the probability of moving obstacle leads to the right.By estimating the direction of motion of the obstacle, then the most appropriate action to avoid to the right / left side can be determined, to minimize collision with these obstacles.If there are more than 1 moving obstacle, then robot should identified the nearest moving obstacle to avoid collision, and the direction of maneuver should be opposite with the direction of moving obstacle (Figure 7).The flowchart of a Navigation system and multiple moving obstacles avoidance method for vision-based service robot using stereo camera shown in Figure 8.Based on the Figure 8, image captured by stereo camera used as testing images to be processed by Haar classifier to detect how many people in the images, and face recognition by PCA.We implement visual tracking for heading a robot to a customer.Robot continuously measures the distance of obstacle and send the data to Laptop.The next step is multiple moving obstacle detection and tracking.If there is no moving obstacle, robot run from start to goal position in normal speed.The advantage using stereo vision in our system is the ability to estimate the distance of customer/obstacles (depth estimation) and direction's movement of obstacles.
If moving obstacle appeared and collision will occurred, robot will maneuver to avoids obstacle.The proposed algorithms for obstacles avoidance shown below:  8. Flowchart of A Navigation system and Multiple moving obstacles avoidance method for vision-based service robot using stereo camera.

Results and Analysis 3.1 Improved Face Recognition System
First, we evaluate the result of our proposed framework of face recognition system and compared with ATT and Indian face database using Face Recognition Evaluator developed by Matlab.The success rate comparison between 3 face databases shown Figure 9.It shown clearly that ITS database have highest success rate than ATT and Indian face database when the illumination of testing images is varied.The success rate using PCA in our proposed method and ITS face database is 95.5 %, execution time, it seems the Indian face database (IFD) is shortest because the size of each image is lowest then ITS and ATT (b) (a) Figure 9. Accuracy comparison of face recognition between 3 faces databases, each using 10 The result of improved face recognition system shown in succesfully identified with his order robot to an identified customer.Robot proposed method.

2 Multiple moving obstacle avoidance using Stereo Vision.
The result of identifying multiple moving if we using stereo vision, we can estimate the distance and direction/angle of the moving ISSN: 1693-6930 Multiple Moving Obstacle Avoidance for Service Robot using Stereo Vision (Widodo Budiharto clearly that ITS database have highest success rate than ATT and Indian face database when the illumination of testing images is varied.The success rate using PCA in our proposed method and ITS face database is 95.5 %, higher than ATT face database 95.4% execution time, it seems the Indian face database (IFD) is shortest because the size of each image is lowest then ITS and ATT (b) [13] (b) comparison of face recognition between 3 faces databases, each using 10 sets face.
The result of improved face recognition system shown in Figure 10, where 1 customer identified with his order.Before delivering a cup, visual tracking applied to directs a robot to an identified customer.Robot succesfully go to the identified customer using our face detected and recognized together with his order framework of face recognition system.

Multiple moving obstacle avoidance using Stereo Vision.
The result of identifying multiple moving obstacle shown in figure below, the advantages if we using stereo vision, we can estimate the distance and direction/angle of the moving Widodo Budiharto) 441 clearly that ITS database have highest success rate than ATT and Indian face database when the illumination of testing images is varied.The success rate using PCA in our proposed higher than ATT face database 95.4% (a).For total execution time, it seems the Indian face database (IFD) is shortest because the size of each comparison of face recognition between 3 faces databases, each using 10 , where 1 customer Before delivering a cup, visual tracking applied to directs a succesfully go to the identified customer using our together with his order using our obstacle shown in figure below, the advantages if we using stereo vision, we can estimate the distance and direction/angle of the moving obstacle.The value probability of obstacle/no obstacle also run well for make a robotics system more robust as as shown Figure 11.For experiment delivering a cup to a customer, the setup experiment is in indoor where a customer and not customer sitted on the chair, and there is someone that walking as a moving obstacle as shown below.Robot succesfully identify a moving obstacle, avoid the collision, giving a cup to a customer then go back to home.For 10 (ten) times experiment using Bayesian approach, the success rate to identify moving obstaacle is 90%, and without Bayesian approach is 60%.The very insteresting video to show the action of this robot can be viewed at: http://www.youtube.com/watch?v=n181CtvGJ88.

Conclusion
This paper presents a robust multiple moving obstacles avoidance method for service robot in indoor environment.By varying illumination in training images, it will increase the success rate in face recognition.The success rate using our proposed method (ITS face database) is 95.5 %, higher than ATT face database 95.4%.Our framework of face recognition system can be used for the vision-based service robot.Geometrical model for moving obstacle avoidance and algorithms for multiple moving obstacles proposed then succesfully implemented to a service robot.Integration of stereo vision and ultrasonic sensors proved applicable as main sensors for distance estimation between service robot and obstacles.Experimental results with various situations have shown that the robot reaches the goal points while avoiding multiple moving obstacle with the proposed method.The state estimation using Bayesian used for absense and direction of obstacles increasing the succes rate of obstacle's detection and measurement.Bayesian decision rule implemented for state estimation makes this method more robust because the optimal solution for avoiding obstacles is obtained by trading between maneuver and stop action.The strength of the visual tracking and maneuvering method proposed shown good performance for navigation system of vision-based service robot.This paper have contribution for solving the drawbacks of common obstacle avoidance methods by proposed a complete mechanism for multiple moving obstacles avoidance of vision-based service robot.For future work, we will focus on navigation system for humanoid service robot using stereo vision.

Figure 1 .
Figure 1.Cartesian representation of mobile robot

Figure 2 .Figure 3 .
Figure 2. General cartesian model of mobile robot using stereo camera Figure 3. Prototype of Service robot Srikandi III using Stereo Vision.

Figure 5 .
Figure 5. Direction estimation using stereo vision . In this research, we construct images under different illumination conditions by generate a rando for ITS face database.Each of face database consists of 10 sets of people's face.Each set of ITS face database consists of 3 poses (front, left, right ) and varied with ITS, Indian and ATT face database used as comparison to see the effect of illumination at face recognition[13].
robot to serves peoples.Unfortunatelly, developing a computational model of face recognition is quite difficult, because faces are complex, meaningful visual stimuli and multidimensional.Modelling cipal Component Analysis (PCA) and physical modelling based on the tian surface.Linear (LDA) is a method of finding such a linear combination of variables which framework for face recognition called ITS face database and will be compared with ATT and The advantages of our framework is able to store ordered item fromIn this research, we construct images under different illumination conditions by generate a random value for Each of face database consists of 10 sets of people's face.Each set of ITS face database consists of 3 poses (front, left, right ) and varied with to see the effect of Camera as vision sensor sometimes have distortion, so Bayesian decision theory used optimal response for the robot based on inaccurate sensor data.Bayesian decision rule probabilistically estimate a dynamic system state from noisy observations.Examples of measurement data include camera images and range scan.If will be referred to as prior probability distribution.The Bayesian update formula is applied to determine the new posterior If we want a service robot should stay on the path to goal in any case, strategies to avoid moving obstacle include:• Maneuver to the right, if detected moving obstacle is moving toward the left.Maneuver to the left, if detected moving obstacle is moving toward the right.• stop, if moving obstacle too close to robot detected both by vision and ultrasonic sensors.

Figure 7 .
Figure 7.A maneuvering model to avoids multiple moving obstacle using stereo vision, 2 multiple moving obstacle with the different direction (a) and the same direction (b).

Figure 10 .
Figure 10.An example of framework of face recognition system.

Table 1 .
Actions to avoid moving obstacle

Algorithm 1 .
Multiple moving obstacles avoidance and maneuvering for service robot.