IJE TRANSACTIONS B: Applications Vol. 28, No. 2 (February 2015) 198-204   

downloaded Downloaded: 229   viewed Viewed: 2234

F Yaghmaee and H Reza Koohi
( Received: July 02, 2014 – Accepted: November 13, 2014 )

Abstract    In this paper we focus on the application of reinforcement learning to obstacle avoidance in dynamic Environments in wireless sensor networks. A distributed algorithm based on reinforcement learning is developed for sensor networks to guide mobile robot through the dynamic obstacles. The sensor network models the danger of the area under coverage as obstacles, and has the property of adoption of itself against possible changes. The proposed protocol can integrate the reward computation of the sensors with information of the intended place of robot so that it guides the robot step by step through the sensor network by choosing the safest path in dangerous zones. Simulation results show that the mobile robot can get to the target point without colliding with any obstacle after a period of learning.Also we discussed about time propagation between obstacle, goal, and mobile robot information. Experimental results show that our proposed method has the ability of fast adoption in real applications in wireless sensor networks.


Keywords    Reinforcement Learning, Sensor Network, Dynamic Obstacle Avoidance, Robot Navigation.


چکیده    ما در این مقاله به کاربرد یادگیری تقویتی در راهنمایی ربات متحرک ضمن ممانعت از موانع پویا در شبکه سنسور بیسیم پرداختیم. شبکه سنسور سطح خطر منطقه ای که تحت پوشش دارد به صورت مانع مدل می کند و این قابلیت را دارد که خود را با تغییرات احتمالی وفق دهد. الگوریتم ارائه شده نتایج محاسبات پاداش و خطا در سنسورها را با اطلاعات مکان ربات در حال حرکت تلفیق می نماید تا بتواند ربات را قدم به قدم در شبکه سنسور ضمن تعیین امن ترین مسیر در مناطق خطرناک راهنمایی کند. نتایج شبیه سازی نشان داد پس از دوره یادگیری، ربات بدون گیر کردن در خطرات می تواند به مقصد برسد. ضمنا در خصوص زمان مورد نیاز برای انتشار اطلاعات مقصد و موانع و ربات متحرک و شناسایی آنها بحث و بررسی شده است. آزمایشات نشان میدهد روش پیشنهادی از دقت خوبی در محیط های واقعی برخوردار است.



1.     Feng, Z., Tan, L., Li, W. and Gulliver, T.A., "Reinforcement learning based dynamic network self-optimization for heterogeneous networks", in Communications, Computers and Signal Processing, PacRim, Pacific Rim Conference on, IEEE., (2009), 319-324.

2.     Chasparis, G.C. and Shamma, J.S., "Distributed dynamic reinforcement of efficient outcomes in multiagent coordination and network formation", Dynamic Games and Applications,  Vol. 2, No. 1, (2012), 18-50.

3.     Brooks, R.A., "A robust layered control system for a mobile robot", IEEE Journal ofRobotics and Automation,  Vol. 2, No. 1, (1986), 14-23.

4.     Gosavi, A., "A tutorial for reinforcement learning", Department of Engineering Management and Systems Engineering, (2011).

5.     Busoniu, L., Babuska, R. and De Schutter, B., "A comprehensive survey of multiagent reinforcement learning", Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on,  Vol. 38, No. 2, (2008), 156-172.

6.     Qiao, J., Hou, Z. and Ruan, X., "Application of reinforcement learning based on neural network to dynamic obstacle avoidance", in Information and Automation, 2008. ICIA 2008. International Conference on, IEEE. Issue, (2008), 784-788.

7.     Valiollahi, S., Ghaderi, R., Ebrahimzade, A., , "A q-learning based  continuous tuning of fuzzy wall tracking without exploration", International Journal of Engineering-Transactions A: Basics, Vol. 25, No. 4, (2012), 355-366.

8.     Abdi, J., Khalili, G.F., Fatourechi, M., Lucas, C. and Sedigh, A.K., "Control of multivariable systems based on emotional temporal difference learning controller", International Journal of Engineering-Transactions A: Basics,  Vol. 17, No. 4, (2004), 363-376.

9.     Mirmomeni, M. and Yazdanpanah, M., "An unsupervised learning method for an attacker agent in robot soccer competitions based on the kohonen neural network", International Journal of Engineering- Transactions A: Basics,  Vol. 21, No. 3, (2008), 255-268.

10.   Koohi, H., Nadernejad, E. and Fathi, M., "Employing sensor network to guide firefighters in dangerous area", International Journal of Engineering-Transactions C: Aspects,,  Vol. 32, No. 2, (2010), 191-202.

11.   Ganesan, D., Krishnamachari, B., Woo, A., Culler, D., Estrin, D. and Wicker, S., Complex behavior at scale: An experimental study of low-power wireless sensor networks. (2002), Technical Report UCLA/CSD-TR 02.

12.   Aslam, J., Li, Q. and Rus, D., "Three poweraware routing algorithms for sensor networks", Wireless Communications and Mobile Computing,  Vol. 3, No. 2, (2003), 187-208.

13.   Baldwin, P., Kohli, S., Lee, E.A., Liu, X. and Zhao, Y., "Modeling of sensor nets in ptolemy ii", in Proceedings of the 3rd international symposium on Information processing in sensor networks, ACM. Vol., No. Issue, (2004), 359-368.

International Journal of Engineering
E-mail: office@ije.ir
Web Site: http://www.ije.ir