Welcome to Journal of Beijing Institute of Technology
Advance Search
Volume 28Issue 3
.
Turn off MathJax
Article Contents
Article Navigation> JOURNAL OF BEIJING INSTITUTE OF TECHNOLOGY> 2019> 28(3): 598-605
Baoling Han, Yuting Zhao, Qingsheng Luo. Walking Stability Control Method for Biped Robot on Uneven Ground Based on Deep Q-Network[J]. JOURNAL OF BEIJING INSTITUTE OF TECHNOLOGY, 2019, 28(3): 598-605. doi: 10.15918/j.jbit1004-0579.18059
Citation: Baoling Han, Yuting Zhao, Qingsheng Luo. Walking Stability Control Method for Biped Robot on Uneven Ground Based on Deep Q-Network[J].JOURNAL OF BEIJING INSTITUTE OF TECHNOLOGY, 2019, 28(3): 598-605.doi:10.15918/j.jbit1004-0579.18059
shu

Walking Stability Control Method for Biped Robot on Uneven Ground Based on Deep Q-Network

doi:10.15918/j.jbit1004-0579.18059
  • 1.

    School of Mechanical Engineering, Beijing Institute of Technology, Beijing 100081, China

  • 2.

    School of Mechatronic Engineering, Beijing Institute of Technology, Beijing 100081, China

  • Received Date:2018-06-12
    Abstract
  • A gait control method for a biped robot based on the deep Q-network (DQN) algorithm is proposed to enhance the stability of walking on uneven ground. This control strategy is an intelligent learning method of posture adjustment. A robot is taken as an agent and trained to walk steadily on an uneven surface with obstacles, using a simple reward function based on forward progress. The reward-punishment (RP) mechanism of the DQN algorithm is established after obtaining the offline gait which was generated in advance foot trajectory planning. Instead of implementing a complex dynamic model, the proposed method enables the biped robot to learn to adjust its posture on the uneven ground and ensures walking stability. The performance and effectiveness of the proposed algorithm was validated in the V-REP simulation environment. The results demonstrate that the biped robot's lateral tile angle is less than 3° after implementing the proposed method and the walking stability is obviously improved.
    • FullText(HTML)
  • loading
    • References (15)
  • [1]
    Tan Yantao, Sun Zhongbo, Li Hongyang, et al. A review of optimal and control strategies for dynamic walking bipedal robots[J]. Acta Automatica Sinica,2016, 42(8):1142-1157. (in Chinese)
    [2]
    Dang Van Chien, Ki Je Sung, Jong-Wook Kim. Sensory reflex control of a humanoid robot using FSR sensor[C]//IEEE International Conference on Advanced Intelligent Mechatronics, IEEE, 2015:1406-1409.
    [3]
    Kim J W, Tran T T, Dang C V, et al. Motion and walking stabilization of humanoids using sensory reflex control[J]. International Journal of Advanced Robotic Systems, 2016, 13(2):1.
    [4]
    Chen Guangrong, Wang Junzheng, Wang Liping. Gait planning and compliance control of a biped robot on stairs with desired ZMP[J]. IFAC Proceedings Volumes, 2014, 47(3):2165-2170.
    [5]
    Li Jian, Chen Weidong, Wang Lijun, el at. Stability control for biped walking on unknown rough surface[J]. Acta Electronica Sinica, 2010, 38(11):2669-2674. (in Chinese)
    [6]
    Sasaki H, Horiuchi T, Kato S. A study on behavior acquisition of mobile robot by deep Q-network[J]. Journal of Advanced Computational Intelligence & Intelligent Informatics, 2017, 8(4):727-733.
    [7]
    JAFRI Ali Raza, Huang Qiang, Yang Jie. et al. Motion planning of humanoid robot for obstacle negotiation[J]. Journal of Beijing Institute of Technology, 2008, 17(4):439-444.
    [8]
    Gu Shixiang, Holly Ethan, Lillicrap Timothy,et al. Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates[C]//IEEE International Conference on Robotics and Automation, IEEE, 2017:3389-3396.
    [9]
    Mnih V, Kavukcuoglu K, Silver D, et al. Humanlevel control through deep reinforcement learning[J]. Nature, 2015, 518(7540):529.
    [10]
    He Yudong, Wang Junzheng, Ke Xianfeng, et al. Stable walking for legged robots[J]. Journal of Mechanical Engineering, 2016, 52(21):1-7. (in Chinese)
    [11]
    Zhang Bin. Joint-space Trajectory planning for robots under multiple constraints[J]. Journal of Mechanical Engineering, 2011, 47(21):1-6. (in Chinese)
    [12]
    Wang Lipeng, Wang Junzheng, Zhao Jiangbo, et al. Foot trajectory generation and gait control method of a quadruped robot on uneven terrain based on zero moment point theory[J]. Transactions of Beijing Institute of Technology, 2015(6):601-606. (in Chinese)
    [13]
    Yang Jie, Huang Qiang, Li Jianxi, et al. Walking pattern generation for humanoid robot considering upper body motion[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, 2007:4441-4446.
    [14]
    Watkins C J C H, Dayan P. Q-learning[J]. Machine Learning, 1992, 8(3-4):279-292.
    [15]
    Liu Yiping, Wensing P M, Orin D E, et al. Dynamic walking in a humanoid robot based on a 3D Actuated Dual-SLIP model[C]//2015 IEEE International Conference on Robotics and Automation (ICRA), IEEE, 2015:5710-5717.
    • Relative Articles
  • Supplements (0)
  • Cited By
  • 加载中

Catalog

    通讯作者:陈斌, bchen63@163.com
    • 1.

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索
    Get Citation
    PDF
    XML

    Article Metrics

    Article views (471) PDF downloads(295) Cited by()
    Proportional views
    Related

    Address:5 Zhongguancun South Street, Haidian District, Beijing

    Tel:86-10-68914374

    Email:blgywb@bit.edu.cn

    Copy Right © Journal of Beijing Institute of Technology

    Supported by:Beijing Renhe Information Technology Co. Ltd

    /

      Return
      Return
        Baidu
        map