Reinforcement Learning-Based Optimal Tracking Control of an Unknown Unmanned Surface Vehicle

Ning Wang, Ying Gao, Hong Zhao, Choon Ki Ahn

Research output: Contribution to journalArticlepeer-review

63 Citations (Scopus)


In this article, a novel reinforcement learning-based optimal tracking control (RLOTC) scheme is established for an unmanned surface vehicle (USV) in the presence of complex unknowns, including dead-zone input nonlinearities, system dynamics, and disturbances. To be specific, dead-zone nonlinearities are decoupled to be input-dependent sloped controls and unknown biases that are encapsulated into lumped unknowns within tracking error dynamics. Neural network (NN) approximators are further deployed to adaptively identify complex unknowns and facilitate a Hamilton-Jacobi-Bellman (HJB) equation that formulates optimal tracking. In order to derive a practically optimal solution, an actor-critic reinforcement learning framework is built by employing adaptive NN identifiers to recursively approximate the total optimal policy and cost function. Eventually, theoretical analysis shows that the entire RLOTC scheme can render tracking errors that converge to an arbitrarily small neighborhood of the origin, subject to optimal cost. Simulation results and comprehensive comparisons on a prototype USV demonstrate remarkable effectiveness and superiority.

Original languageEnglish
Article number9154585
Pages (from-to)3034-3045
Number of pages12
JournalIEEE Transactions on Neural Networks and Learning Systems
Issue number7
Publication statusPublished - 2021 Jul


  • Completely unknown dynamics
  • optimal tracking control
  • reinforcement earning-based control
  • unknown dead-zone input nonlinearities
  • unmanned surface vehicle (USV)

ASJC Scopus subject areas

  • Software
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence


Dive into the research topics of 'Reinforcement Learning-Based Optimal Tracking Control of an Unknown Unmanned Surface Vehicle'. Together they form a unique fingerprint.

Cite this