Nonlinear ego-motion estimation from optical flow for online control of a quadrotor UAV

Volker Grabe, Heinrich Bulthoff, Davide Scaramuzza, Paolo Robuffo Giordano

Research output: Contribution to journalArticle

23 Citations (Scopus)

Abstract

For the control of unmanned aerial vehicles (UAVs) in GPS-denied environments, cameras have been widely exploited as the main sensory modality for addressing the UAV state estimation problem. However, the use of visual information for ego-motion estimation presents several theoretical and practical difficulties, such as data association, occlusions, and lack of direct metric information when exploiting monocular cameras. In this paper, we address these issues by considering a quadrotor UAV equipped with an onboard monocular camera and an inertial measurement unit (IMU). First, we propose a robust ego-motion estimation algorithm for recovering the UAV scaled linear velocity and angular velocity from optical flow by exploiting the so-called continuous homography constraint in the presence of planar scenes. Then, we address the problem of retrieving the (unknown) metric scale by fusing the visual information with measurements from the onboard IMU. To this end, two different estimation strategies are proposed and critically compared: a first exploiting the classical extended Kalman filter (EKF) formulation, and a second one based on a novel nonlinear estimation framework. The main advantage of the latter scheme lies in the possibility of imposing a desired transient response to the estimation error when the camera moves with a constant acceleration norm with respect to the observed plane. We indeed show that, when compared against the EKF on the same trajectory and sensory data, the nonlinear scheme yields considerably superior performance in terms of convergence rate and predictability of the estimation. The paper is then concluded by an extensive experimental validation, including an onboard closed-loop control of a real quadrotor UAV meant to demonstrate the robustness of our approach in real-world conditions.

Original languageEnglish
Pages (from-to)1114-1135
Number of pages22
JournalInternational Journal of Robotics Research
Volume34
Issue number8
DOIs
Publication statusPublished - 2015 Jul 3

Fingerprint

On-line Control
Nonlinear Estimation
Optical flows
Optical Flow
Motion Estimation
Motion estimation
Unmanned aerial vehicles (UAV)
Camera
Cameras
Units of measurement
Kalman Filter
Extended Kalman filters
Homography
Metric
Data Association
Unit
Transient Response
Robust Estimation
Closed-loop Control
Experimental Validation

Keywords

  • aerial robotics
  • Sensor fusion
  • visual-based control

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Mechanical Engineering
  • Software
  • Artificial Intelligence
  • Applied Mathematics
  • Modelling and Simulation

Cite this

Nonlinear ego-motion estimation from optical flow for online control of a quadrotor UAV. / Grabe, Volker; Bulthoff, Heinrich; Scaramuzza, Davide; Giordano, Paolo Robuffo.

In: International Journal of Robotics Research, Vol. 34, No. 8, 03.07.2015, p. 1114-1135.

Research output: Contribution to journalArticle

Grabe, Volker ; Bulthoff, Heinrich ; Scaramuzza, Davide ; Giordano, Paolo Robuffo. / Nonlinear ego-motion estimation from optical flow for online control of a quadrotor UAV. In: International Journal of Robotics Research. 2015 ; Vol. 34, No. 8. pp. 1114-1135.
@article{97334203bbfa4abf8e6a9d40260f497e,
title = "Nonlinear ego-motion estimation from optical flow for online control of a quadrotor UAV",
abstract = "For the control of unmanned aerial vehicles (UAVs) in GPS-denied environments, cameras have been widely exploited as the main sensory modality for addressing the UAV state estimation problem. However, the use of visual information for ego-motion estimation presents several theoretical and practical difficulties, such as data association, occlusions, and lack of direct metric information when exploiting monocular cameras. In this paper, we address these issues by considering a quadrotor UAV equipped with an onboard monocular camera and an inertial measurement unit (IMU). First, we propose a robust ego-motion estimation algorithm for recovering the UAV scaled linear velocity and angular velocity from optical flow by exploiting the so-called continuous homography constraint in the presence of planar scenes. Then, we address the problem of retrieving the (unknown) metric scale by fusing the visual information with measurements from the onboard IMU. To this end, two different estimation strategies are proposed and critically compared: a first exploiting the classical extended Kalman filter (EKF) formulation, and a second one based on a novel nonlinear estimation framework. The main advantage of the latter scheme lies in the possibility of imposing a desired transient response to the estimation error when the camera moves with a constant acceleration norm with respect to the observed plane. We indeed show that, when compared against the EKF on the same trajectory and sensory data, the nonlinear scheme yields considerably superior performance in terms of convergence rate and predictability of the estimation. The paper is then concluded by an extensive experimental validation, including an onboard closed-loop control of a real quadrotor UAV meant to demonstrate the robustness of our approach in real-world conditions.",
keywords = "aerial robotics, Sensor fusion, visual-based control",
author = "Volker Grabe and Heinrich Bulthoff and Davide Scaramuzza and Giordano, {Paolo Robuffo}",
year = "2015",
month = "7",
day = "3",
doi = "10.1177/0278364915578646",
language = "English",
volume = "34",
pages = "1114--1135",
journal = "International Journal of Robotics Research",
issn = "0278-3649",
publisher = "SAGE Publications Inc.",
number = "8",

}

TY - JOUR

T1 - Nonlinear ego-motion estimation from optical flow for online control of a quadrotor UAV

AU - Grabe, Volker

AU - Bulthoff, Heinrich

AU - Scaramuzza, Davide

AU - Giordano, Paolo Robuffo

PY - 2015/7/3

Y1 - 2015/7/3

N2 - For the control of unmanned aerial vehicles (UAVs) in GPS-denied environments, cameras have been widely exploited as the main sensory modality for addressing the UAV state estimation problem. However, the use of visual information for ego-motion estimation presents several theoretical and practical difficulties, such as data association, occlusions, and lack of direct metric information when exploiting monocular cameras. In this paper, we address these issues by considering a quadrotor UAV equipped with an onboard monocular camera and an inertial measurement unit (IMU). First, we propose a robust ego-motion estimation algorithm for recovering the UAV scaled linear velocity and angular velocity from optical flow by exploiting the so-called continuous homography constraint in the presence of planar scenes. Then, we address the problem of retrieving the (unknown) metric scale by fusing the visual information with measurements from the onboard IMU. To this end, two different estimation strategies are proposed and critically compared: a first exploiting the classical extended Kalman filter (EKF) formulation, and a second one based on a novel nonlinear estimation framework. The main advantage of the latter scheme lies in the possibility of imposing a desired transient response to the estimation error when the camera moves with a constant acceleration norm with respect to the observed plane. We indeed show that, when compared against the EKF on the same trajectory and sensory data, the nonlinear scheme yields considerably superior performance in terms of convergence rate and predictability of the estimation. The paper is then concluded by an extensive experimental validation, including an onboard closed-loop control of a real quadrotor UAV meant to demonstrate the robustness of our approach in real-world conditions.

AB - For the control of unmanned aerial vehicles (UAVs) in GPS-denied environments, cameras have been widely exploited as the main sensory modality for addressing the UAV state estimation problem. However, the use of visual information for ego-motion estimation presents several theoretical and practical difficulties, such as data association, occlusions, and lack of direct metric information when exploiting monocular cameras. In this paper, we address these issues by considering a quadrotor UAV equipped with an onboard monocular camera and an inertial measurement unit (IMU). First, we propose a robust ego-motion estimation algorithm for recovering the UAV scaled linear velocity and angular velocity from optical flow by exploiting the so-called continuous homography constraint in the presence of planar scenes. Then, we address the problem of retrieving the (unknown) metric scale by fusing the visual information with measurements from the onboard IMU. To this end, two different estimation strategies are proposed and critically compared: a first exploiting the classical extended Kalman filter (EKF) formulation, and a second one based on a novel nonlinear estimation framework. The main advantage of the latter scheme lies in the possibility of imposing a desired transient response to the estimation error when the camera moves with a constant acceleration norm with respect to the observed plane. We indeed show that, when compared against the EKF on the same trajectory and sensory data, the nonlinear scheme yields considerably superior performance in terms of convergence rate and predictability of the estimation. The paper is then concluded by an extensive experimental validation, including an onboard closed-loop control of a real quadrotor UAV meant to demonstrate the robustness of our approach in real-world conditions.

KW - aerial robotics

KW - Sensor fusion

KW - visual-based control

UR - http://www.scopus.com/inward/record.url?scp=84935095381&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84935095381&partnerID=8YFLogxK

U2 - 10.1177/0278364915578646

DO - 10.1177/0278364915578646

M3 - Article

AN - SCOPUS:84935095381

VL - 34

SP - 1114

EP - 1135

JO - International Journal of Robotics Research

JF - International Journal of Robotics Research

SN - 0278-3649

IS - 8

ER -