Video deraining and desnowing using temporal correlation and low-rank matrix completion

Jin Hwan Kim, Jae Young Sim, Chang-Su Kim

Research output: Contribution to journalArticle

50 Citations (Scopus)

Abstract

A novel algorithm to remove rain or snow streaks from a video sequence using temporal correlation and low-rank matrix completion is proposed in this paper. Based on the observation that rain streaks are too small and move too fast to affect the optical flow estimation between consecutive frames, we obtain an initial rain map by subtracting temporally warped frames from a current frame. Then, we decompose the initial rain map into basis vectors based on the sparse representation, and classify those basis vectors into rain streak ones and outliers with a support vector machine. We then refine the rain map by excluding the outliers. Finally, we remove the detected rain streaks by employing a low-rank matrix completion technique. Furthermore, we extend the proposed algorithm to stereo video deraining. Experimental results demonstrate that the proposed algorithm detects and removes rain or snow streaks efficiently, outperforming conventional algorithms.

Original languageEnglish
Article numberA9
Pages (from-to)2658-2670
Number of pages13
JournalIEEE Transactions on Image Processing
Volume24
Issue number9
DOIs
Publication statusPublished - 2015 Sep 1

Fingerprint

Rain
Snow
Optical flows
Support vector machines

Keywords

  • Desnowing
  • Low rank matrix completion and sparse representation
  • Rain streak removal
  • Video deraining

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Software

Cite this

Video deraining and desnowing using temporal correlation and low-rank matrix completion. / Kim, Jin Hwan; Sim, Jae Young; Kim, Chang-Su.

In: IEEE Transactions on Image Processing, Vol. 24, No. 9, A9, 01.09.2015, p. 2658-2670.

Research output: Contribution to journalArticle

@article{a4c2153e568d4863b66317e61221aa3b,
title = "Video deraining and desnowing using temporal correlation and low-rank matrix completion",
abstract = "A novel algorithm to remove rain or snow streaks from a video sequence using temporal correlation and low-rank matrix completion is proposed in this paper. Based on the observation that rain streaks are too small and move too fast to affect the optical flow estimation between consecutive frames, we obtain an initial rain map by subtracting temporally warped frames from a current frame. Then, we decompose the initial rain map into basis vectors based on the sparse representation, and classify those basis vectors into rain streak ones and outliers with a support vector machine. We then refine the rain map by excluding the outliers. Finally, we remove the detected rain streaks by employing a low-rank matrix completion technique. Furthermore, we extend the proposed algorithm to stereo video deraining. Experimental results demonstrate that the proposed algorithm detects and removes rain or snow streaks efficiently, outperforming conventional algorithms.",
keywords = "Desnowing, Low rank matrix completion and sparse representation, Rain streak removal, Video deraining",
author = "Kim, {Jin Hwan} and Sim, {Jae Young} and Chang-Su Kim",
year = "2015",
month = "9",
day = "1",
doi = "10.1109/TIP.2015.2428933",
language = "English",
volume = "24",
pages = "2658--2670",
journal = "IEEE Transactions on Image Processing",
issn = "1057-7149",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "9",

}

TY - JOUR

T1 - Video deraining and desnowing using temporal correlation and low-rank matrix completion

AU - Kim, Jin Hwan

AU - Sim, Jae Young

AU - Kim, Chang-Su

PY - 2015/9/1

Y1 - 2015/9/1

N2 - A novel algorithm to remove rain or snow streaks from a video sequence using temporal correlation and low-rank matrix completion is proposed in this paper. Based on the observation that rain streaks are too small and move too fast to affect the optical flow estimation between consecutive frames, we obtain an initial rain map by subtracting temporally warped frames from a current frame. Then, we decompose the initial rain map into basis vectors based on the sparse representation, and classify those basis vectors into rain streak ones and outliers with a support vector machine. We then refine the rain map by excluding the outliers. Finally, we remove the detected rain streaks by employing a low-rank matrix completion technique. Furthermore, we extend the proposed algorithm to stereo video deraining. Experimental results demonstrate that the proposed algorithm detects and removes rain or snow streaks efficiently, outperforming conventional algorithms.

AB - A novel algorithm to remove rain or snow streaks from a video sequence using temporal correlation and low-rank matrix completion is proposed in this paper. Based on the observation that rain streaks are too small and move too fast to affect the optical flow estimation between consecutive frames, we obtain an initial rain map by subtracting temporally warped frames from a current frame. Then, we decompose the initial rain map into basis vectors based on the sparse representation, and classify those basis vectors into rain streak ones and outliers with a support vector machine. We then refine the rain map by excluding the outliers. Finally, we remove the detected rain streaks by employing a low-rank matrix completion technique. Furthermore, we extend the proposed algorithm to stereo video deraining. Experimental results demonstrate that the proposed algorithm detects and removes rain or snow streaks efficiently, outperforming conventional algorithms.

KW - Desnowing

KW - Low rank matrix completion and sparse representation

KW - Rain streak removal

KW - Video deraining

UR - http://www.scopus.com/inward/record.url?scp=84930506287&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84930506287&partnerID=8YFLogxK

U2 - 10.1109/TIP.2015.2428933

DO - 10.1109/TIP.2015.2428933

M3 - Article

VL - 24

SP - 2658

EP - 2670

JO - IEEE Transactions on Image Processing

JF - IEEE Transactions on Image Processing

SN - 1057-7149

IS - 9

M1 - A9

ER -