Prioritized image and video colorization based on Gaussian pyramid of gradient images

Dae Young Hyun, Jun Hee Heu, Chang-Su Kim, Sang Uk Lee

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

We propose a colorization algorithm for grayscale images and videos that demands simple user scribbles on a selected set of source pixels. The proposed algorithm colorizes non source pixels by propagating the colors of source pixels. To achieve reliable colorization results, we adopt a prioritization approach, which colorizes large smooth regions first and small detailed regions later using a Gaussian pyramid of gradient images. At each level of the pyramid, we determine the priority of each non source pixel using the luminance similarities to neighboring pixels and the geometric proximities to source pixels. Then we repeatedly colorize the pixel with the highest priority by interpolating the colors of neighboring source pixels. Also, for video colorization, the proposed algorithm transfers the colors of the first frame to subsequent frames based on optical flow estimation. To correct incorrectly transferred colors, the accuracies of transferred colors are calculated, and inaccurate colors are refined using neighboring colors. Simulation results show that the proposed algorithm provides accurate and reliable colorization results. We also demonstrate that the proposed algorithm can be used for color restoration and recolorization.

Original languageEnglish
Article number023027
JournalJournal of Electronic Imaging
Volume21
Issue number2
DOIs
Publication statusPublished - 2012 Jan 1

Fingerprint

pyramids
Pixels
pixels
Color
color
gradients
Optical flows
luminance
restoration
Restoration
proximity
Luminance

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Computer Science Applications
  • Atomic and Molecular Physics, and Optics

Cite this

Prioritized image and video colorization based on Gaussian pyramid of gradient images. / Hyun, Dae Young; Heu, Jun Hee; Kim, Chang-Su; Lee, Sang Uk.

In: Journal of Electronic Imaging, Vol. 21, No. 2, 023027, 01.01.2012.

Research output: Contribution to journalArticle

@article{345e5d6457c949edb0f8974dfac25e81,
title = "Prioritized image and video colorization based on Gaussian pyramid of gradient images",
abstract = "We propose a colorization algorithm for grayscale images and videos that demands simple user scribbles on a selected set of source pixels. The proposed algorithm colorizes non source pixels by propagating the colors of source pixels. To achieve reliable colorization results, we adopt a prioritization approach, which colorizes large smooth regions first and small detailed regions later using a Gaussian pyramid of gradient images. At each level of the pyramid, we determine the priority of each non source pixel using the luminance similarities to neighboring pixels and the geometric proximities to source pixels. Then we repeatedly colorize the pixel with the highest priority by interpolating the colors of neighboring source pixels. Also, for video colorization, the proposed algorithm transfers the colors of the first frame to subsequent frames based on optical flow estimation. To correct incorrectly transferred colors, the accuracies of transferred colors are calculated, and inaccurate colors are refined using neighboring colors. Simulation results show that the proposed algorithm provides accurate and reliable colorization results. We also demonstrate that the proposed algorithm can be used for color restoration and recolorization.",
author = "Hyun, {Dae Young} and Heu, {Jun Hee} and Chang-Su Kim and Lee, {Sang Uk}",
year = "2012",
month = "1",
day = "1",
doi = "10.1117/1.JEI.21.2.023027",
language = "English",
volume = "21",
journal = "Journal of Electronic Imaging",
issn = "1017-9909",
publisher = "SPIE",
number = "2",

}

TY - JOUR

T1 - Prioritized image and video colorization based on Gaussian pyramid of gradient images

AU - Hyun, Dae Young

AU - Heu, Jun Hee

AU - Kim, Chang-Su

AU - Lee, Sang Uk

PY - 2012/1/1

Y1 - 2012/1/1

N2 - We propose a colorization algorithm for grayscale images and videos that demands simple user scribbles on a selected set of source pixels. The proposed algorithm colorizes non source pixels by propagating the colors of source pixels. To achieve reliable colorization results, we adopt a prioritization approach, which colorizes large smooth regions first and small detailed regions later using a Gaussian pyramid of gradient images. At each level of the pyramid, we determine the priority of each non source pixel using the luminance similarities to neighboring pixels and the geometric proximities to source pixels. Then we repeatedly colorize the pixel with the highest priority by interpolating the colors of neighboring source pixels. Also, for video colorization, the proposed algorithm transfers the colors of the first frame to subsequent frames based on optical flow estimation. To correct incorrectly transferred colors, the accuracies of transferred colors are calculated, and inaccurate colors are refined using neighboring colors. Simulation results show that the proposed algorithm provides accurate and reliable colorization results. We also demonstrate that the proposed algorithm can be used for color restoration and recolorization.

AB - We propose a colorization algorithm for grayscale images and videos that demands simple user scribbles on a selected set of source pixels. The proposed algorithm colorizes non source pixels by propagating the colors of source pixels. To achieve reliable colorization results, we adopt a prioritization approach, which colorizes large smooth regions first and small detailed regions later using a Gaussian pyramid of gradient images. At each level of the pyramid, we determine the priority of each non source pixel using the luminance similarities to neighboring pixels and the geometric proximities to source pixels. Then we repeatedly colorize the pixel with the highest priority by interpolating the colors of neighboring source pixels. Also, for video colorization, the proposed algorithm transfers the colors of the first frame to subsequent frames based on optical flow estimation. To correct incorrectly transferred colors, the accuracies of transferred colors are calculated, and inaccurate colors are refined using neighboring colors. Simulation results show that the proposed algorithm provides accurate and reliable colorization results. We also demonstrate that the proposed algorithm can be used for color restoration and recolorization.

UR - http://www.scopus.com/inward/record.url?scp=84906761807&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84906761807&partnerID=8YFLogxK

U2 - 10.1117/1.JEI.21.2.023027

DO - 10.1117/1.JEI.21.2.023027

M3 - Article

AN - SCOPUS:84906761807

VL - 21

JO - Journal of Electronic Imaging

JF - Journal of Electronic Imaging

SN - 1017-9909

IS - 2

M1 - 023027

ER -