GPU-based real-time super-resolution system for high-quality UHD video up-conversion

Dae Yeol Lee, Jooyoung Lee, Ji Hoon Choi, Jong-Ok Kim, Hui Yong Kim, Jin Soo Choi

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Super-resolution (SR) is a technique that reconstructs high-resolution images using the information present in low-resolution images. Due to their potentials of being used in wide range of image and video applications, various SR algorithms have been studied and proposed in the literature until recently. However, many of the algorithms provide insufficient perceptual quality, possess high computational complexity, or have high memory requirement, which make them hard to apply on consumer-level products. Therefore, in this paper we propose an effective super-resolution method that not only provides an excellent visual quality but also a high-speed performance suitable for video conversion applications. The proposed super-resolution adopts self-similarity framework, which reconstructs the high-frequency (HF) information of the high-resolution image by referring to the image pairs generated from self-similar regions. The method further enhances the perceptual sharpness of the video through region-adaptive HF enhancement algorithm and applies iterative back projection to maintain its consistency with the input image. The proposed method is suitable for parallel processing and therefore is able to provide its superb visual quality on a high conversion speed through GPU-based acceleration. The experimental results show that the proposed method has superior HF reconstruction performance compared to other state-of-the-art upscaling solutions and is able to generate videos that are visually as sharp as the original high-resolution videos. On a single PC with four GPUs, the proposed method can convert Full HD resolution video into UHD resolution with real-time conversion speed. Due to its fast and high-quality conversion capability, the proposed method can be applied on various consumer products such as UHDTV, surveillance system, and mobile devices.

Original languageEnglish
Pages (from-to)456-484
Number of pages29
JournalJournal of Supercomputing
Volume74
Issue number1
DOIs
Publication statusPublished - 2018 Jan 1

Fingerprint

Upconversion
Video Quality
Super-resolution
Image resolution
Real-time
Consumer products
High Resolution
Mobile devices
Computational complexity
Data storage equipment
Upscaling
Sharpness
Self-similarity
Processing
Parallel Processing
Mobile Devices
Surveillance
Convert
Graphics processing unit
Computational Complexity

Keywords

  • GPU
  • Real-time
  • Region-adaptive
  • Self-similarity
  • Super-resolution
  • UHD video

ASJC Scopus subject areas

  • Software
  • Theoretical Computer Science
  • Information Systems
  • Hardware and Architecture

Cite this

GPU-based real-time super-resolution system for high-quality UHD video up-conversion. / Lee, Dae Yeol; Lee, Jooyoung; Choi, Ji Hoon; Kim, Jong-Ok; Kim, Hui Yong; Choi, Jin Soo.

In: Journal of Supercomputing, Vol. 74, No. 1, 01.01.2018, p. 456-484.

Research output: Contribution to journalArticle

Lee, Dae Yeol ; Lee, Jooyoung ; Choi, Ji Hoon ; Kim, Jong-Ok ; Kim, Hui Yong ; Choi, Jin Soo. / GPU-based real-time super-resolution system for high-quality UHD video up-conversion. In: Journal of Supercomputing. 2018 ; Vol. 74, No. 1. pp. 456-484.
@article{f8784b628c3f43c484a78f334d2c4083,
title = "GPU-based real-time super-resolution system for high-quality UHD video up-conversion",
abstract = "Super-resolution (SR) is a technique that reconstructs high-resolution images using the information present in low-resolution images. Due to their potentials of being used in wide range of image and video applications, various SR algorithms have been studied and proposed in the literature until recently. However, many of the algorithms provide insufficient perceptual quality, possess high computational complexity, or have high memory requirement, which make them hard to apply on consumer-level products. Therefore, in this paper we propose an effective super-resolution method that not only provides an excellent visual quality but also a high-speed performance suitable for video conversion applications. The proposed super-resolution adopts self-similarity framework, which reconstructs the high-frequency (HF) information of the high-resolution image by referring to the image pairs generated from self-similar regions. The method further enhances the perceptual sharpness of the video through region-adaptive HF enhancement algorithm and applies iterative back projection to maintain its consistency with the input image. The proposed method is suitable for parallel processing and therefore is able to provide its superb visual quality on a high conversion speed through GPU-based acceleration. The experimental results show that the proposed method has superior HF reconstruction performance compared to other state-of-the-art upscaling solutions and is able to generate videos that are visually as sharp as the original high-resolution videos. On a single PC with four GPUs, the proposed method can convert Full HD resolution video into UHD resolution with real-time conversion speed. Due to its fast and high-quality conversion capability, the proposed method can be applied on various consumer products such as UHDTV, surveillance system, and mobile devices.",
keywords = "GPU, Real-time, Region-adaptive, Self-similarity, Super-resolution, UHD video",
author = "Lee, {Dae Yeol} and Jooyoung Lee and Choi, {Ji Hoon} and Jong-Ok Kim and Kim, {Hui Yong} and Choi, {Jin Soo}",
year = "2018",
month = "1",
day = "1",
doi = "10.1007/s11227-017-2136-1",
language = "English",
volume = "74",
pages = "456--484",
journal = "The Journal of Supercomputing",
issn = "0920-8542",
publisher = "Springer Netherlands",
number = "1",

}

TY - JOUR

T1 - GPU-based real-time super-resolution system for high-quality UHD video up-conversion

AU - Lee, Dae Yeol

AU - Lee, Jooyoung

AU - Choi, Ji Hoon

AU - Kim, Jong-Ok

AU - Kim, Hui Yong

AU - Choi, Jin Soo

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Super-resolution (SR) is a technique that reconstructs high-resolution images using the information present in low-resolution images. Due to their potentials of being used in wide range of image and video applications, various SR algorithms have been studied and proposed in the literature until recently. However, many of the algorithms provide insufficient perceptual quality, possess high computational complexity, or have high memory requirement, which make them hard to apply on consumer-level products. Therefore, in this paper we propose an effective super-resolution method that not only provides an excellent visual quality but also a high-speed performance suitable for video conversion applications. The proposed super-resolution adopts self-similarity framework, which reconstructs the high-frequency (HF) information of the high-resolution image by referring to the image pairs generated from self-similar regions. The method further enhances the perceptual sharpness of the video through region-adaptive HF enhancement algorithm and applies iterative back projection to maintain its consistency with the input image. The proposed method is suitable for parallel processing and therefore is able to provide its superb visual quality on a high conversion speed through GPU-based acceleration. The experimental results show that the proposed method has superior HF reconstruction performance compared to other state-of-the-art upscaling solutions and is able to generate videos that are visually as sharp as the original high-resolution videos. On a single PC with four GPUs, the proposed method can convert Full HD resolution video into UHD resolution with real-time conversion speed. Due to its fast and high-quality conversion capability, the proposed method can be applied on various consumer products such as UHDTV, surveillance system, and mobile devices.

AB - Super-resolution (SR) is a technique that reconstructs high-resolution images using the information present in low-resolution images. Due to their potentials of being used in wide range of image and video applications, various SR algorithms have been studied and proposed in the literature until recently. However, many of the algorithms provide insufficient perceptual quality, possess high computational complexity, or have high memory requirement, which make them hard to apply on consumer-level products. Therefore, in this paper we propose an effective super-resolution method that not only provides an excellent visual quality but also a high-speed performance suitable for video conversion applications. The proposed super-resolution adopts self-similarity framework, which reconstructs the high-frequency (HF) information of the high-resolution image by referring to the image pairs generated from self-similar regions. The method further enhances the perceptual sharpness of the video through region-adaptive HF enhancement algorithm and applies iterative back projection to maintain its consistency with the input image. The proposed method is suitable for parallel processing and therefore is able to provide its superb visual quality on a high conversion speed through GPU-based acceleration. The experimental results show that the proposed method has superior HF reconstruction performance compared to other state-of-the-art upscaling solutions and is able to generate videos that are visually as sharp as the original high-resolution videos. On a single PC with four GPUs, the proposed method can convert Full HD resolution video into UHD resolution with real-time conversion speed. Due to its fast and high-quality conversion capability, the proposed method can be applied on various consumer products such as UHDTV, surveillance system, and mobile devices.

KW - GPU

KW - Real-time

KW - Region-adaptive

KW - Self-similarity

KW - Super-resolution

KW - UHD video

UR - http://www.scopus.com/inward/record.url?scp=85029427362&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85029427362&partnerID=8YFLogxK

U2 - 10.1007/s11227-017-2136-1

DO - 10.1007/s11227-017-2136-1

M3 - Article

AN - SCOPUS:85029427362

VL - 74

SP - 456

EP - 484

JO - The Journal of Supercomputing

JF - The Journal of Supercomputing

SN - 0920-8542

IS - 1

ER -