Instance segmentation-based review photo validation scheme

Sungwoo Park, Jaeuk Moon, Seongkuk Cho, Eenjun Hwang

Research output: Contribution to journalArticlepeer-review

Abstract

User reviews of products in online shopping malls significantly influence the purchase decision of those products. Consequently, many shopping malls have diverse reward systems to encourage users to upload their reviews. In particular, for fashion products such as clothing, photo reviews are preferred over text reviews and are usually rewarded more. Despite the large number of irrelevant photo reviews for reward purposes, the traditional methods of manually filtering these reviews are expensive and time-consuming. Recently, various deep learning-based studies have been conducted using bounding box regression for photo review validation. In this paper, we propose a more effective review photo validation scheme based on instance segmentation and triplet loss. More specifically, we first identify the clothing in the review and commercial photos using the instance segmentation model. Then, we calculate their similarity using triplet loss to train the triplet network which determines whether they are the same product or not, and utilize both the segmentation model and triplet network for review photo validation. To evaluate the effectiveness of the proposed scheme, we conducted extensive experiments using a public fashion dataset. The experimental results show that our instance segmentation outperforms the bounding box models in both accuracy of the triplet network and accuracy of the review photo validation by up to 10%.

Original languageEnglish
Pages (from-to)3489-3510
Number of pages22
JournalJournal of Supercomputing
Volume79
Issue number3
DOIs
Publication statusPublished - 2023 Feb

Keywords

  • Deep learning
  • Instance segmentation
  • Mask R-CNN
  • Review photo validation
  • Triplet network

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Software
  • Information Systems
  • Hardware and Architecture

Fingerprint

Dive into the research topics of 'Instance segmentation-based review photo validation scheme'. Together they form a unique fingerprint.

Cite this