Constructing and evaluating a novel crowdsourcing-based paraphrased opinion spam dataset

Seongsoon Kim, Seongwoon Lee, Donghyeon Park, Jaewoo Kang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Opinion spam, intentionally written by spammers who do not have actual experience with services or products, has recently become a factor that undermines the credibility of information online. In recent years, studies have attempted to detect opinion spam using machine learning algorithms. However, limitations of gold-standard spam datasets still prove to be a major obstacle in opinion spam research. In this paper, we introduce a novel dataset called Paraphrased OPinion Spam (POPS), which contains a new type of review spam that imitates real human opinions using crowdsourcing. To create such a seemingly truthful review spam dataset, we asked task participants to paraphrase truthful reviews, and include factual information and domain knowledge in their reviews. The classification experiments and semantic analysis results show that our POPS dataset most linguistically and semantically resembles truthful reviews. We believe that our new deceptive opinion spam dataset1 will help advance opinion spam research.

Original languageEnglish
Title of host publication26th International World Wide Web Conference, WWW 2017
PublisherInternational World Wide Web Conferences Steering Committee
Pages827-836
Number of pages10
ISBN (Print)9781450349147
DOIs
Publication statusPublished - 2017 Jan 1
Event26th International World Wide Web Conference, WWW 2017 - Perth, Australia
Duration: 2017 Apr 32017 Apr 7

Other

Other26th International World Wide Web Conference, WWW 2017
CountryAustralia
CityPerth
Period17/4/317/4/7

Fingerprint

Learning algorithms
Learning systems
Gold
Semantics
Experiments

Keywords

  • Crowdsourcing
  • Deceptive opinion spam
  • Paraphrased opinion spam

ASJC Scopus subject areas

  • Software
  • Computer Networks and Communications

Cite this

Kim, S., Lee, S., Park, D., & Kang, J. (2017). Constructing and evaluating a novel crowdsourcing-based paraphrased opinion spam dataset. In 26th International World Wide Web Conference, WWW 2017 (pp. 827-836). [3052607] International World Wide Web Conferences Steering Committee. https://doi.org/10.1145/3038912.3052607

Constructing and evaluating a novel crowdsourcing-based paraphrased opinion spam dataset. / Kim, Seongsoon; Lee, Seongwoon; Park, Donghyeon; Kang, Jaewoo.

26th International World Wide Web Conference, WWW 2017. International World Wide Web Conferences Steering Committee, 2017. p. 827-836 3052607.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kim, S, Lee, S, Park, D & Kang, J 2017, Constructing and evaluating a novel crowdsourcing-based paraphrased opinion spam dataset. in 26th International World Wide Web Conference, WWW 2017., 3052607, International World Wide Web Conferences Steering Committee, pp. 827-836, 26th International World Wide Web Conference, WWW 2017, Perth, Australia, 17/4/3. https://doi.org/10.1145/3038912.3052607
Kim S, Lee S, Park D, Kang J. Constructing and evaluating a novel crowdsourcing-based paraphrased opinion spam dataset. In 26th International World Wide Web Conference, WWW 2017. International World Wide Web Conferences Steering Committee. 2017. p. 827-836. 3052607 https://doi.org/10.1145/3038912.3052607
Kim, Seongsoon ; Lee, Seongwoon ; Park, Donghyeon ; Kang, Jaewoo. / Constructing and evaluating a novel crowdsourcing-based paraphrased opinion spam dataset. 26th International World Wide Web Conference, WWW 2017. International World Wide Web Conferences Steering Committee, 2017. pp. 827-836
@inproceedings{667fa64fd5604ba4b6c3b61c5609f6aa,
title = "Constructing and evaluating a novel crowdsourcing-based paraphrased opinion spam dataset",
abstract = "Opinion spam, intentionally written by spammers who do not have actual experience with services or products, has recently become a factor that undermines the credibility of information online. In recent years, studies have attempted to detect opinion spam using machine learning algorithms. However, limitations of gold-standard spam datasets still prove to be a major obstacle in opinion spam research. In this paper, we introduce a novel dataset called Paraphrased OPinion Spam (POPS), which contains a new type of review spam that imitates real human opinions using crowdsourcing. To create such a seemingly truthful review spam dataset, we asked task participants to paraphrase truthful reviews, and include factual information and domain knowledge in their reviews. The classification experiments and semantic analysis results show that our POPS dataset most linguistically and semantically resembles truthful reviews. We believe that our new deceptive opinion spam dataset1 will help advance opinion spam research.",
keywords = "Crowdsourcing, Deceptive opinion spam, Paraphrased opinion spam",
author = "Seongsoon Kim and Seongwoon Lee and Donghyeon Park and Jaewoo Kang",
year = "2017",
month = "1",
day = "1",
doi = "10.1145/3038912.3052607",
language = "English",
isbn = "9781450349147",
pages = "827--836",
booktitle = "26th International World Wide Web Conference, WWW 2017",
publisher = "International World Wide Web Conferences Steering Committee",

}

TY - GEN

T1 - Constructing and evaluating a novel crowdsourcing-based paraphrased opinion spam dataset

AU - Kim, Seongsoon

AU - Lee, Seongwoon

AU - Park, Donghyeon

AU - Kang, Jaewoo

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Opinion spam, intentionally written by spammers who do not have actual experience with services or products, has recently become a factor that undermines the credibility of information online. In recent years, studies have attempted to detect opinion spam using machine learning algorithms. However, limitations of gold-standard spam datasets still prove to be a major obstacle in opinion spam research. In this paper, we introduce a novel dataset called Paraphrased OPinion Spam (POPS), which contains a new type of review spam that imitates real human opinions using crowdsourcing. To create such a seemingly truthful review spam dataset, we asked task participants to paraphrase truthful reviews, and include factual information and domain knowledge in their reviews. The classification experiments and semantic analysis results show that our POPS dataset most linguistically and semantically resembles truthful reviews. We believe that our new deceptive opinion spam dataset1 will help advance opinion spam research.

AB - Opinion spam, intentionally written by spammers who do not have actual experience with services or products, has recently become a factor that undermines the credibility of information online. In recent years, studies have attempted to detect opinion spam using machine learning algorithms. However, limitations of gold-standard spam datasets still prove to be a major obstacle in opinion spam research. In this paper, we introduce a novel dataset called Paraphrased OPinion Spam (POPS), which contains a new type of review spam that imitates real human opinions using crowdsourcing. To create such a seemingly truthful review spam dataset, we asked task participants to paraphrase truthful reviews, and include factual information and domain knowledge in their reviews. The classification experiments and semantic analysis results show that our POPS dataset most linguistically and semantically resembles truthful reviews. We believe that our new deceptive opinion spam dataset1 will help advance opinion spam research.

KW - Crowdsourcing

KW - Deceptive opinion spam

KW - Paraphrased opinion spam

UR - http://www.scopus.com/inward/record.url?scp=85051552928&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85051552928&partnerID=8YFLogxK

U2 - 10.1145/3038912.3052607

DO - 10.1145/3038912.3052607

M3 - Conference contribution

AN - SCOPUS:85051552928

SN - 9781450349147

SP - 827

EP - 836

BT - 26th International World Wide Web Conference, WWW 2017

PB - International World Wide Web Conferences Steering Committee

ER -