Page-based anomaly detection in large scale web clusters using adaptive MapReduce

Junsup Lee, Sungdeok Cha

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

While anomaly detection systems typically work on single server, most commercial web sites operate cluster environments, and user queries trigger transactions scattered through multiple servers. For this reason, anomaly detectors in a same server farm should communicate with each other to integrate their partial profile. In this paper, we describe a real-time distributed anomaly detection system that can deal with over one billion transactions per day. In our system, base on Google MapReduce algorithm, an anomaly detector in each node shares profiles of user behaviors and propagates intruder information to reduce false alarms. We evaluated our system using web log data from www.microsoft.com. The web log data, about 250GB in size, contains over one billion transactions recorded in a day.

Original languageEnglish
Title of host publicationRecent Advances in Intrusion Detection - 11th International Symposium, RAID 2008, Proceedings
Pages404-405
Number of pages2
DOIs
Publication statusPublished - 2008 Dec 1
EventRecent Advances in Intrusion Detection - 11th International Symposium, RAID 2008, Proceedings - Cambridge, MA, United States
Duration: 2008 Sep 152008 Sep 17

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume5230 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

OtherRecent Advances in Intrusion Detection - 11th International Symposium, RAID 2008, Proceedings
CountryUnited States
CityCambridge, MA
Period08/9/1508/9/17

    Fingerprint

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Lee, J., & Cha, S. (2008). Page-based anomaly detection in large scale web clusters using adaptive MapReduce. In Recent Advances in Intrusion Detection - 11th International Symposium, RAID 2008, Proceedings (pp. 404-405). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5230 LNCS). https://doi.org/10.1007/978-3-540-87403-4_28