TY - JOUR
T1 - Automatic extraction of named entities of cyber threats using a deep Bi-LSTM-CRF network
AU - Kim, Gyeongmin
AU - Lee, Chanhee
AU - Jo, Jaechoon
AU - Lim, Heuiseok
N1 - Funding Information:
Funding was provide by Korea Creative Content Agency (Grant No. R2017030045).
Publisher Copyright:
© 2020, Springer-Verlag GmbH Germany, part of Springer Nature.
Copyright:
Copyright 2020 Elsevier B.V., All rights reserved.
PY - 2020
Y1 - 2020
N2 - Countless cyber threat intelligence (CTI) reports are used by companies around the world on a daily basis for security reasons. To secure critical cybersecurity information, analysts and individuals should accordingly analyze information on threats and vulnerabilities. However, analyzing such overwhelming volumes of reports requires considerable time and effort. In this study, we propose a novel approach that automatically extracts core information from CTI reports using a named entity recognition (NER) system. During the process of constructing our proposed NER system, we defined meaningful keywords in the security domain as entities, including malware, domain/URL, IP address, Hash, and Common Vulnerabilities and Exposures. Furthermore, we linked these keywords with the words extracted from the text data of the report. To achieve a higher performance, we utilized the character-level feature vector as an input to bidirectional long-short-term memory using a conditional random field network. We finally achieved an average F1-score of 75.05%. We release 498,000 tag datasets created during our research.
AB - Countless cyber threat intelligence (CTI) reports are used by companies around the world on a daily basis for security reasons. To secure critical cybersecurity information, analysts and individuals should accordingly analyze information on threats and vulnerabilities. However, analyzing such overwhelming volumes of reports requires considerable time and effort. In this study, we propose a novel approach that automatically extracts core information from CTI reports using a named entity recognition (NER) system. During the process of constructing our proposed NER system, we defined meaningful keywords in the security domain as entities, including malware, domain/URL, IP address, Hash, and Common Vulnerabilities and Exposures. Furthermore, we linked these keywords with the words extracted from the text data of the report. To achieve a higher performance, we utilized the character-level feature vector as an input to bidirectional long-short-term memory using a conditional random field network. We finally achieved an average F1-score of 75.05%. We release 498,000 tag datasets created during our research.
KW - Bidirectional long-short-term memory conditional random field
KW - Cyber threat intelligence
KW - Cybersecurity
KW - Named entity recognition
KW - Vulnerability
UR - http://www.scopus.com/inward/record.url?scp=85085090748&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85085090748&partnerID=8YFLogxK
U2 - 10.1007/s13042-020-01122-6
DO - 10.1007/s13042-020-01122-6
M3 - Article
AN - SCOPUS:85085090748
VL - 11
SP - 2341
EP - 2355
JO - International Journal of Machine Learning and Cybernetics
JF - International Journal of Machine Learning and Cybernetics
SN - 1868-8071
IS - 10
ER -