Explaining and Interpreting LSTMs

Leila Arras, José Arjona-Medina, Michael Widrich, Grégoire Montavon, Michael Gillhofer, Klaus Robert Müller, Sepp Hochreiter, Wojciech Samek

Research output: Chapter in Book/Report/Conference proceedingChapter

1 Citation (Scopus)

Abstract

While neural networks have acted as a strong unifying force in the design of modern AI systems, the neural network architectures themselves remain highly heterogeneous due to the variety of tasks to be solved. In this chapter, we explore how to adapt the Layer-wise Relevance Propagation (LRP) technique used for explaining the predictions of feed-forward networks to the LSTM architecture used for sequential data modeling and forecasting. The special accumulators and gated interactions present in the LSTM require both a new propagation scheme and an extension of the underlying theoretical framework to deliver faithful explanations.

Original languageEnglish
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
PublisherSpringer Verlag
Pages211-238
Number of pages28
DOIs
Publication statusPublished - 2019 Jan 1

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11700 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Fingerprint

Neural Networks
Propagation
Neural networks
Feedforward Networks
Data Modeling
Network Architecture
Faithful
Network architecture
Data structures
Forecasting
Prediction
Interaction
Design
Architecture
Relevance
Framework

Keywords

  • Explainable artificial intelligence
  • Interpretability
  • LSTM
  • Model transparency
  • Recurrent neural networks

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Arras, L., Arjona-Medina, J., Widrich, M., Montavon, G., Gillhofer, M., Müller, K. R., ... Samek, W. (2019). Explaining and Interpreting LSTMs. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 211-238). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11700 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-28954-6_11

Explaining and Interpreting LSTMs. / Arras, Leila; Arjona-Medina, José; Widrich, Michael; Montavon, Grégoire; Gillhofer, Michael; Müller, Klaus Robert; Hochreiter, Sepp; Samek, Wojciech.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer Verlag, 2019. p. 211-238 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11700 LNCS).

Research output: Chapter in Book/Report/Conference proceedingChapter

Arras, L, Arjona-Medina, J, Widrich, M, Montavon, G, Gillhofer, M, Müller, KR, Hochreiter, S & Samek, W 2019, Explaining and Interpreting LSTMs. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11700 LNCS, Springer Verlag, pp. 211-238. https://doi.org/10.1007/978-3-030-28954-6_11
Arras L, Arjona-Medina J, Widrich M, Montavon G, Gillhofer M, Müller KR et al. Explaining and Interpreting LSTMs. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer Verlag. 2019. p. 211-238. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-28954-6_11
Arras, Leila ; Arjona-Medina, José ; Widrich, Michael ; Montavon, Grégoire ; Gillhofer, Michael ; Müller, Klaus Robert ; Hochreiter, Sepp ; Samek, Wojciech. / Explaining and Interpreting LSTMs. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer Verlag, 2019. pp. 211-238 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inbook{cf64791aede74f50a8290e126ed24474,
title = "Explaining and Interpreting LSTMs",
abstract = "While neural networks have acted as a strong unifying force in the design of modern AI systems, the neural network architectures themselves remain highly heterogeneous due to the variety of tasks to be solved. In this chapter, we explore how to adapt the Layer-wise Relevance Propagation (LRP) technique used for explaining the predictions of feed-forward networks to the LSTM architecture used for sequential data modeling and forecasting. The special accumulators and gated interactions present in the LSTM require both a new propagation scheme and an extension of the underlying theoretical framework to deliver faithful explanations.",
keywords = "Explainable artificial intelligence, Interpretability, LSTM, Model transparency, Recurrent neural networks",
author = "Leila Arras and Jos{\'e} Arjona-Medina and Michael Widrich and Gr{\'e}goire Montavon and Michael Gillhofer and M{\"u}ller, {Klaus Robert} and Sepp Hochreiter and Wojciech Samek",
year = "2019",
month = "1",
day = "1",
doi = "10.1007/978-3-030-28954-6_11",
language = "English",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "211--238",
booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

}

TY - CHAP

T1 - Explaining and Interpreting LSTMs

AU - Arras, Leila

AU - Arjona-Medina, José

AU - Widrich, Michael

AU - Montavon, Grégoire

AU - Gillhofer, Michael

AU - Müller, Klaus Robert

AU - Hochreiter, Sepp

AU - Samek, Wojciech

PY - 2019/1/1

Y1 - 2019/1/1

N2 - While neural networks have acted as a strong unifying force in the design of modern AI systems, the neural network architectures themselves remain highly heterogeneous due to the variety of tasks to be solved. In this chapter, we explore how to adapt the Layer-wise Relevance Propagation (LRP) technique used for explaining the predictions of feed-forward networks to the LSTM architecture used for sequential data modeling and forecasting. The special accumulators and gated interactions present in the LSTM require both a new propagation scheme and an extension of the underlying theoretical framework to deliver faithful explanations.

AB - While neural networks have acted as a strong unifying force in the design of modern AI systems, the neural network architectures themselves remain highly heterogeneous due to the variety of tasks to be solved. In this chapter, we explore how to adapt the Layer-wise Relevance Propagation (LRP) technique used for explaining the predictions of feed-forward networks to the LSTM architecture used for sequential data modeling and forecasting. The special accumulators and gated interactions present in the LSTM require both a new propagation scheme and an extension of the underlying theoretical framework to deliver faithful explanations.

KW - Explainable artificial intelligence

KW - Interpretability

KW - LSTM

KW - Model transparency

KW - Recurrent neural networks

UR - http://www.scopus.com/inward/record.url?scp=85072819669&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072819669&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-28954-6_11

DO - 10.1007/978-3-030-28954-6_11

M3 - Chapter

AN - SCOPUS:85072819669

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 211

EP - 238

BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

PB - Springer Verlag

ER -