Predicting pairwise relations with neural similarity encoders

F. Horn, Klaus Muller

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Matrix factorization is at the heart of many machine learning algorithms, for example, dimensionality reduction (e.g. kernel PCA) or recommender systems relying on collaborative filtering. Understanding a singular value decomposition (SVD) of a matrix as a neural network optimization problem enables us to decompose large matrices efficiently while dealing naturally with missing values in the given matrix. But most importantly, it allows us to learn the connection between data points’ feature vectors and the matrix containing information about their pairwise relations. In this paper we introduce a novel neural network architecture termed similarity encoder (SimEc), which is designed to simultaneously factorize a given target matrix while also learning the mapping to project the data points’ feature vectors into a similarity preserving embedding space. This makes it possible to, for example, easily compute out-of-sample solutions for new data points. Additionally, we demonstrate that SimEc can preserve non-metric similarities and even predict multiple pairwise relations between data points at once.

Original languageEnglish
Pages (from-to)821-830
Number of pages10
JournalBulletin of the Polish Academy of Sciences: Technical Sciences
DOIs
Publication statusPublished - 2018 Jan 1

Fingerprint

coders
matrices
Neural networks
Collaborative filtering
machine learning
Recommender systems
Singular value decomposition
Network architecture
Factorization
factorization
embedding
Learning algorithms
preserving
learning
Learning systems
decomposition
optimization

Keywords

  • Dimensionality reduction
  • Kernel PCA
  • Matrix factorization
  • Neural networks
  • Similarity preserving embeddings
  • SVD

ASJC Scopus subject areas

  • Atomic and Molecular Physics, and Optics
  • Information Systems
  • Engineering(all)
  • Computer Networks and Communications
  • Artificial Intelligence

Cite this

Predicting pairwise relations with neural similarity encoders. / Horn, F.; Muller, Klaus.

In: Bulletin of the Polish Academy of Sciences: Technical Sciences, 01.01.2018, p. 821-830.

Research output: Contribution to journalArticle

@article{3344edcea6d44f4fa26eee0209c70238,
title = "Predicting pairwise relations with neural similarity encoders",
abstract = "Matrix factorization is at the heart of many machine learning algorithms, for example, dimensionality reduction (e.g. kernel PCA) or recommender systems relying on collaborative filtering. Understanding a singular value decomposition (SVD) of a matrix as a neural network optimization problem enables us to decompose large matrices efficiently while dealing naturally with missing values in the given matrix. But most importantly, it allows us to learn the connection between data points’ feature vectors and the matrix containing information about their pairwise relations. In this paper we introduce a novel neural network architecture termed similarity encoder (SimEc), which is designed to simultaneously factorize a given target matrix while also learning the mapping to project the data points’ feature vectors into a similarity preserving embedding space. This makes it possible to, for example, easily compute out-of-sample solutions for new data points. Additionally, we demonstrate that SimEc can preserve non-metric similarities and even predict multiple pairwise relations between data points at once.",
keywords = "Dimensionality reduction, Kernel PCA, Matrix factorization, Neural networks, Similarity preserving embeddings, SVD",
author = "F. Horn and Klaus Muller",
year = "2018",
month = "1",
day = "1",
doi = "10.24425/bpas.2018.125929",
language = "English",
pages = "821--830",
journal = "Bulletin of the Polish Academy of Sciences: Technical Sciences",
issn = "0239-7528",
publisher = "Palac Kultury i Nauki",

}

TY - JOUR

T1 - Predicting pairwise relations with neural similarity encoders

AU - Horn, F.

AU - Muller, Klaus

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Matrix factorization is at the heart of many machine learning algorithms, for example, dimensionality reduction (e.g. kernel PCA) or recommender systems relying on collaborative filtering. Understanding a singular value decomposition (SVD) of a matrix as a neural network optimization problem enables us to decompose large matrices efficiently while dealing naturally with missing values in the given matrix. But most importantly, it allows us to learn the connection between data points’ feature vectors and the matrix containing information about their pairwise relations. In this paper we introduce a novel neural network architecture termed similarity encoder (SimEc), which is designed to simultaneously factorize a given target matrix while also learning the mapping to project the data points’ feature vectors into a similarity preserving embedding space. This makes it possible to, for example, easily compute out-of-sample solutions for new data points. Additionally, we demonstrate that SimEc can preserve non-metric similarities and even predict multiple pairwise relations between data points at once.

AB - Matrix factorization is at the heart of many machine learning algorithms, for example, dimensionality reduction (e.g. kernel PCA) or recommender systems relying on collaborative filtering. Understanding a singular value decomposition (SVD) of a matrix as a neural network optimization problem enables us to decompose large matrices efficiently while dealing naturally with missing values in the given matrix. But most importantly, it allows us to learn the connection between data points’ feature vectors and the matrix containing information about their pairwise relations. In this paper we introduce a novel neural network architecture termed similarity encoder (SimEc), which is designed to simultaneously factorize a given target matrix while also learning the mapping to project the data points’ feature vectors into a similarity preserving embedding space. This makes it possible to, for example, easily compute out-of-sample solutions for new data points. Additionally, we demonstrate that SimEc can preserve non-metric similarities and even predict multiple pairwise relations between data points at once.

KW - Dimensionality reduction

KW - Kernel PCA

KW - Matrix factorization

KW - Neural networks

KW - Similarity preserving embeddings

KW - SVD

UR - http://www.scopus.com/inward/record.url?scp=85060692605&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85060692605&partnerID=8YFLogxK

U2 - 10.24425/bpas.2018.125929

DO - 10.24425/bpas.2018.125929

M3 - Article

AN - SCOPUS:85060692605

SP - 821

EP - 830

JO - Bulletin of the Polish Academy of Sciences: Technical Sciences

JF - Bulletin of the Polish Academy of Sciences: Technical Sciences

SN - 0239-7528

ER -