CitiusNLP at SemEval-2020 Task 3: Comparing Two Approaches for Word Vector Contextualization

This article describes some unsupervised strategies submitted to SemEval 2020 Task 3, a task which consists of considering the effect of context to compute word similarity. More precisely, given two words in context, the system must predict the degree of similarity of those words considering the context in which they occur, and the system score is compared against human prediction. We compare one approach based on pre-trained BERT models with other strategy relying on static word embeddings and syntactic dependencies. The BERT-based method clearly outperformed the dependency-based strategy.

keywords: Word Similarity, BERT, Transformers, Contextual Word Embeddings, Syntactic Dependencies