In this article, we compare two different strategies to contextualize the meaning of words in a sentence: both
distributional models that make use of syntax-based methods following the Principle of Compositionality and
Transformer technology such as BERT-like models. As the former methods require controlled syntactic struc-
tures, the two approaches are compared against datasets with syntactically fixed sentences, namely subject-
predicate and subject-predicate-object expressions. The results show that syntax-based compositional ap-
proaches working with syntactic dependencies are competitive with neural-based Transformer models, and
could have a greater potential when trained and developed using the same resources.
Keywords: Compositional Distributional Models, Contextualized Word Embeddings, Transformers, Compositionality, Dependency-based Parsing.