Tutorial 5: Aleksandr Drozd, Anna Rogers "Text Representation Learning and Compositional Semantics"

Back to list of tutorials

  • Day 1 (Nov.17), 15:00-16:30
  • Room 1102 (11th floor)
  • Speaker: Aleksandr Drozd (RIKEN), Anna Rogers (University of Massachusetts (Lowell))


Natural language processing is a fast-growing field, with a rapid evolution of approaches and models. In the last 6 years, we have come a long way from word embeddings to contextualized representations to pre-trained transformers, with numerous success stories for NLP system performance on question answering, text classification, machine translation and other tasks.
Despite the successes, we are still very far from reliable verbal reasoning, and one of the unresolved issues is semantic compositionality. It is not only a practical challenge, but also a theoretical one, as there is still no consensus on what a compositional representation of morphologically complex word, phrase or a sentence should be like.
This tutorial provides an introduction to both state-of-the-art NLP models and aspects of linguistic theory in which they are explicitly or implicitly grounded, particularly compositionality. We will overview of the latest proposals for representing words, sentences, and texts, as well as the discussion of interpretable components in meaning representations. In addition, we will discuss some of the problems with the current evaluation methodology and frequently used benchmarks.


Aleksandr Drozd is a research scientist at RIKEN Center for Computational Science, High Performance Artificial Intelligence Systems Research Team. His interests lie at the intersection of high performance computing and artificial intelligence, particularly areas like learning and evaluating text representations. Aleksandr holds a Ph.D. degree from the Department of Mathematical and Computing Sciences at the Tokyo Institute of Technology (Japan).

Anna Rogers is a post-doctoral associate in the Computer Science Department at Text Machine lab, University of Massachusetts (Lowell). She works at the intersection of linguistics, natural language processing, and machine learning. Anna holds a Ph.D. degree from the University of Tokyo (Japan). Her current projects span intrinsic evaluation of word embeddings, compositionality, temporal and analogical reasoning.