What exactly does word2vec learn?

<!-- twitter --> <p>What exactly does <code class="language-plaintext highlighter-rouge">word2vec</code> learn, and how? Answering this question amounts to understanding representation learning in a minimal yet interesting language modeling task. Despite the fact that <code class="language-plaintext highlighter-rouge">word2vec</code> is a well-known precursor to modern language models, for many years, researchers lacked a quantitative and predictive theory describing its learning pro