Skip to content

Natural Language Processing

Jhalak Patel edited this page Oct 10, 2017 · 6 revisions

Natural Language Processing Basics

Word Embeddings:

  1. Sebastian Ruders Blog on NLP

Word2Vec

  1. Overview and Insights to Word2Vec

  2. Skip Gram Word2Vec Tutorial Skip Gram Network Arch

  3. Contiguous Bag of Words Tutorial

  4. Kaggle NLP Word2Vec

  5. Word2Vec with Negative Sampling

CBOW and Skip Gram

Both CBOW and Skip Gram are used to learn an underlying representation of any word i.e. text to number (vector) conversion. AS learning word representation is unsupervised, we need some way to "create" labels to train the model. Skip-gram and CBOW are two ways of creating those "FAKE TASK" for the neural network. They are like output layers of the neural network, where we create labels for the input words and in turn, we learn the word embeddings.

Any bag-of-words model assumes that we can learn what a word means by looking at the words that tend to appear near it. The CBOW model trains each word against its context. It asks "given this set of context words, what missing word is likely to also appear at the same time?" Skip-gram trains each the context against the word. It asks "given this single word, what are the other words that are likely to appear near it at the same time?"

Math

  1. Euclidean Distance vs Cosine Distance