Deep Learning in NLP

Go to [[Week 2 - Introduction]] or back to the [[Main AI Page]] Part of the page on [[Artificial Intelligence/Introduction to AI/Week 2 - Introduction/Natural Language Processing]] For more details see [[Deep Learning]]

Deep learning refers to neural networks with many layers (the deep part) that takes as features as input and extracts higher-level features from this data. 

Much like in CNNs (Convolutional Neural Networks.)

Deep learning networks are able to learn a hierarchy of representations and different levels of abstractions of their input. Deep learning networks can use supervised learning or unsupervised learning and can be formed as hybrids of other approaches (such as incorporating a recurrent neural network with a deep learning network).

Long Short Term Memory cells used as neurons in NLP are more complicated than traditional perceptron neurons, as they have multiple gates and the ability to forget their current informational state.

An interesting application is where a CNN is used to analyse an image and then an LSTM is used to generate a textual representation of the image.