Intuition behind LSTM , GRU and Attention

Image for post
Image for post
Photo by Igor Vymorkov on Unsplash

LSTM

Image for post
Image for post
Cell State

Step-by-Step LSTM Walk Through

Forget Gate :

Image for post
Image for post
Forget Gate

Input Gate :

Image for post
Image for post
Input Gate

Output Gate :

Image for post
Image for post
Output Gate

GRU

Encoder-Decoder :

Image for post
Image for post
Encoder-Decoder Architecture

Attention (Intuition) :

Image for post
Image for post
Attention Mechanism

Attention (Overview) :

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store