STOCHASTIC Gradient Descent (in 3 minutes)

Visual and intuitive Overview of stochastic gradient descent in 3 minutes. ------------------- References: - The third explanation is from here: https://arxiv.org/abs/1802.06175 - Other references mentioned in the video: https://arxiv.org/abs/1509.01240, https://proceedings.mlr.press/v40/Ge15.pdf - AI plays hide and seek: https://openai.com/blog/emergent-tool-use/ - AI plays Dota 2: https://openai.com/five/ - InterFaceGAN: https://www.youtube.com/watch?v=uoftpl3Bj6w - Boyd and Vandenberghe's book on Convex Optimization (Sections 9.2 and 9.3): https://web.stanford.edu/class/ee364a/lectures/unconstrained.pdf https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf ------------------ Timestamps: - 0:00 Intro - 0:27 Definition - 1:00 Stochastic Gradient Descent is too good - 1:37 First Explanation - 1:52 Second Explanation - 2:13 Third Explanation - 2:40 Outro ------------------- Music: www.bensound.com This video would not have been possible without the help of Gökçe Dayanıklı.