改善深层神经网络：超参数调试、正则化以及优化笔记

前言

课程链接：改善深层神经网络：超参数调试、正则化以及优化
公式绘制 AxMath
部分内容存在引用
欢迎指正文中的各种错误

Train/Dev/Test Sets

Bias and Varience

截取引用自Understanding the Bias-Variance Tradeoff
Error due to Bias: The error due to bias is taken as the difference between the expected (or average) prediction of our model and the correct value which we are trying to predict. Of course you only have one model so talking about expected or average prediction values might seem a little strange. However, imagine you could repeat the whole model building process more than once: each time you gather new data and run a new analysis creating a new model. Due to randomness in the underlying data sets, the resulting models will have a range of predictions. Bias measures how far off in general these models' predictions are from the correct value.
Error due to Variance: The error due to variance is taken as the variability of a model prediction for a given data point. Again, imagine you can repeat the entire model building process multiple times. The variance is how much the predictions for a given point vary between different realizations of the model.

Basic Recipe for Machine Learning

Regularization

How does Regularization Prevent from Overfitting

Inverted Dropout

Normalize Inputs

Why Normalize Inputs

截取自课程

Vanishing/Exploding gradients

Gradient Checking

Mini Batch

Exponentially weighted averages

Gradient Descent with Momentum

RMSprop

Adam Optimization Algorithm

Implementing Batch Norm

Adding Batch Norm to a Network

Softmax Layer