StanfordのCS231nという授業の教材を使って、機械学習を学んだ。
自分のメモのまとめ。
(写経に近いので注意)
Module 1: Neural Networks
- Image Classification: Data-driven Approach, k-Nearest Neighbor, train/val/test splits
- L1/L2 distances, hyperparameter search, cross-validation
- Linear classification: Support Vector Machine, Softmax
- parameteric approach, bias trick, hinge loss, cross-entropy loss, L2 regularization, web demo
- Optimization: Stochastic Gradient Descent
- optimization landscapes, local search, learning rate, analytic/numerical gradient
- Backpropagation, Intuitions
- chain rule interpretation, real-valued circuits, patterns in gradient flow
- Neural Networks Part 1: Setting up the Architecture
- model of a biological neuron, activation functions, neural net architecture, representational power
- Neural Networks Part 2: Setting up the Data and the Loss
- preprocessing, weight initialization, batch normalization, regularization (L2/dropout), loss functions
- Neural Networks Part 3: Learning and Evaluation
- gradient checks, sanity checks, babysitting the learning process, momentum (+nesterov), second-order methods, Adagrad/RMSprop, hyperparameter optimization, model ensembles
- Putting it together: Minimal Neural Network Case Study
- minimal 2D toy data example
Module 2: Convolutional Neural Networks(更新予定)
- Convolutional Neural Networks: Architectures, Convolution / Pooling Layers
- layers, spatial arrangement, layer patterns, layer sizing patterns, AlexNet/ZFNet/VGGNet case studies, computational considerations
- Understanding and Visualizing Convolutional Neural Networks
- tSNE embeddings, deconvnets, data gradients, fooling ConvNets, human comparisons
- Transfer Learning and Fine-tuning Convolutional Neural Networks