[PyTorch] Learn Deep Learning Easily and Quickly

🙆🏻‍♀ Learn easily and quickly from MLP, the basic framework of deep learning, to CNN and RNN.🙆🏻‍♂

I organized the lectures given at Inflearn and published a book titled 'Python Deep Learning PyTorch'.

Thank you for your interest : )

(Inflearn lectures have been updated as of 2020.10.06. We will continue to update the lectures.)

http://m.yes24.com/Goods/Detail/93376077?ozsrank=10

http://mbook.interpark.com/shop/product/detail?prdNo=339742291&is1=book&is2=product

Learn Deep Learning Quickly and Easily with PyTorch

🗒 Lecture Updates

- Added a PyTorch background lecture (about 1 hour) for those who are not familiar with PyTorch.

- Added a 1-hour lecture on paper review.

🗒 Course Introduction

Most of the artificial intelligence we talk about these days utilizes deep learning models. Neural networks, which are the basis of deep learning, are not new algorithms. They have existed for a long time, but due to the nature of learning, they have not been used much. Starting with this basic neural network, we will learn why deep learning has become popular, what its characteristics are, and further about convolutional neural networks and recurrent neural networks, which are called basic deep learning models.

🌈 Multi Layer Perceptron (MLP)

Learn about the perceptron, which is called the first artificial intelligence, its limitations, and MLP that overcomes it.

You can think of MLP as the basic structure of a neural network. I will explain the learning algorithm of MLP step by step.

We talk about feed forward and back propagation and their advantages and disadvantages.

🌈 Definition of Deep Learning

We will learn in depth about the definition of deep learning and how it differs from general neural networks.

We will cover activation functions, drop out, and batch normalization that can alleviate the gradient vanishing/overfitting problem, which is a shortcoming of NN.

We further discuss Auto-Encoders, which can not only classify but also learn about new features.

🌈 Convolutional Neural Network (CNN)

Looking at the history of deep learning, I think the most developed model is this CNN model. We will cover the CNN model that started with image classification and has made tremendous progress. We will talk about the characteristics of the learning algorithm and the differences from general NN.

We also discuss various architectures (Resnet, Densenet), initialization, optimizer techniques, and transfer learning to improve the performance of CNN.

🌈 Recurrent Neural Network (RNN)

We will cover RNN and LSTM, which are the basic models of text models (language models).

🌈 Various fields of deep learning

We talk about various fields where deep learning is used.

🌈 Generalization of Deep Learning

Generalization is a very big issue in AI in general, including deep learning and machine learning. It is a difficult problem that has not been solved yet. Therefore, various studies are being conducted to solve it. We will also introduce various studies on this generalization.

The existing generalization performance has been calculated as Training error - Estimated error. This is because it has been considered that the more similar the performance on the training data and the performance on the test data are, the better. However, a research result has been published that breaks this existing common sense, and the authors of the paper claim that deep learning is not learning, but 'memorizing.'

• Understanding deep learning requires rethinking generalization

It is a well-known fact that CNN is a very effective model for image classification. This is because it captures the features of the image well with Graphical Representation Learning. However, there is a research result that this CNN does not learn the shape of the image, but learns the texture. This paper was accepted orally by ICLR, an AI top conference, without a single line of formula and only with experimental results. It is said that it has very, very high academic value.

• ImageNET-Trained CNNS Are Biased Towards Texture

We introduce a paper on the differences between humans and deep learning in terms of generalization, and how deep learning can generalize similarly to humans.

• Generalization in humans and deep neural networks

The most basic way to improve the performance of image classification and generalization is Data Augmentation. We will introduce Cutout and Cutmix as an extension of this to provide a more effective method of augmentation. In addition, we will introduce the concept of Knowledge Distillation, which is the concept of a teacher imparting knowledge to a student.

• Cutout/Cutmix, Knowledge Distillation

Is it right to train the image of a 'young girl' as Girl? Or is it right to train it as Woman? Wouldn't it be more effective for the generalization of deep learning if it were trained as about half Girl and half Woman? This method is called Label Softening, and label softening is also performed using Knowledge Distillation. I will introduce a study on regularization using these methods.

• Regularizing Class-wise Predictions via Self-knowledge Distillation

🌈 Other papers worth reading reviews

We'll review some easy and good papers worth reading!

The paper review will be continuously updated!

If you add a little noise to a picture of an elephant, it will look like an elephant to the human eye, but deep learning will predict it as a Koala. As the field of deceiving deep learning through noise has emerged, many related studies have appeared, and I would like to introduce a study that shows that adding just one noise can cause deep learning to mismatch almost all images.

• Universal Perturbation

It is known that the model structure of deep learning should be simple for simple problems and complex for complex problems. In relation to this, there is a paper that studied the relationship between the model size of deep learning, the number of learning epochs, and overfitting.

• Deep Double Descent

In the figure below, you can see that all four pictures are 'dogs'. However, they all have different styles. If CNN learns dogs in one domain, it cannot match dogs in other domains. This is because it assumes that the training data and test data are in the same distribution. Generalization to multiple domains is called Domain Generalization, and we will introduce research on this.

• Domain Generalization With MixStyle

🙋🏻‍♂️ Expected questions related to the lecture

Q. Is a lot of mathematical knowledge required?
A. It is somewhat necessary for the MLP section, but you can still take the overall lecture without it.

Q. Do I need to know Python?
A. Yes, the class is conducted on the assumption that you have some knowledge of Python. However, we also cover PyTorch techniques for deep learning for about an hour.

Q. What kind of papers do you review?
A. I mainly review papers that are easy to read and highly contributing to deep learning. I will add a video review of the paper whenever I find a new good paper.

Introducing the practical knowledge sharer

🙆🏻‍♀ Justin

- Master's program in Industrial Engineering, Yonsei University

- DataScience/Deep Learning Research

- https://github.com/Justin-A