Are you still struggling with various technical pronouns on the Internet? Do you think they are very similar? To understand the difference between these terms, you need to understand some terminology about machine learning, such as gradient descent, to help you understand.
Gradient descentThis is an iterative optimization algorithm used in machine learning to find the best result (the minimum of the curve).
The meaning of the gradient is the slope of the slope or slope.
The meaning of the decline is the decline in the cost function.
The algorithm is iterative, meaning that the algorithm needs to be used multiple times to get the results to get the optimization results. The iterative nature of the gradient descent allows the graphical representation of the under-fitting to evolve to obtain the best fit to the data.
There is a parameter called the learning rate in the gradient descent. As shown on the left side of the figure, the learning rate is much higher at the beginning, so the step size is larger. As the point falls, the learning rate becomes smaller and smaller, and the descending step size becomes smaller. At the same time, the cost function is also decreasing, or the cost is decreasing, sometimes called the loss function or loss, and both are the same. (Loss/cost reduction is a good thing)
Only when the data is very large (in machine learning, almost always), we need to use epochs, batch size, and iterate these terms, in which case it is impossible to enter data into the computer at once. Therefore, in order to solve this problem, we need to divide the data into small pieces, pass them to the computer one by one, update the weight of the neural network at the end of each step, and fit the given data.
EPOCHSWhen a complete data set passes through the neural network once and returns once, the process is called an epoch.
However, when an epoch is too large for a computer, it needs to be divided into smaller pieces.
Why use more than one epoch?
I know it sounds strange at first, it's not enough to pass a complete data set in a neural network once, and we need to pass the complete data set multiple times in the same neural network. But keep in mind that we are using a limited data set, and we use an iterative process, gradient descent, to optimize the learning process and diagrams. So just updating the weight once or using an epoch is not enough.
As the number of epoch increases, the number of updates to the weights in the neural network also increases, and the curve becomes over-fitting from under-fitting.
So, how many epochs are appropriate?
Unfortunately, there is no correct answer to this question. For different data sets, the answer is different. But the diversity of data affects the number of suitable epochs. For example, there are only black cat data sets, as well as data sets for cats of various colors.
BATCH SIZEThe total number of samples in a batch. Remember: batch size and number of batches are different.
What is BATCH?
When you can't pass data through the neural network at once, you need to divide the data set into several batches.
Just like this article is divided into several parts, such as introduction, gradient descent, Epoch, Batch size and iteration, to make the article easier to read and understand.
IterationTo understand iterations, you only need to know the multiplication table or a calculator. Iteration is the number of times a batch needs to complete an epoch. Remember: in an epoch, the number of batches and the number of iterations are equal.
For example, for a data set with 2000 training samples. Divide 2000 samples into batches of size 500, then 4 iteraTIons are required to complete an epoch.
China Laptop Stand Adjustable,Adjustable Macbook Stand manufacturer, choose the high quality Ergonomic Adjustable Laptop Stand,Laptop Stand Aluminum Adjustable, etc.
Shenzhen Chengrong Technology Co.ltd is a high-quality enterprise specializing in metal stamping and CNC production for 12 years. The company mainly aims at the R&D, production and sales of Notebook Laptop Stands and Mobile Phone Stands. From the mold design and processing to machining and product surface oxidation, spraying treatment etc ,integration can fully meet the various processing needs of customers. Have a complete and scientific quality management system, strength and product quality are recognized and trusted by the industry, to meet changing economic and social needs .
Portable Laptop Stand Adjustable,Best Laptop Stand Adjustable,Aluminum Laptop Stand Adjustable
Shenzhen ChengRong Technology Co.,Ltd. , https://www.laptopstandsupplier.com