Python Why backpropagation dominates neural networks instead of interpolation

AI Thread Summary
The discussion centers on the differences between machine learning, specifically neural networks using backpropagation, and traditional interpolation/extrapolation methods. The confusion arises from the assumption that machine learning lacks a defined underlying equation, while it actually involves artificial neurons that implement specific functions with adjustable parameters (weights and biases). These parameters are optimized through backpropagation to minimize a target function. Unlike interpolation, which relies on known data points to estimate values, neural networks can model complex decision processes through multiple layers of abstraction. The intermediate layers in neural networks are not directly provided with data, allowing for a more nuanced interpretation of the input-output relationship, which is a key advantage over classical methods.
jonjacson
Messages
450
Reaction score
38
TL;DR Summary
I don't understand what are the advantages of backpropagation in a neural network versus using classical interpolation/extrapolation methods.
Hi guys,

I was learning machine learning and I found something a bit confusing.
When I studied physics I saw the method of least squares to find the best parameters for the given data, in this case we assume we know the equation and we just minimize the error. So if it is a straight line model we compute the best slope and constant.
With machine learning we don't know the underlying equation or model so we use a general method to "fit" as best as we can the real data.
But, isn't that what we do with interpolation/extrapolation?
What is it so special about a neural network and the backpropagation method that we can't achieve with interpolation?
 
Technology news on Phys.org
jonjacson said:
TL;DR Summary: I don't understand what are the advantages of backpropagation in a neural network versus using classical interpolation/extrapolation methods.

With machine learning we don't know the underlying equation or model
I don't think this is correct. Each artificial neuron implements a definite function with parameters (weights and bias), which need to be determined to minimize a definite target function, which is built by composing the functions of the neurons.
 
Hill said:
I don't think this is correct. Each artificial neuron implements a definite function with parameters (weights and bias), which need to be determined to minimize a definite target function, which is built by composing the functions of the neurons.
The weights are used to compute how much of an output from one neuron enters the next ones, but that is not a function.
 
jonjacson said:
The weights are used to compute how much of an output from one neuron enters the next ones, but that is not a function.
Here is one way to define the function (sorry, I'd need to dig to find the source of this paper):

1702209231417.png

.......
1702209291789.png

1702209321470.png

.......
1702209388276.png

1702209430175.png
 
jonjacson said:
With machine learning we don't know the underlying equation or model so we use a general method to "fit" as best as we can the real data.
But, isn't that what we do with interpolation/extrapolation?
What is it so special about a neural network and the backpropagation method that we can't achieve with interpolation?
I am not an expert, but here is my understanding:
The backpropagation of neural networks is to develop weights for intermediate layers that can help to model a sophisticated decision process. Those intermediate layers are not given data like interpolation is. They are free to interpretation. Sometimes they are obscure and other times we can imagine a meaning for them. The only hard training data are the inputs and outputs at the first and last layer.
 
Dear Peeps I have posted a few questions about programing on this sectio of the PF forum. I want to ask you veterans how you folks learn program in assembly and about computer architecture for the x86 family. In addition to finish learning C, I am also reading the book From bits to Gates to C and Beyond. In the book, it uses the mini LC3 assembly language. I also have books on assembly programming and computer architecture. The few famous ones i have are Computer Organization and...
I have a quick questions. I am going through a book on C programming on my own. Afterwards, I plan to go through something call data structures and algorithms on my own also in C. I also need to learn C++, Matlab and for personal interest Haskell. For the two topic of data structures and algorithms, I understand there are standard ones across all programming languages. After learning it through C, what would be the biggest issue when trying to implement the same data...
Back
Top