A Understanding the Cost Function in Machine Learning: A Practical Guide

[emmasaunders12]

Could someone please help me work through the differentiation in a paper (not homework), I am having trouble finding out how they came up with their cost function.

The loss function is L=wE, where E=(G-Gest)^2 and G=F'F

The derivative of the loss function wrt F is proportional to F'(G-Gest)

Can't seem to figure it out.

Thanks

Emma

 

[fresh_42]

I have some trouble to understand you:

Do all functions depend on, say time $t$, which the primes refer to? And why isn't $G-G=0$? I first thought it could be the strange notation of a function, but then you defined a single $G$. And last, could it be $L \propto F(G-G)'$?

 

[emmasaunders12]

I have some trouble to understand you:

Do all functions depend on, say time $t$, which the primes refer to? And why isn't $G-G=0$? I first thought it could be the strange notation of a function, but then you defined a single $G$. And last, could it be $L \propto F(G-G)'$?

Thanks for the response, its the loss function of a neural network, so I've corrected to G and Gest, primes refer to transpose

 

[PeroK]

Thanks for the response, its the loss function of a neural network, so I've corrected to G and Gest, primes refer to transpose

Perhaps someone else can help, but without a lot more context I have no idea what mathematically we are dealing with here.

 

[emmasaunders12]

Perhaps someone else can help, but without a lot more context I have no idea what mathematically we are dealing with here.

The specific problem is described on page 4 here https://arxiv.org/pdf/1505.07376v3.pdf