- #1
joshthekid
- 46
- 1
So I am currently learning some regression techniques for my research and have been reading a text that describes linear regression in terms of basis functions. I got linear basis functions down and no exactly how to get there because I saw this a lot in my undergrad basically, in matrix notation
y=wTx
you then define your loss function as
1/n Σn(wi*xi-yi)2
then you take the partial derivatives with respect to w set it equal to zero and solve.
So now I want to use a non-linear basis functions, let's say I want to use m gaussians basis functions, φi, the procedure is the same but I am not sure exactly on the construction of the model. Let's say I have L features is the model equation of the form
yn=ΣmΣLwiφi(xj)
in other words I have created a linear combination of M new features, φ(x), which are constructed with all L of the previous features for each data point n:
yn=w0+w1(φ1(x1)+φ1(x2)...+...φ1(xL) ...+...wm(φ1(x1)+φ2(x2)...+...φm(xL))
where xi are features / variables for my model and not data values? I hope this makes sense. Thanks in advance.
y=wTx
you then define your loss function as
1/n Σn(wi*xi-yi)2
then you take the partial derivatives with respect to w set it equal to zero and solve.
So now I want to use a non-linear basis functions, let's say I want to use m gaussians basis functions, φi, the procedure is the same but I am not sure exactly on the construction of the model. Let's say I have L features is the model equation of the form
yn=ΣmΣLwiφi(xj)
in other words I have created a linear combination of M new features, φ(x), which are constructed with all L of the previous features for each data point n:
yn=w0+w1(φ1(x1)+φ1(x2)...+...φ1(xL) ...+...wm(φ1(x1)+φ2(x2)...+...φm(xL))
where xi are features / variables for my model and not data values? I hope this makes sense. Thanks in advance.