I Auto Regressive Moving Average Model (ARMA) Ljung-Box Test

mertcan

Hi, according to ARMA model it is said that in order to check out white noise terms, Ljung-Box is applied involving sum of squared autocorrelations of errors with relevant lags. In short, sum of them is chi-square distribution but n-p-q degree of freedom when we have ARMA(p,0,q) model. My question : Is there a mathematical proof of why we subtract p+q from n?

By the way I know the proof of why we do the similar subtraction like above in multilinear regression. And mathematical proof of it is displayed in (https://stats.stackexchange.com/questions/24921/distribution-of-sum-of-squares-error-for-linear-regression/400261?noredirect=1#comment749409_400261). But even though there is a similarity I can not derive for ARMA model.

• FactChecker
Related Set Theory, Logic, Probability, Statistics News on Phys.org

BvU

Homework Helper
In short, sum of them is chi-square distribution but n-p-q degree of freedom when we have ARMA(p,0,q) model. My question : Is there a mathematical proof of why we subtract p+q from n?
No reponse so far, so I'll give it a try:
Degrees of freedom is number of points minus number of parameters in the model that are derived from the data. ARMA(p,0,q) has p+q parameters derived from the data. (I'd say p+q+1 because of the average, but apparently the 0 in there means the 0 is hypothesized).

Not really a proof, more an explanation....

• mertcan and FactChecker

mertcan

No reponse so far, so I'll give it a try:
Degrees of freedom is number of points minus number of parameters in the model that are derived from the data. ARMA(p,0,q) has p+q parameters derived from the data. (I'd say p+q+1 because of the average, but apparently the 0 in there means the 0 is hypothesized).

Not really a proof, more an explanation....
Thanks for return, according to definition : In statistics, the number of degrees of freedom is the number of values in the final calculation of a statistics that are free to vary. Could you show me maybe using some mathematical demonstration how p+q+1 are not free to vary in ARMA? Could you help me imagine the case?

BvU

Homework Helper
Degrees of freedom is number of points minus number of parameters in the model that are derived from the data. ARMA(p,0,q) has p+q parameters derived from the data.
I don't feel like a genuine expert on ARMA (had to look it up for this thread). But I can try to answer
Could you help me imagine the case?
Simple rule that works for me: take a simple example !
e.g. ARMA(1,0) = AR(1) for two data points has no degrees of freedom: $c$ and $\phi_1$ are fully determined.

In general math: The $n+1$ coefficients for a polynomial of order $n$ through $n+1$ points can be calculated $\ \Rightarrow\$ no degrees of freedom

See also here: if you need to establish the quality of the model, the noise level comes in as well.

• FactChecker

mertcan

I don't feel like a genuine expert on ARMA (had to look it up for this thread). But I can try to answer

Simple rule that works for me: take a simple example !
e.g. ARMA(1,0) = AR(1) for two data points has no degrees of freedom: cc and ϕ1ϕ1 are fully determined.

In general math: The n+1n+1 coefficients for a polynomial of order nn through n+1n+1 points can be calculated ⇒ ⇒ no degrees of freedom

See also here: if you need to establish the quality of the model, the noise level comes in as well.
Thanks for return: What do you say about the following:

Let's say we have ARMA(4,0,0) process and

$$yt=ϕ1∗yt−1+ϕ2∗yt−2+ϕ3∗yt−3+ϕ4∗yt−4+error_t$$​

As you can see EXPECTATION OF $$y_1=\phi_1*y_{0}+\phi_2*y_{-1}+\phi_3*y_{-2}+\phi_4*y_{-3}$$
EXPECTATION OF $$y_2=\phi_1*y_{1}+\phi_2*y_{0}+\phi_3*y_{-1}+\phi_4*y_{-2}$$
EXPECTATION OF $$y_3=\phi_1*y_{2}+\phi_2*y_{1}+\phi_3*y_{0}+\phi_4*y_{-1}$$
EXPECTATION OF $$y_4=\phi_1*y_{3}+\phi_2*y_{2}+\phi_3*y_{1}+\phi_4*y_{0}$$

By the way we do not know $$y_{0},y_{-1},y_{-2},y_{-3}$$ so may be we can write
$$y_1=error_1$$
$$y_2=\phi_1*y_{1}+error_2$$
$$y_3=\phi_1*y_{2}+\phi_2*y_{1}+error_3$$
$$y_4=\phi_1*y_{3}+\phi_2*y_{2}+\phi_3*y_{1}+error_4$$
$$error_1=\phi_1*y_{0}+\phi_2*y_{-1}+\phi_3*y_{-2}+\phi_4*y_{-3}$$
$$error_2=\phi_2*y_{0}+\phi_3*y_{-1}+\phi_4*y_{-2}$$
$$error_3=\phi_3*y_{0}+\phi_4*y_{-1}$$
$$error_4=\phi_4*y_{0}$$
In short we have 8 equations and 8 unknowns as $$error_1, error_2, error_3, error_4, y_{0}, y_{-1}, y_{-2}, y_{-3}$$
So error terms 1 to 4 can not be random because we solved them in linear system they are not free as in $$y_5,y_6.......$$ so we have n-4 degree of freedom.What do you say?

• FactChecker

FactChecker

Gold Member
2018 Award
In short we have 8 equations and 8 unknowns as $$error_1, error_2, error_3, error_4, y_{0}, y_{-1}, y_{-2}, y_{-3}$$
So error terms 1 to 4 can not be random because we solved them in linear system they are not free as in $$y_5,y_6.......$$ so we have n-4 degree of freedom.What do you say?
That looks good to me. I think that your example is a higher-dimensional example of the very simple one that @BvU gave. I like the simple example to make the point and I like your higher-dimensional example to show the generalization.

• mertcan

mertcan

@FactChecker and @BvU could you help me for the case ARMA (0,0,q)?
Actually I converted MA to infinite AR process but can not get proper result. I think derivation is different than AR processes?

mertcan

I think I carved out it but I also wonder your return for cross check...?

mertcan

That looks good to me. I think that your example is a higher-dimensional example of the very simple one that @BvU gave. I like the simple example to make the point and I like your higher-dimensional example to show the generalization.
I don't feel like a genuine expert on ARMA (had to look it up for this thread). But I can try to answer

Simple rule that works for me: take a simple example !
e.g. ARMA(1,0) = AR(1) for two data points has no degrees of freedom: cc and ϕ1ϕ1 are fully determined.

In general math: The n+1n+1 coefficients for a polynomial of order nn through n+1n+1 points can be calculated ⇒ ⇒ no degrees of freedom

See also here: if you need to establish the quality of the model, the noise level comes in as well.
I also tired to set some equations for ARMA(3,0,2) model. As you know we lose p+q=5 degree of freedom which means we set constraints 5 error terms. So could you check my proof if it?

$$y1=error_1$$
$$y2=ϕ1∗y1+error_2$$
$$y3=ϕ1∗y2+ϕ2∗y1+θ1∗error2+θ2∗error_1+error_3$$
$$error1=ϕ1∗y0+ϕ2∗y−1+ϕ3∗y−2+θ1∗error_0+θ2∗error_−1$$
$$error2=ϕ2∗y0+ϕ3∗y−1+θ2∗error_0$$​

$$error_3=\phi_3*y_0$$

$$y4=ϕ1∗y3+ϕ2∗y2+ϕ3∗y1+θ1∗error_3+θ2∗error_2+error_4$$
$$y5=ϕ1∗y4+ϕ2∗y3+ϕ3∗y2+θ1∗error_4+θ2∗error_3+error_5$$​

if we set error_4 and error_5 are zero then we have 8 equations and 8 unknowns

$$error_1,error_2,error_3,error_0,error_−1,error_−2,y0,y−1,y−2error_1,error_2,error_3,error_0,error_−1,error_−2,y0,y−1,y−2$$​

. In short

$$error_1,error_2,error_3,error_0,error_−1,error_4,error_5error_1,error_2,error_3,error_0,error_−1,error_4,error_5$$​

has been set without considering their randomnesses. What do you think about that?

"Auto Regressive Moving Average Model (ARMA) Ljung-Box Test"

Physics Forums Values

We Value Quality
• Topics based on mainstream science
• Proper English grammar and spelling
We Value Civility
• Positive and compassionate attitudes
• Patience while debating
We Value Productivity
• Disciplined to remain on-topic
• Recognition of own weaknesses
• Solo and co-op problem solving