Angle between two vectors with many dimensions

[sampsont]

The angle between two ℝ² or ℝ³ vectors makes sense.

I have a vector like [0 0.707 1 0.707 0 -.707 -1 -.707]^T
(Actually my vector is A/D conversions of a sine wave from the wall power outlet. The sample rate is 4800 so there are 80 samples, separated by 1/Fs for a 60Hz sine wave.

If I have two of these vectors each with 80 rows, does the angle between these vectors have any physical significance?

Should I think of the vector to mean there are 80 dimensions, one dimension for each of the 80 A/D conversions?

(My application is trying to estimate the phase/frequency of the sine wave.)

Thanks!

 

[Hesch]

(My application is trying to estimate the phase/frequency of the sine wave.)

You cannot determine the phase of one sine wave, but the phase between two sine waves. ( You must have a sort of reference. In the same way you cannot speak of a voltage in a conductor, you must have a reference, e.g. ground ).

Speaking of the phase as for an AC power, the phase between current and voltage is normally meant. To determine this phase, you must sample the voltage and the current simultaneously ( two A/D-channels ).

To be accurate you then Fourier transform both signals. The phase will then become the difference in angles for the 1. harmonic of the two signals.

Each of the Fourier transforms will become a series of complex coefficients for the harmonics in the signal, in polar format expressed as A/ φ
The phase between voltage and current = φ_voltage - φ_current as for the 1. harmonic coeffcients.

 

[sampsont]

Thanks for the response! Much appreciated. Actually, I already have the phase and frequency measurements implemented using the DFT, and more specifically, the Goertzel algorithm which is an IIR filter that essentially achieves a single bin DFT.

We have an A/D for each input channel so all signals are sampled and converted at the same time. We also have our sampler locked to the GPS IRIG signal so we know which sample corresponds to the one second roll-over from the GPS. All phase measurements are referenced to a cosine that starts at the one second roll-over. These phase measurements are called Synchrophasors.

There is a specification for the dynamic behavior of the system to various inputs, such as harmonics, interharmonics, noise, frequency steps/sweeps etc. My implementation using frequency domain methods does not pass the spec for two main reasons:

1. Spectral Leakage - The input frequency can vary from 55Hz to 65Hz. When the input frequency moves away from 60Hz, the phase estimate loses accuracy because of the sinc function side-lobes. I added code to either throw away or add in samples in the Goertzel dynamically to make one complete period fit in the measurement window. I call it LineLock. This helps a lot but the error is still too large, for example between adding n samples or n+1 samples. For example, if the frequency increases above 60HZ, there is a point where it is good to throw away one sample because the line cycle is finishing sooner. As the frequency continues to increase, there comes a time where is is good to throw away two samples. The frequency between throwing away one sample and two samples has some spectral leakage. By the way, windowing does not solve the problem because it degrades the phase measurement too much.
2. Noise - The Goertzel is excellent to filter out harmonics and out of band noise but any noise at the fundamental causes noise in the phase measurement.

The specification is so tight it is almost impossible to meet. The frequency measurement accuracy must be within 0.005Hz over the 55Hz to 65Hz range. Requirements for frequency step inputs drastically limit the amount of filtering that can be used.

My hope is to add in some time-domain statistical methods to augment or maybe replace the estimation using Goertzel. I'm trying to up my math skills to the point of seeing if the Kalman filter or some such can help. This leads me to my desire to understand the significance of the angle between two vectors. For example the angle between the previous measurement window, (previous 80 samples), and the current measurement window, (most recent 80 samples). Can this help me to deduce the phase change? If the input frequency is 60Hz, the angle seems like it should be zero because the two vectors would be the same. If the frequency deviates from 60Hz, the two vectors would begin to change and the angle would change. I'll try this with some data but I would like to have the intuition of what the angle really is.

Thanks!

 

[sampsont]

The norm of a vector, in my case a sine wave is the RMS. There must be an integer number of cycles in the vector and you have to divide the norm by the number of samples. At least this much is intuitive. I'm still wondering what to make of the angle.

 

[Mark44]

The coordinate-free definition of the dot product can be used to find the angle between two vectors in an arbitrary vector space, as long as it is a normed vector space.

If $\vec{u}$ and $\vec{v}$ are the vectors in question, then $\vec{u} \cdot \vec{v} = \parallel \vec{u} \parallel \parallel \vec{v} \parallel \cos(\theta)$, where $\theta$ is the angle between the two vectors. Equivalently, we have $\cos(\theta) = \frac{\vec{u} \cdot \vec{v}}{ \parallel u \parallel \parallel v \parallel}$, from which we can get $\theta$.

 

[sampsont]

Thanks, but what does θ really mean? My two vectors are each a cycle of the 60Hz from the power outlet. Each vector has 80 samples.

What does the angle between my two vectors tell me? What can I use it for? Can it give me any indication of the relative phase of a sine waves?

 

[Mark44]

Thanks, but what does θ really mean? My two vectors are each a cycle of the 60Hz from the power outlet. Each vector has 80 samples.

Let's take $u = <0, \frac{\sqrt 2}2, 1, \frac{\sqrt 2}2, 0>$ and $v = <-1, -\frac{\sqrt 2} 2, 0, \frac{\sqrt 2}2, 1>$
(My vector u is a shortened version of the one you gave, and uses $\frac{\sqrt 2}2$ instead of the .707 you have.)

Using the formula I gave previously, $\cos(\theta) = \frac{\vec{u} \cdot \vec{v}}{|\vec{u}| |\vec{v}|} = \frac 0 {\sqrt 2 \sqrt 3} = 0$
This implies that $\theta = \pi/2$ or 90°. I cooked up the second vector to be 90° out of phase with the first.

My example is somewhat contrived, but my intent is to show what the angle between two higher-dimension vectors could represent.

What does the angle between my two vectors tell me? What can I use it for? Can it give me any indication of the relative phase of a sine waves?

See above.

 

[Hesch]

What does the angle between my two vectors tell me? What can I use it for? Can it give me any indication of the relative phase of a sine waves?

Say you do a Fast Fourier Transform (FFT) at some signal, some complex coefficient to some harmonic could be: 3 + j4.
This means that this harmonic has a phase referred to the Fourier interval = θ.

Tan(θ) = 4/3 → θ ≈ 53°.

The absolute value of the coefficient = 5, so this coefficient could be written: 5*cos(53°) + 5*sin(53°).

I think the latter way of writing it is familiar.

An outlet will have a phase between voltage and current ≠ 0, if the load is capacitive or inductive.

 

[sampsont]

Ahhh. The DFT coefficient of the fundamental frequency gives the phase between an ideal cosine with zero phase, and your signal. The linear algebra method does the same thing except the phase reference is your first vector instead of the ideal cosine. Is this correct?

 

[Hesch]

Yes, but:

The DFT coefficients of all harmonic frequecies gives the differences in phases, one signal to another.
Think of a Bode plot where the phase varies as a function of frequency. Likewise the phases of different harmonics in a DFT ( or FFT ) depend on the order of the harmonic.

 

[The Bill]

Thanks, but what does θ really mean?

In general, just as there are points and lines in 3D space, there are planes in spaces of dimension 2 or higher. Of course, the dimension needs to be higher than 2 to have multiple distinct planes through a point.

Any two linearly independent vectors define a plane, and the angle between them is the angle between them in that plane.

 

[Stephen Tashi]

I have a vector like [0 0.707 1 0.707 0 -.707 -1 -.707]^T
(Actually my vector is A/D conversions of a sine wave from the wall power outlet. The sample rate is 4800 so there are 80 samples, separated by 1/Fs for a 60Hz sine wave.

(My application is trying to estimate the phase/frequency of the sine wave.)

Then should we discuss how to do that? Searching for a physical interpretation of the inner product of two of your data vectors might be irrelevant to that purpose. If it is relevant then the significance will turn up on the way to solving the problem.

 

[sampsont]

The DFT is defined as:
Y(m) = ∑ x(n) e^-jωnm/T
I only care about m = 1 right now.
Y(1) = ∑ x(n) e^-jωn/T
Y(1) = ∑ [x(n) cos(ωn/T) - x(n) jsin(ωn/T)]

Dot product:
U ⋅ V = ∑ U[n] V[n]

It looks to me like the dot product is the first, (real) part of the DFT, where one of the vectors is an ideal cosine and the other is the signal, x(n).

If you think in dimension 2, this might be the adjacent length of a right triangle.

The product of the norm of U and V must be the length of the hypotenuse of a right triangle so that:

cos θ = U ⋅ V/||U|| ||V||

I'm not sure why the hypotenuse is the product of the length of U and V.

I'm about 1/3 of the way through Kuldeep Singh's Linear Algebra Step by Step book. I'm amazed at how well this book is written. It's like climbing a gentle ramp instead of trying to scale walls. As I come across anything that seems to pertain to my problem of estimating power line Synchrophasors I usually stop and try to apply new concepts with my sampled data.

Question: Does the LA phase measurement suffer from spectral leakage like the DFT? In other words, for the DFT, if your vectors are not an exact integer number of cycles of the input signal, the phase estimation loses accuracy.

Stephen Tashi: I'd love to discuss how to estimate the phase/frequency of the power line. The DFT/Goertzel method is really good but not good enough to meet the spec. I think the main problem is that noise in the same band as the fundamental 60Hz is causing a little noise in my phase measurement.

It seems like my DFT method needs to be augmented a little somehow with another approach. Also it seems like I should take advantage of the fact that the input should be a fairly good sine wave. The expected state value should be a sine wave. That sounds like maybe the Kalman filter. I'll probably finish my LA book before I try to figure out the Kalman filter.

Thanks for all the help and suggestions!