Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Vector with minimal angle to a set of vectors

  1. Oct 5, 2011 #1
    Let [itex]\mathbb X[/itex] be a set of mean-shifted vectors [itex]\mathbf X_1, \mathbf X_2, \ldots, \mathbf X_k[/itex] in [itex]\mathbb R^n[/itex]. By mean-shifted, I mean that [itex]\text{Mean}(\mathbf X_i) = 0[/itex] [itex]\forall i \in {0, \ldots, k}[/itex]. I want to find a vector [itex]\mathbf T \in \mathbb R^n[/itex] that maximizes the function

    [itex]f(\mathbf T) = \sum \limits_{\mathbf X \in \mathbb X} \big ( cos(\theta_{\mathbf X, \mathbf T}) \big )^2 = \sum \limits_{\mathbf X \in \mathbb X} \big ( \frac{\mathbf X \cdot \mathbf T}{||\mathbf X|| \cdot ||\mathbf T||} \big )^2[/itex]

    (Edit: maximizing this [itex]f[/itex] does not actually get what I want. Instead, I want to maximize the geometric mean of the correlations - see my followup posts).

    In other words, I want to find a vector [itex]\mathbf T[/itex] that has a small angle [itex]\theta[/itex] to each [itex]\mathbf X \in \mathbb X[/itex]. (Actually there must be infinitely many solutions, since any linear transformation of [itex]\mathbf T[/itex] is also a solution).

    My real motivation is statistical: this is equivalent to finding a vector that has a high Pearson correlation coefficient with all the vectors in [itex]\mathbb X[/itex].

    I can use numerical algorithms to optimize [itex]f[/itex], but I would prefer to find a closed-form solution. This problem is part of a larger algorithm that I am trying to speed up, and it spends most of its time trying to find [itex]T[/itex]. Is a closed-form solution possible? Or is this problem similar to an existing problem, which I can adapt?

    If [itex]k=2[/itex], the solution should just be to rotate [itex]\mathbf X_1[/itex] halfway to [itex]\mathbf X_2[/itex], using some multidimensional rotation as discussed in http://forums.xkcd.com/viewtopic.php?f=17&t=29603". Perhaps this rotation concept could be extended to find the "rotational average" of [itex]\mathbb X[/itex], if that makes sense?

    Any help towards finding a closed-form solution to this problem would be appreciated.
    Last edited by a moderator: Apr 26, 2017
  2. jcsd
  3. Oct 5, 2011 #2
    Actually, I just realized that [itex]f[/itex] in my previous post is not exactly what I want to maximize. I want [itex]T[/itex] to be equally and maximally close to all vectors in [itex]\mathbb X[/itex], so maximizing the minimum correlation with [itex]T[/itex] would be better:

    [itex]f = \displaystyle{\mathop{\mbox{min}}_{\mathbf X \in \mathbb X}} \big ( \frac{\mathbf X \cdot \
    \mathbf T}{||\mathbf X|| \cdot ||\mathbf T||} \big )^2 [/itex]

    However, this makes [itex]f[/itex] non-differentiable, so perhaps using the http://mathoverflow.net/questions/35191/a-differentiable-approximation-to-the-minimum-function" [Broken] would work:

    [itex]f = \big ( \frac{1}{|\mathbb X|}\sum\limits_{\mathbf X \in \mathbb X} \frac{\mathbf X \cdot \
    \mathbf T}{||\mathbf X|| \cdot ||\mathbf T||} \big )^{\frac{1}{k}} [/itex]

    for [itex]k \rightarrow - \infty[/itex].
    Last edited by a moderator: May 5, 2017
  4. Oct 5, 2011 #3
    Argh, actually that's not what I want either. If one vector in [itex]\mathbb X[/itex] is very different from the others, it will pull [itex]\mathbf T[/itex] towards it too much. Instead, I think I want to maximize the geometric mean, or the limit as [itex]k \rightarrow 0[/itex] of the generalized mean.

    Sorry I keep changing my mind. Basically, I want [itex]\mathbf T[/itex] to be a good angular approximation of all the vectors in [itex]\mathbb X[/itex].
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook