Calculation of "momentum" vector's head coordinates

[LucKy]

TL;DR

Is it possible to calculate coordinates of Green vector’s "head" given available information? Or given information is not enough?

Hello all

I’m not sure whether this question should be posted here or in general math section.

I summarized my question on below picture.

Question is:
Is it possible to calculate coordinates of Green vector’s "head" given available information? Or given information is not enough?
If solution is possible, I will appreciate if someone could share a method on how to do this.
I can solve this problem only if I have coordinates of Blue vector’s tail, but this information is unavailable (not enough RAM, explanation below).

It is not homework, it for my personal hobbyist project:

I’m trying to implement Neural Network Gradient Descent with Momentum to run on MCUs with very limited RAM.
Blue vector represents update of NN weights at previous training cycle,
Green vector -represents “momentum” of previous update and it should be added to next weights update calculation later.
Dimensions – NN weights.
Because of limited memory I can’t store information about weights updates at previous cycle (it will double size of each NN's layer in memory).
I only figured out to store data about Blue vector's magnitude and cosine of angle with some reference vector (Red vector).

 

[jedishrfu]

Can’t you store blue vectors tail and the cosine with red? Ie don’t store reds coordinates.

What about a reduced float datatype like bf16 that effectively double your capacity giving you space to store the blue vectors tail.

One thing that confuses me is the coordinates of the blue vector tail even needed ?

It seems like youre trying to place the blue vector at some point in your n dimensional space or do you mean the components of the blue vector itself.