Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

A Sum of random variables, given sum of observed variables

  1. Jun 24, 2016 #1
    I have a model in which, for each store, predicted revenues are perturbed by a multiplicative shock:

    [itex]R = e^\eta r[/itex]

    where [itex]r[/itex] is predicted and [itex]R[/itex] is observed. [itex]\eta[/itex] is mean zero.

    I can find [itex] \eta [/itex] as follows: [itex]\ln( r) - \ln( R) = \eta [/itex]. I'm summing the squares of the [itex]\eta[/itex]'s.

    However, there are some markets where I only observe the sum of sales of multiple stores. The simplest example would be for two stores:
    where I only observe [itex] R_1 + R_2 [/itex]. I would like to calculate [itex] \eta_1^2 + \eta_2^2 [/itex] (or at least its expected value). Is there any way I can do this? Is there anything at all that I can say about [itex] \eta_1^2 + \eta_2^2 [/itex] or [itex] \eta_1 + \eta_2 [/itex] ? I'm OK with making assumptions about the distribution of [itex]\eta[/itex] if that's necessary.

  2. jcsd
  3. Jun 24, 2016 #2

    jim mcnamara

    User Avatar

    Staff: Mentor

    Can you discern from the raw data which category the observables fall into? (summed or not summed)
  4. Jun 24, 2016 #3
    Yes. If they are summed, I know how many stores are included in the group.
  5. Jun 26, 2016 #4

    Stephen Tashi

    User Avatar
    Science Advisor

    The technically correct language is probably that you would like to "estimate" it , not "calculate" it - unless you are asking how to compute each sampled value of [itex] \eta_1^2 + \eta_2^2 [/itex] from the corresponding sampled value of [itex]\eta_1 + \eta_2 [/itex]. (The importance of this technicality is that if you are doing web searches on on the problem, it is a problem of "statistical estimation" or finding a good "estimator" etc. )

    First, let's see if I visualize your data correctly. I visualize N different "revenue producing areas"
    The data for the ##n##_th revenue producing area consists of ##M## samples of the total revenue for stores in that area: ##S_{n,i}, n = 1,2,3,...N, i = 1,2,3,...M## .

    For each ##k##_th store in ##n##_th revenue producing area, the model for the store's revenue is ## R_{n,k} = e^{\eta_{n,k}} r_k ##.

    The random variables involved are the ##\eta_{n,k}##.

    Is the "predicted" ##r_k## a constant value for each sample? Or do you have data with records like:

    area = 3, sample = 5, store r_1 = $ 634.30, store r_2 = $532.23, observed sum of revenues S = $12432.60
    area = 3, sample = 6, store r_1 = $ 236.00, store r_2 = $530.00, observed sum of revenues S = $655.92

  6. Jun 27, 2016 #5
    You're correct about the "estimation" vs "calculation" comment.

    Regarding your question at the end, I am assuming that [itex] r [/itex] is the same for each store within an area (I use zip codes as my area). In other words, my model spits out a single predicted revenue number for each zip code.

    Data Generating Processes for zip code A, which has 3 stores, and zip code B, which has 2 stores: :
    What I observe for A is: The number of stores (3) and the total revenue for those three stores, [itex] R_A [/itex] . What I observe for B is the number of stores (2) and the total revenue for these stores, [itex] R_A [/itex].
  7. Jun 27, 2016 #6

    Stephen Tashi

    User Avatar
    Science Advisor

    Ok, that's clear.

    Now, as to what you wish to estimate - do wish to estimate the distribution of the revenue from each store ?

    Is that the eventual purpose of estimating the distribution of ## \eta_{1,B} +\eta_{2,B} ## ? I don't see the direction connection between estimating the distribution of that sum (or its expectation) and the problem of estimating the distribution of revenue for each individual store.
  8. Jun 27, 2016 #7
    Thanks for the help thus far.

    The short answer is that, given observed [itex] R [/itex] and predicted [itex] r [/itex] for each location, I would like to estimate [itex] \eta_{1}^{2}+\eta_{2}^{2}+...+\eta_{N}^{2} [/itex], where [itex] N[/itex] is the number of stores in that location.

    The more explanatory answer is that I am trying to construct a model for predicting revenues. I am doing this my minimizing the sum of squared residuals (so, a nonlinear least squares estimation). If I observed each store's revenue, then it would just be a matter of minimizing the sum of [itex] (\ln(R_s )-\ln(r_s))^{2} [/itex], where [itex] s [/itex] is the number of stores . However, I'm concerned that using this method on the data I have (where I have location data but not individual store data) gives each location the same weight, even though they have different numbers of stores (and, therefore, different numbers of realizations of [itex] \eta [/itex]). I've tried things like multiplying each location's squared residual by the number of stores in each location, but I wanted to see if there was any way to show mathematically that this is a reasonable thing to do (or, show why the "basic" way is fine)

    To get technical... the basic estimation is like this:
    [tex] \hat{\theta}=\underset{\theta}{argmin}\sum_{s}(\ln(R_{s})-\ln(r))^{2} [/tex]
    where [itex] r=f(\theta) [/itex]
  9. Jun 27, 2016 #8

    Stephen Tashi

    User Avatar
    Science Advisor

    The ##\eta_i## are random variables, so their sum is a random variable. For a given sales area (zip code) you presumably have several realizations of the total revenue ( daily sales figures or something like that). Are you seeking to estimate the value of ##\eta^2_{1,k} + \eta^2_{2,k} + ... + \eta^2_{N,k} ## for each day k = 1,2,....M ?

    If you knew each stores revenue, why wouldn't you fit your model by minimizing the sum of ##(R_{s,k} - r_{s,k})^2## ?

    Certain transformations of data make least squares fitting easier, but the least squares fit for the transformed data isn't necessarily the least squares fit for the untransformed data. For example the "error" between $100 and $10 is 1 in terms of ##log_{10}##, as is the error between $1000 and $100.
  10. Jun 28, 2016 #9
    Thanks again for all your help on this. To answer your questions...

    Yes, I have panel data, so I have data for multiple zip codes over multiple months.
    Yes, that's what I'm trying to do. I'm summing the error over each month and each location.
    If I knew each store's revenue (which I don't), that's what I would do.

    Right. I'm currently using the multiplicatve error term for two reasons. First, that's what the paper I'm basing my work off of uses. Second, using additive error makes some of the errors for large zip codes gigantic and essentially swamp the small errors that exist in small zip codes. I'm also running some models that use additive error.

    I'm starting to concede that there's not much I can do here, simply because I don't have enough information. As an example, if I have a zip code with two stores and predicted revenue of $5 per store.

    If each [itex] \eta [/itex] were equal to its expectation (which is zero), then my actual revenue would equal expected revenue, which is $10. Suppose, instead, that observed revenue is $100. There are many different values of [itex] \eta_{1} [/itex] and [itex]\eta_{2}[/itex] that would give this. For example:

    [itex]\eta_{1}=0 [/itex] and [itex] \eta_{2}\thickapprox2.94 [/itex]
    [itex]\eta_{1}=\eta_{2}\thickapprox2.3 [/itex]
    or many other combinations.

    These will lead to different values of [itex]\eta_{1}^{2}+\eta_{2}^{2} [/itex] (in this example the squared sum will be 8.64 in the first instance and 10.58 in the second instance). I don't know what the actual realizations of [itex]\eta[/itex] will be. I could assume that [itex]\eta_{1}=\eta_{2}[/itex], but I'm not sure that I can do that (since that would mean that the two error terms are perfectly correlated, and everything I've done with least squares assumes i.i.d. error terms; Thinking about this another way, the zip code would really just have a single realization of [itex]\eta[/itex], with that term being used for both [itex]\eta_{1}[/itex] and [itex] \eta_{2} [/itex]).
  11. Jun 28, 2016 #10

    Stephen Tashi

    User Avatar
    Science Advisor

    I agree with the impossibility of deducing individual values for the realizations of [itex] \eta_1, \eta_2 [/itex].

    I'm trying to understand your focus on the random variable ##\eta^2_1 + \eta^2_2##. Is the idea that observed values of ##\eta^2_1 + \eta^2_2## would allow you to compare the predicted variance of total sales ##R## with observed variance of total sales ? (You mention least squares, but you haven't given the definition of the predicted variable. You don't have a model predicting the standard deviations for the distributions of the ##\eta_i## do you ? )

    I agree with your pessimism about deducing the values of individual realizations of the random variables ##\eta_1, \eta_2## from a datum that only gives their sum. However, the seemingly more ambitious problem of estimating the distributions of random variables from data that only gives their sum is, as the saying goes, a "well known" problem. It's known as a problem of "deconvolution".

    When a given distribution can be expressed as the sum of two random variables in more than one way, a deconvolution problem is usually solved by adding some condition on the solution that makes it unique.

    I did a brief web search on "deconvolution, lognormals" and found some highly specialized articles where people claim to have deconvolved data into the sum of lognormally distributed random variables, but the specialized contents makes the mathematics obscure - and I'm not sure the mathematics is good. A complication in searching is that many people don't make a distinction between "decomposing" a distribution as "mixture" of distributions and "deconvolving" a distribution into a "convolution" of distributions. The latter task is what you have.

    I'm optimistic that we can solve your deconvolution problem. It will require some thinking. If you are writing up your work for an article, you'll have to decide whether the audience will accept some mathematics that is not routinely seen in introductory statistics courses.
  12. Jun 29, 2016 #11
    I'm attempting to minimize the sum of squares. So, if there were 3 stores and I observed them all, I would minimize [itex] \eta_{1}^{2}+\eta_{2}^{2}+\eta_{3}^{2}[/itex]
    . If, instead, stores 2 and 3 are aggregated together, I think that I need to minimize [itex] \eta_{1}^{2}+\{\mathrm{some\ approximation\ of\ \eta_{2}^{2}+\eta_{3}^{2}}\}[/itex]. The trick, of course, is figuring out how to approximate [itex]\eta_{2}^{2}+\eta_{3}^{2}[/itex]

    Two more comments:
    - I'm not married to the idea of the multiplicative error term - I would switch to a simple linear error term if that makes things easier. I'm using the [tex ]e^{\eta} [\itex] error term because revenues are always positive.

    - I had never heard of deconvolution, but I'm reading up on the subject now.

    - This is my job market paper in economics. The more obscure math it contains the better :)
  13. Jun 29, 2016 #12
    I had an idea. This could be way off base, but here goes...
    I'm going to simplify everything here by using an additive error term (rather than multiplying by [itex] e^\eta [/itex]). For the simple two-stores-in-a-zip example, the actual data-generating-process is:
    If I observed both store's revenues, I would be looking to minimize
    [tex] \eta_{1}^{2}+\eta_{2}^{2}=(R_{1}-r_{1})^{2}+(R_{2}-r_{2})^{2}[/tex]

    However, I don't observe both revenues. Instead, my goal is to minimize the expectation of these, when all I observe is [itex]
    (\eta_{1}+\eta_{2})=R_{1}+R_{2}-r_{1}-r_{2} [/itex]
    So, I want to compute the following conditional expectation:
    [tex] \mathrm{E}[\eta_{1}^{2}+\eta_{2}^{2}|(\eta_{1}+\eta_{2})^{2}] [/tex]
    It seems as though it should be possible to solve the above if I assume that the [itex]\eta[/itex]'s are distributed normally with mean zero and a constant s.d.
  14. Jun 29, 2016 #13

    Stephen Tashi

    User Avatar
    Science Advisor

    I don't understand that terminology. The ##\eta_i## are random variables. So what would it mean to "minimize" them? You could minimize some parameter of their distribution. For example, you could set their standard deviations equal to zero.

    I think you are talking about observed values instead of random variables. The usual scenario for least squares is that we pick parameters for our prediction model (or distribution or whatever) that minimize the least squares error between a predicted value and an observed value. I don't see that a particular k_th observation ##\eta^2_{1,k} + \eta^2_{2,k} + \eta^2_{3,k}## is a difference between a predicted value of something and the observed value of the thing.

    As I understand your model, least squares estimate of the observed value of the random variable ##\eta_1 + \eta_2 + \eta_3## would be zero because the ##\eta_i## each have mean zero. The mean value of a random variable is the best least squares predictor of its realized values. Are you trying to find the best squares estimator of ##\eta^2_{1,k} + \eta^2_{2,k} + \eta^2_{3,k}## ? I don't see a direct relation between that estimator and finding the best fit of a revenue model to revenue data.
  15. Jun 29, 2016 #14

    Stephen Tashi

    User Avatar
    Science Advisor

    I understand that model as the two equations:

    That's not good terminolgy because it doesn't mention anything about a sum of squares or anything about data.

    We need to use terminology that distinguishes among random variables, the expectations of random variables, samples of random variables, and means of samples of random variables. You also need to distinguish between population parameters (e.g. expected values) and estimators of population parameters (e.g. the formula for finding a sample mean is an estimator of the population mean).

    That's the abstract goal of least squares fitting, but least squares fitting is usually done to data, not to theoretical distributions. If you knew both the theoretical distribution of the actual phenomena as well as the your theoretical family of models, you would know the distribution of the squares of errors. Then it would make sense to say that your goal is to pick a member of the family of models that minimized the expected value of squares of the errors.

    The usual situation is that we have data not known to be from a particular theoretical distribution. We assume it is from some family of distributions, but we still don't know exactly which member of the family it is. We don't know the distribution of the squares of the errors, so we can't find the expected value of that distribution.

    What we do is pick a formula for an estimator for the mean square error that is a function both of the data (which are known values) and of the parameters that define a member of our family. Then we find the values of the parameters that minimize the estimator.

    You want to "estimate" it.

    You have an interesting idea. It needs to be put in the correct language.

    A given population quantity can have several different estimators, each of which is "good" in a different way. Without going into the technicalities of what "good" shall mean in this particular problem, your idea is to find a good estimator for the (population) mean of ##\eta^2_1 + \eta^2_2 ## as a function of the variables ##\sigma_1, \sigma_2## and the given data, which are observations of ##(\eta_1 + \eta_2)##.

    Formulas for good estimators are often (but not always) suggested by looking at formulas for the population parameters that they estimate. For the example, the formula for the expected value of a random variable involves an integral. For an estimator of the population mean, we use the formula for the sample mean, which "imitates" the integral by using a finite sum. So I agree that it makes sense to look at the theoretical calculation for ##E( \eta^2_1 + \eta^2_2 | (\eta_1 + \eta_2) )##.
  16. Jul 12, 2016 #15


    User Avatar
    Science Advisor
    Homework Helper

    If the η's are independent, E[Σηi2] = E[(Σηi)2].

    My guess is the answer is symmetric. You cannot distinguish between stores, so the argmin solution is to average out R across stores in the area. Put differently, the sum of squared differences will be minimized when rA1 = rA2 = ...
  17. Jul 13, 2016 #16


    User Avatar
    Science Advisor

    If you can't write the final function you need as a function of your inputs then you won't be able to get estimates.

    Have you tried solving for this function first? Once you have that then you can start looking at constructing test statistics but until you have that you can't do anything useful with the data to do what you need to do.
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook

Have something to add?
Draft saved Draft deleted