Formulas for : CVRMSE and Net Mean Bias

[paluee]

Hi folks,

I am in the energy industry, and we do linear/multilinear regressions stats.
One of the concepts being introduced to supplement standard way of doing regressions is make use of:

(1) Coefficient of Variation Root Mean Squared Error

(2) Net Mean Bias or Abs (Mean bias)

Unfortunately, I can't find the explicit formula's.
Below is a list one goes thru to ensure a good fit/model,
some of the rules of thumb used in our industry:
*******************************************
1. R2 (Quality of fit): 0.75 or higher, with 1.00 as perfect fit.
2. CVRMSE Eng (Coefficient of Variation Root Mean Squared Error): 25 or lower for consumption meters.
3. CVRMSE Dmd (Coefficient of Variation Root Mean Squared Error): 35 or lower for demand meters.
4. Abs (Mean bias) - Must be equal to or less than 0.005.
5. Abs (T-Stats) - Positive 2.0 or higher for CDD and HDD, and greater than 2.0 or less than
-2.0 for any user variable.
6. HDD and CDD T-Stat > 0. This box must be checked to be ASHRAE and IPMVP
compliant

*******************************************
So in the context of our industry, say one take
a years worth of data, meaning 12 months of data, therefore 12 data points.
One performs a regression on this 12 data points to obtain an equation,
then one tries to interpolate these data points: a01=month1, a02=month2, etc.

Here is some output:
Data||ActualRaw||RegressInterpolatedValues
a01 51520 51345
a02 37120 37738
a03 33280 37512
a04 28800 28757
a05 17920 18641
a06 17280 17953
a07 19200 18965
a08 20800 18727
a09 27200 27243
a10 31040 31719
a11 38720 35605
a12 52480 51028

One of the methods used was to find the sum of each column and get a percent difference, and then get the differences of each month doing
=(actual-Regress)/actual * 100%
This was to get a measure of deviation of the regression to the actualdata.

But I must now use : CVRMSE and Net_Mean_Bias.

Does anybody have the explicit formula's for these 2 statistics.
Hope somebody knows these.

Here is an excerpt from our industry on this:
*************************************
users only heeded the R2 value and T-statistics, keeping them in
the range as detailed above. This was the recognized practice in the energy analysis industry. In the 2002, ASHRAE came
out with Guideline 14, which specified not an R2 value, but instead a CVRMSE and a Mean Bias Error (along with Tstatistics).
The energy analysis industry has been slow to catch on, still relying mostly on the R2 value. This was mostly
due to the fact that the software never had the CVRMSE and Mean Bias Error implemented—making it hard for energy
analysts to tune to these values

*************************************

SO hope some can tell me the formulas, or at least point me
to some source.



Paluee

 

[statdad]

The cvrmse is calculated the same way as the ordinary coefficient of variation, with the rmse replacing the standard deviation. that is, with

$$c.v. = \frac{s}{\bar x}$$

you get

$$cvrmse = \frac{RMSE}{\bar x}$$

There is some discussion of mean bias error in this document

http://www.tva.gov/sami/met/eval/ch5.PDF

although I'm not sure if it's what you are dealing with.

It would be interesting to see the statistical reasoning behind this: I've never worked with these before, but it seems rather sketchy (not a reference to your comments).