
#1
Dec2712, 03:02 PM

P: 122

Hello,
I was hoping that somebody could advise me how to do evaluate the performance of a model. The model simulates the power demand from a fleet of electric cars over 24 hours. For example take the blue line as the actual demand for a given day and the black lines (100) are simulated demands. Could anybody advise me on how I could quantify the performance of a the model? For example it varies by 5%, 10% etc? Would you use RMSE? I'm not sure if error is the correct word because the blue line varies from day to day depending which day you pick. Thank you 



#2
Dec2812, 01:55 PM

Mentor
P: 10,809

What do you want to do with your performance value?
How do you simulate those 100 curves? I think the blue curve is an average over many cars? To start, you could consider the mean squared deviation between blue line and simulations as timedependent function. 



#3
Dec2912, 04:28 PM

P: 122

Hi,
Thanks for your reply. The black curve were generated by a model that I have made to predict the power demands of the cars over the day. The model uses data collect by cars. The blue is the actual average power demand used by the cars. I create it my averaging the power demand of all the days in the dataset. I want a performance value to quote how accurate or not the model is in general. Is it capable of generating reliable power demand profiles? How reliable? Is mean squared deviation the same as mean square error? http://en.wikipedia.org/wiki/Mean_squared_error Is it essentially computing the difference between the blue line and the black line? Would it be a good idea to compute the mean squared deviation between the blue line and the 100 black lines and then take the average mean squared deviation? I computed the mse between the blue and 1 blak line ans an example in this workbook. I got an answer of 309. But what does this tell me about it accuracy? https://dl.dropbox.com/u/54057365/All/MSE.xlsx Thanks 



#4
Dec2912, 05:05 PM

Mentor
P: 10,809

How to quantify error in model 



#5
Dec3012, 04:05 PM

P: 122

Are you implying above that I compute the MSE between the blue line and the 100 black lines? Getting 100 MSE values? How would you check for timedependence? Thank you 



#6
Dec3012, 04:29 PM

Mentor
P: 10,809




Register to reply 
Related Discussions  
Why Can't We Quantify Wffs  Set Theory, Logic, Probability, Statistics  3  
Is it possible to quantify human language?  General Math  3  
how to quantify seebeck effect  Classical Physics  0  
How to quantify knifeedge sharpness  Mechanical Engineering  4  
How do we quantify the viability of a scientific model?  General Discussion  4 