Metric for rating funtion accuracy from R^m to R.

[TylerH]

I'm writing program in which I generate pseudo random expressions with the hope of finding one that closely approximates the given data set. The functions map from R^m (an m-tuple of reals) to a real. What I need is a way to rate the functions by their accuracy. Are there any known methods for doing this? Maybe something from stats.

Ideally this would be a method that would take a list of expected outputs and actual outputs and return a real number. A uniform distribution would be good, but not required.

 

[Office_Shredder]

A standard thing is to do something like
$$\sum_{j} \left( y_j - z_j \right)^2$$

where the y_j are the actual data that you measured, and the z_j are the predictions that you made (this is called the sum of squared errors). It's used mostly because it's easy to work with sums of squares when doing analytical calculations (like trying to minimize it). If you're doing a numerical method, people will often instead use the sum of the absolute values of the errors, especially if they consider the property "makes sure there are no significant outliers at the cost of being off by slightly more on average" to be a negative quality

 

[TylerH]

I went with the abs value one because, by nature of the fact they're all random, there are going to be a lot of bad solutions. So I thought a metric which doesn't underestimate their badness would be better. Thanks.

 

[Office_Shredder]

I went with the abs value one because, by nature of the fact they're all random, there are going to be a lot of bad solutions. So I thought a metric which doesn't underestimate their badness would be better. Thanks.

I don't really understand what this post is trying to get at... do you mean that your data has a lot of noise in it?

 

[CellCoree]

There are several methods that can be used to rate the accuracy of a function mapping from R^m to R. One approach is to use a regression analysis, which involves fitting a mathematical model to the data set and then evaluating the model's performance in predicting the actual outputs. This can provide a numerical measure of the function's accuracy, such as the coefficient of determination (R^2).

Another method is to use a loss function, which measures the discrepancy between the expected outputs and the actual outputs. This can be used to calculate a mean squared error or root mean squared error, which can provide a measure of the function's accuracy.

Additionally, techniques such as cross-validation or holdout validation can be used to evaluate the performance of the function on a separate set of data, which can provide a more robust measure of accuracy.

It is important to note that the choice of metric for rating function accuracy may depend on the specific goals and requirements of your program. It may be helpful to consult with a statistician or data scientist to determine the most appropriate method for your particular application.