How Can I Quantitatively Validate My Numerical Model Against Experimental Data?

  • Context: Graduate 
  • Thread starter Thread starter py_engineer
  • Start date Start date
  • Tags Tags
    Model Numerical
Click For Summary
SUMMARY

This discussion focuses on validating a numerical model against experimental data by comparing simulated results of 'RA' versus temperature 'T'. The user seeks to quantify the similarity between their model and experimental data using statistical parameters. Recommendations include employing mean absolute error (MAE) or mean absolute percentage error (MAPE) as effective metrics for this purpose, particularly when calculated on independent observations. The discussion emphasizes the importance of theoretical grounding in the chosen validation parameters.

PREREQUISITES
  • Understanding of Partial Differential Equations (PDEs)
  • Familiarity with statistical metrics such as Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAPE)
  • Experience with data visualization techniques, particularly plotting in MATLAB
  • Knowledge of non-linear data analysis methods
NEXT STEPS
  • Research statistical methods for model validation, focusing on MAE and MAPE
  • Explore data visualization techniques in MATLAB for effective comparison of datasets
  • Learn about non-linear regression analysis to better understand the behavior of 'RA' versus 'T'
  • Investigate theoretical frameworks for model validation in numerical simulations
USEFUL FOR

Researchers, data scientists, and engineers involved in numerical modeling and validation, particularly those working with experimental data comparison and statistical analysis.

py_engineer
Messages
11
Reaction score
0
Hi,

I developed this numerical model where I solve a set of PDEs that allows me to simulate an imaging detector with different parameters, etc.

Now, I would like to compare my model with a particular case where experimental data has been obtained. I made a very simple plot to explain what I want to do:

http://www-personal.umich.edu/~pyemelie/plot.bmp

The variable in the experimental data is 'RA' and is plotted versus temperature T.

Using my numerical model, I can simulate the detector in the conditions of the experimental case, and simulate 'RA' as well versus temperature (at the same temperature points than the experimental data, as shown in the little figure).

If I plot the experimental 'RA' and my simulated 'RA' versus T on the same graph, that should give something similar to what you can see in the image.

Now, I would like to find some theoretical parameters, correlation parameters or something like that (I am not really good with Statistics..) in order to give a quantitative value of how close my simulation data set is from the experimental one. Could anyone give me some advice/recommendations on this??

I don't want to go too deep in the model validation. I think that just a graphical analysis (such as just plotting the simulated and experimental data on the same graph) and a quantitative parameter would do.. But I would like still to use some kind of parameter with a theoretical background (meaning, I don't just want to make up my own parameter).

I want to point out that (in case it's relevant), as you see in the plot, the 'RA' versus T is not a linear plot, and that the 'RA' data points can vary by several orders of magnitude.

Thanks a lot!
 
Last edited by a moderator:
Physics news on Phys.org
It's not clear to me whether you are more interested in RA or log(RA), but I don't see why a simple mean absolute error or mean absolute percentage error couldn't be used to characterize your model's error, though it should be calculated on observations not used to fit the model function.


-Will Dwinnell
http://matlabdatamining.blogspot.com/"
 
Last edited by a moderator:

Similar threads

  • · Replies 23 ·
Replies
23
Views
4K
  • · Replies 30 ·
2
Replies
30
Views
4K
  • · Replies 2 ·
Replies
2
Views
1K
  • · Replies 12 ·
Replies
12
Views
3K
Replies
0
Views
837
  • · Replies 0 ·
Replies
0
Views
2K
Replies
1
Views
3K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
Replies
24
Views
3K