Note: This discussion is about an older version of the COMSOL Multiphysics® software. The information provided may be out of date.

Discussion Closed This discussion was created more than 6 months ago and has been closed. To start a new discussion with a link back to this one, click here.

How to validate results?

Please login with a confirmed email address before reporting spam

Hello all,

I have recently completed a simulation, i am getting results but don't know whether they are right or wrong? I did search some papers... in some of them they have validated their results with the previously published work. I dont know how to do that? can anyone help?

3 Replies Last Post Nov 10, 2016, 6:40 a.m. EST

Please login with a confirmed email address before reporting spam

Posted: 7 years ago Nov 9, 2016, 2:51 p.m. EST
Hello,

I think reading "Verification and Validation in Scientific Computing" from William Oberkampf
should give you some hints how to approach verification and validation of your model.

For verifikation you have to show that your numerical solution to a given problem is sufficiently accurate. Which depends on the requierements for your simulation.

You should check all model assumptions, parameters, boundary conditions, formulaes and their implementation for mistakes or bugs.

Given everything is implemented in the right way, you then should do a systematic mesh or/and timestep refinement study.

Usually no exact analytical solution for your given task is available (otherwise you would not need to do a simulation), you can do a code to code comparison with previously published work. This means comparing some of the important system responses from your model with the reference.

This can be done qualitatively by optical comparison of two diagrammes. Best suited for this are 1D Plots, for 2D Data you could compare isocontour plots if they are available in the reference papers.

For a quantitative comparison (which is better) you need some kind of data. This this you can calculate the differences between the data and for example getting the maximum absolute or relative difference.

But most often you do not have acess to the data from the reference study. You could also check some characteristic values of the system responses. Something like min or max of a scalar field. This data is sometimes printed in papers.

Best suited for a direct comparison would be results from a numerical model which has exactly the same assumptions, physics and numerical solution techniques as your model. If the other model has a lower complexity than yours a comparison could still be usefull, not as a final test but more as an intermediate test to see if you're on the right path.

During my master thesis I learned that there's a wide variety of previously proposed and solved numerical models with historically increasing level of complexity. Most of these can be used for a code to code comparison.

For validation you have to compare the prediction given by your model with results obtained from an experiment. In the best case scenario a fairly good agreement could be reached. But for that you need first to verify your computer model and than you would need a very precise determination of the coefficients used for material properties or analytical submodels.

If you start right away with comparison of experimental data (validation) without verification, a disagreement could occur from implementation mistakes, a mesh which is too coarse. So a two step process is necessary.

Best regards
Hello, I think reading "Verification and Validation in Scientific Computing" from William Oberkampf should give you some hints how to approach verification and validation of your model. For verifikation you have to show that your numerical solution to a given problem is sufficiently accurate. Which depends on the requierements for your simulation. You should check all model assumptions, parameters, boundary conditions, formulaes and their implementation for mistakes or bugs. Given everything is implemented in the right way, you then should do a systematic mesh or/and timestep refinement study. Usually no exact analytical solution for your given task is available (otherwise you would not need to do a simulation), you can do a code to code comparison with previously published work. This means comparing some of the important system responses from your model with the reference. This can be done qualitatively by optical comparison of two diagrammes. Best suited for this are 1D Plots, for 2D Data you could compare isocontour plots if they are available in the reference papers. For a quantitative comparison (which is better) you need some kind of data. This this you can calculate the differences between the data and for example getting the maximum absolute or relative difference. But most often you do not have acess to the data from the reference study. You could also check some characteristic values of the system responses. Something like min or max of a scalar field. This data is sometimes printed in papers. Best suited for a direct comparison would be results from a numerical model which has exactly the same assumptions, physics and numerical solution techniques as your model. If the other model has a lower complexity than yours a comparison could still be usefull, not as a final test but more as an intermediate test to see if you're on the right path. During my master thesis I learned that there's a wide variety of previously proposed and solved numerical models with historically increasing level of complexity. Most of these can be used for a code to code comparison. For validation you have to compare the prediction given by your model with results obtained from an experiment. In the best case scenario a fairly good agreement could be reached. But for that you need first to verify your computer model and than you would need a very precise determination of the coefficients used for material properties or analytical submodels. If you start right away with comparison of experimental data (validation) without verification, a disagreement could occur from implementation mistakes, a mesh which is too coarse. So a two step process is necessary. Best regards

Please login with a confirmed email address before reporting spam

Posted: 7 years ago Nov 10, 2016, 3:24 a.m. EST
Thankyou sir for your guidance,

I have done some literature review, in one paper the author has validated his results by using a previously published work. He made a separate geometry for the validation which was similar to the published work. He used same parameters and data ( i do not know how did he get all the information about that paper but he said so in his paper) of published work. I got confused at first time that why do we have to make another simulation in order to validate our actual simulation.
The published work was kind of similar but simpler compare to his work.
I have also chosen a paper for validation and make a separate simulation for it. However, i am not getting the exact results, seeking more papers to validate.
Thankyou sir for your guidance, I have done some literature review, in one paper the author has validated his results by using a previously published work. He made a separate geometry for the validation which was similar to the published work. He used same parameters and data ( i do not know how did he get all the information about that paper but he said so in his paper) of published work. I got confused at first time that why do we have to make another simulation in order to validate our actual simulation. The published work was kind of similar but simpler compare to his work. I have also chosen a paper for validation and make a separate simulation for it. However, i am not getting the exact results, seeking more papers to validate.

Please login with a confirmed email address before reporting spam

Posted: 7 years ago Nov 10, 2016, 6:40 a.m. EST
A direct comparison of simulation results is only useful if both
models have exactly the same geometry, physics and model parameters.
A comparable numerical solution technique and discretization would also be
helpful.

I would suggest reading:
On the Role of Code Comparisons in Verification and Validation
prod.sandia.gov/techlib/access-control.cgi/2003/032752.pdf

"I got confused at first time that why do we have to make another simulation in order to validate our actual simulation"

You can only compare your simulation if the model in the reference paper is exactly the same. If the other uses another geometry you also have the use that geometry. This way you can compare results for the same physical system.

You can only get exaclty the same results if everything is the same, included the comsol version you're using. Most often this is not the case, so a disagreement should be expected. Also it would be helpful if you establish some kind of maximum difference value if you compare solutions.

If you only look at a maximum value something like |sol - solref|<= 5% could be used.

One other possibility is that mistakes have been made in the other model.
A direct comparison of simulation results is only useful if both models have exactly the same geometry, physics and model parameters. A comparable numerical solution technique and discretization would also be helpful. I would suggest reading: On the Role of Code Comparisons in Verification and Validation http://prod.sandia.gov/techlib/access-control.cgi/2003/032752.pdf "I got confused at first time that why do we have to make another simulation in order to validate our actual simulation" You can only compare your simulation if the model in the reference paper is exactly the same. If the other uses another geometry you also have the use that geometry. This way you can compare results for the same physical system. You can only get exaclty the same results if everything is the same, included the comsol version you're using. Most often this is not the case, so a disagreement should be expected. Also it would be helpful if you establish some kind of maximum difference value if you compare solutions. If you only look at a maximum value something like |sol - solref|

Note that while COMSOL employees may participate in the discussion forum, COMSOL® software users who are on-subscription should submit their questions via the Support Center for a more comprehensive response from the Technical Support team.