Verification and Validation Cases

The terms Verification and Validation are often used interchangeably to mean the process of checking the accuracy of a numerical model. For many, this entails comparing model predictions with experimental measurements. However, there is now a fairly broad-based consensus that comparing model and experiment is largely what is considered Validation.

What's the difference between Verification and Validation?

ASTM E1355, "Standard Guide for Evaluating the Predictive Capability of Deterministic Fire Models," defines Verification as,

The process of determining that the implementation of a calculation method accurately represents the developer's conceptual description of the calculation method and the solution to the calculation method.

and it defines Validation as,

The process of determining the degree to which a calculation method is an accurate representation of the real world from the perspective of the intended uses of the calculation method.

Simply put, Verification is a check of the math, while Validation is a check of the physics. If the model predictions closely match the results of experiments, using whatever metric is appropriate, it is assumed by most that the model suitably describes, via its mathematical equations, what is happening. It is also assumed that the solution of these equations must be correct.

So why do we need to perform model Verification? Why not just skip to Validation and be done with it? The reason is that rarely do model and measurement agree so well in all applications that anyone would just accept its results unquestionably. Because there is inevitably differences between model and experiment, we need to know if these differences are due to limitations or errors in the numerical solution, or the physical sub-models, or both.

Validation typically involves:

1. Comparing model predictions with experimental measurements.
2. Quantifying the differences in light of uncertainties in both the measurements and the model inputs.
3. Deciding if the model is appropriate for the given application.

The Validation case studies in the repository only help with (1) and (2). Number (3) is the responsibility of the model user. A common question asked of any mathematical model is whether it is validated. To say that FDS is "validated" means that the model has been shown to be of a given level of accuracy for a given range of parameters for a given type of fire scenario.

Although the FDS developers continuously perform validation studies, it is ultimately the end user of the model who decides if the model is adequate for the job at hand. Thus, these case studies provide the raw material, but they do not in themselves constitute a complete and comprehensive validation study.

Online Resources

Verification and Validation Documentation

The most recent versions of the Verification and Validation Guides can be found through the following links.

Verification Guide (.pdf) Right-Click to Save

Validation Guide (.pdf) Right-Click to Save

FDS Input Files, Output Files and Relevant Data

The entire set of Verification and Validation files are available in the SVN repository. For more information on how to checkout the FDS-SMV project to your local computer, see the Accessing the SVN Repository Wiki.

For convinence the following links take you to files and directories that contain analytical solutions for Verification cases and experimental data for Validation cases, along with the corresponding FDS input and output files. Descriptions of the case studies can be found in the FDS Verification and Validation Guides.

Verification Directory:
http://code.google.com/p/fds-smv/source/browse/trunk/FDS/trunk/Verification

Validation Directory:
http://code.google.com/p/fds-smv/source/browse/trunk/FDS/trunk/Validation