Roache, P. J. (2004b), “Calculation Verification: an Overview”, Proc. Workshop on CFD Uncertainty Analysis, 21-22 October 2004. Instituto Superior Técnico, Lisbon, Portugal. L. Eça, and M. Hoekstra, eds.

This reference should have been cited with general references in Section 5.1 on Calculation Verification by the GCI.

Roache, P. J. (2004a), “Building PDE Codes to be Verifiable and Validatable,” Computing in Science and Engineering, Special Issue on Verification and Validation, September/October 2004, pp 30-38.

This reference should have been cited in Chapter 9 on Validation.

The following are excerpts. Some of these points are made elsewhere in the book, but not as specifically or forcefull as made here.

....

The New Paradigm of Experiments Designed Specifically for Code Validation

In my opinion, the most revolutionary concept in computational physics during my career, other than simulation itself, has been the new paradigm of experiments designed specifically for validation.1,2,5 The new paradigm recognizes that requirements for validation are distinct and that validation experiments are much easier in some respects but more demanding in others.

In aerodynamics, for example, the emphasis in precomputational days was on wind-tunnel experiments, which attempted to replicate free-flight conditions. Great effort was expended on achieving near-uniform inflow and model fidelity, and minimizing wall and blockage effects. The latter required small models, which sacrificed parameter fidelity (Reynolds number) and aggravated geometric fidelity.

The new paradigm approaches the problem differently, sacrificing some fidelity between the wind-tunnel flow and free flight, but requiring that more nearly complete details of the experimental conditions and field data be obtained. No longer is it so important to achieve uniform inflow, but it is critical to report in detail what those spatially varying inflow conditions are, so that they may be input to the computational simulation. The idea is that if the validation is good (by whatever criteria are appropriate) for a flow perturbed from the free-flight conditions, it will probably be good for the free-flight condition. Thus blockage effects are not such major issues (and the tunnel wall itself may be modeled), and models can be larger (or tunnels smaller and therefore cheaper), thereby improving fidelity of Reynolds number and model geometry. Analogous situations occur in other experimental fields.

......

Unrealistic Expectations Placed on Experimentalists

It is unrealistic, even arrogant, for a code builder or user to require an experimentalist to match idealized boundary conditions. Simple constant-value boundary conditions that are a mere convenience for the code builder can require major effort, cost, and time for an experimentalist; they often compromise other more desirable qualities of the experiment, and in fact may be literally impossible to achieve. A major contribution by the code builder to the synergistic cooperation between computationalists and experimentalists (which is also part of the new paradigm) is achieved by the relatively simple work of building the code with general boundary conditions. This also happens to be what is most needed for independent code verification (or confirmation) using the MMS.

Is Western Culture at Risk?

In an age of spreading pseudoscience and anti-rationalism, it behooves those of us who believe in the good of science and engineering to be above reproach whenever possible. Public confidence is further eroded with every error we make. Although many of society's problems can be solved with a simple change of values, major issues such as radioactive waste disposal and environmental modeling require technological solutions that necessarily involve computational physics. As Robert Laughlin20 noted in this magazine, “there is a serious danger of this power [of simulations] being misused, either by accident or through deliberate deception.” Our intellectual and moral traditions will be served well by conscientious attention to verification of codes, verification of calculations, and validation, including the attention given to building new codes or modifying existing codes with specific features that enable these activities.

References

1.P.J. Roache, Verification and Validation in Computational Science and Engineering, Hermosa Publishers, 1998.

2.P.J. Roache, “Quantification of Uncertainty in Computational Fluid Dynamics,” Ann. Rev. Fluid Mechanics, vol. 29, 1997, pp. 123–160.

3.Guide for the Verification and Validation of Computational Fluid Dynamics Simulations, AIAA G-077-1998, Am. Inst. Aeronautics and Astronautics, 1998.

...

20.R.B. Laughlin, “The Physical Basis of Computability,” Computing in Science & Eng., vol. 4, no. 3, 2002, pp. 22–25.