Purpose

This standard was developed in response to Action 4 from the 2004 report "A Renewed Commitment to Excellence," which stated the following:

"Develop a standard for the development, documentation, and operation of models and simulations

a. Identify best practices to ensure that knowledge of operations is captured in the user interfaces (e.g., users are not able to enter parameters that are out of bounds),

b. Develop process for tool verification and validation, certification, reverification, revalidation, and recertification based on operational data and trending,

c. Develop standard for documentation, configuration management, and quality assurance,

d. Identify any training or certification requirements to ensure proper operational capabilities,

e. Provide a plan for tool management, maintenance, and obsolescence consistent with modeling/simulation environments and the aging or changing of the modeled platform or system,

f. Develop a process for user feedback when results appear unrealistic or defy explanation."

Subsequently, in 2006, the NASA Chief Engineer provided the following further guidance:

g. "Include a standard method to assess the credibility of the models and simulations presented to the decision maker when making critical decisions (i.e., decisions that effect human safety or mission success) using results from models and simulations,

h. Assure that the credibility of models and simulations meet the project requirements."

Each of the requirements and recommendations in this standard can be traced to one or more of the eight objectives listed above. The traceability matrix of the requirements in this standard to the eight objectives can be found online upon accessing this standard at URL http://standards.nasa.gov; refer to "Requirements Traceability Matrix." Some of these objectives are met by recommendations rather than by requirements as a result of either (a) the practical impossibility of satisfying the requirement in all cases, or (b) further guidance received from NASA Headquarters.

These eight objectives are encapsulated in the overall goal for this standard, which is to ensure that the credibility of the results from MS is properly conveyed to those making critical decisions. Critical decisions based on MS results, as defined by this standard, are those technical decisions related to design, development, manufacturing, ground, or flight operations that may impact human safety or program/project-defined mission success criteria. The intent is to reduce the risks associated with critical decisions. This standard covers the development and operation (or execution) of MS as well as the processes of analysis and presentation of the results from the MS.

This standard addresses aspects of MS that are common across NASA activities. Disciplinespecific details of MS should be addressed in future documents, such as Recommended Practices (usually entitled "Handbooks" in the NASA document hierarchy), and are not included in this standard.

The scope of this standard covers the development and maintenance of models, the operation of simulations, the analysis of the results, training, recommended practices, the assessment of the MS credibility, and the reporting of the MS results. Some of the key features of this standard are requirements and recommendations for verification, validation, uncertainty quantification, training, credibility assessment, and reporting to decision makers; also included are the crosscutting areas of documentation and configuration management (CM).

The requirements/recommendations in sections 4.7 and 4.8 are the culmination of the standard. The requirements/recommendations in sections 4.1 – 4.6 are intended to support the requirements in sections 4.7 and 4.8. This is accomplished by ensuring that sufficient details of the MS process along with intermediate results are available to support the requirements in sections 4.7 and 4.8 and to respond to in-depth queries by the decision maker. Appendix A provides guidance for assessing the risk of using MS in engineering decisions. Appendix B provides details related to some of the requirements/recommendations in sections 4.7 and 4.8. Appendix C contains a template for a compliance matrix.