Technical note | Factor Screening Techniques for Combat Simulation Models

Abstract

A stochastic simulation model can be viewed as a system that takes a series of input variables, and then processes them to produce a realisation of the required simulation scenario output. This model will contain a set of parameters, known as factors, that characterise the simulation process. There may in fact be significant complexity with the underlying simulation architecture, and additionally there may be a very large number of factors. The cost of this is slow performance in simulation run times. In many cases there may be factors in the underlying model that are insignificant statistically in the scenario of interest. This phenomenon is referred to as the parsimony principle, or the Pareto 80-20 rule. Hence there is much interest in the scientific literature on the screening of factors in the underlying meta-model applied to the dynamical system. This report thus investigates the subject of factor screening, for stochastic simulation models, and overviews several solutions to this problem. In particular, sequential bifurcation will be shown to provide a very efficient approach to the problem of factor screening, in comparison to standard one factor at a time classification methods. Several variations of sequential bifurcation will be examined, and their performance in several factor screening examples will be investigated. Directions for possible future research will also be discussed.

Executive Summary

The research project reported here is in support of the Land Capability Analysis (LCA) MSTC's development of an efficient factor screening method to be used with meta-models applied in the analysis of combat simulations generated with computational simulation packages. Combat simulations may involve a large number of factors, that determine various settings of characteristics of the model. In view of the Pareto 80-20 rule, or parsimony principle, only a small number of these factors may have a significant effect on the system response. Hence the challenge is to find an efficient technique or tool to screen these factors, to allow for more efficient use of the simulator.

The report begins by examining the standard testing procedure, where factors are screened one at a time. Then the report examines the process known as sequential bifurcation, where factors can be screened in groups. This procedure is shown to be more efficient than screening factors individually.

In order to account for stochastic variability in the system response, it is necessary to examine sequential bifurcation in the presence of stochatic variability. Hence a series of appropriate techniques are examined. The performance of these will to shown to vary in terms of efficiency and accuracy. Consequently, some rules of thumb will be proposed to facilitate their application in practice.

As a result of these investigations, future research directions will be outlined, that can be used as a basis for further investigations in LCA's combat simulation capability.