# Solving Uncertainty in Simulation Models

I shall be telling this with a sigh
Somewhere ages and ages hence:
Two roads diverged in a wood, and I–
I took the one less travelled by,
And that has made all the difference

…Robert Frost, 1916

Uncertainty in your operations is most likely complex and will need systematic treatment through simulation modeling. S@R carries out thorough analysis of companies risk and uncertainties with aim of producing good decision support tools. Making sure the client takes a huge step forward from scenario analysis.

### The Four Levels of Uncertainty

The uncertainty that remains after the best possible analysis has been done is what we call residual uncertainty (Courtney, Kirkland & Viguerie, 1997).

In our world ‘the best possible analysis’ means that we have a model ‘good enough’ to describe the business under study. The question then is – do we need to take into account the uncertainties that always will be inherent in its operations and markets?  And if we have to, is it possible?

A useful distinction between the different situations that can arise is given by Courtney et al.  as four levels of residual uncertainty (see figure below, McKinsey Quarterly, Dec. 2008):

1. A Clear-Enough Future; managers can develop a single forecast of the future that is precise enough for strategy development.
2. Alternate Futures; the future can be described as one of a few discrete scenarios. Analysis cannot identify which outcome will occur, although it may help establish probabilities.
3. A Range of Futures; a range of potential futures can be identified. That range is defined by a limited number of key variables, but the actual outcome may lie anywhere along a continuum bounded by that range.
4. True Ambiguity; multiple dimensions of uncertainty interact to create an environment that is virtually impossible to predict.

In real life there is however a problem with identifying the level we are facing. The definition of 1th level uncertainty indicates that the residual uncertainty is irrelevant to the strategic decisions under study. But how is it possible to know this before an uncertainty analysis has been performed?

The answer has to be that the best possible analysis performed has been a risk/uncertainty analysis taking into account all known uncertainties in the business’s environment and that of all the business’s feasible strategies one is always best (1th order stochastic dominance).

The best strategy will then be the one giving a probability distribution for the business’s equity value that is located to the right and under the distributions for all other strategies. In this case the resulting equity value is of less importance since it anyway will be larger than under any other strategy. With this established, the actual analysis can be performed as a deterministic calculation.

For the 2nd level uncertainties with alternate futures, a scenario analysis is often advocated. However the same applies to each alternative future as for the 1th level uncertainties (also see scenario analysis). In addition some assumptions have to be made on the probabilities of each of the alternative futures.

As an example we can take a company analyzing investment in production facilities in two alternative countries. In one country there is a sovereign risk of a future new tax scenario and if it is imposed two different scenarios is possible. In the other country there is a fixed tax scenario – not expected to change. In this case you will need at least three (maximum five) models all taking into account the inherent risk in the business, giving the probability distribution for equity value for;

1. current operations,
2. current operations + Investment in the country with no sovereign risk, and
3. current operations + Investment in the country with sovereign risk;
1. no new tax scenario and
2. with each of the two different new tax scenarios.

The reason for different models even if the operations in the new facility will be the same regardless of country, lies in the fact that the business strategy might differ between countries and the investment strategy might differ for different tax scenarios. The model with sovereign risk will switch between the different tax scenarios models, according to the probability of their occurrence – generating the distribution for equity value given the sovereign risk.

To invest, at least one of the equity distributions for ‘Current operations + Investment’ should be located to the right and under the distributions for ‘Current operations’ (or be stochastic dominant). Likewise, the best investment alternative will have an equity distribution located to the right and under the distributions for the other alternative (or be stochastic dominant).

Having the equity distribution for the dominant strategy, opens for measurement of the strategy’s inherent risk beyond the use of simple value at risk calculations, putting emphasis on the possibility of large losses and further unwanted capital infusions.

As we now can see, directly applying a standard scenario analysis can quickly lead decision makers astray.

The above classification for the two first levels can in general only be performed after a full the risk/uncertainty analysis and can never be used ex ante to select the appropriate method.

The 3rd level uncertainties describes the normal situation where all exogenous variables have a range of possible values. Assuming that we can find (estimate or guesstimate) the probability distribution over that range, we can attack the problem by Monte Carlo simulation and calculate the probability distributions for our endogenous variables.

The 4th level uncertainties comprises at least two different situations; where there are unknown but knowable probabilities and where there are unknown and unknowable probabilities:

Ambiguity is uncertainty about probability, created by missing information that is relevant and could be known (Camerer & Weber, 1992).

This leads us to a more comprehensive discussion of the situations that will arise in decision making processes:

More generally, we propose that in most decision problems, “choice” is nothing but the terminal act of a problem-solving activity, preceded by the formulation of the problem itself, the identification of the relevant information, the application of pre-existing competences or the development of new ones to the problem solution and, finally, the identification of alternative courses of action. (Dosi & Egidi, 1991)

### The origin of uncertainty

Uncertainty may have two origins:

1. the lack of all the information which would be necessary to make decisions with certain outcomes (substantive uncertainty), and
2. limitations on the computational and knowledge based capabilities, given the available information (procedural uncertainty).

The first source of uncertainty comes from information incompleteness, and the second from the inability to recognize, interpret and act on the relevant information, even when it is available – knowledge incompleteness.

To distinguish between the two different situations giving Courtney‘s 4th level uncertainty we will follow Dosi & Egidi:

1. Weak substantive uncertainty (analogous to Knight’s “risk”) is all circumstances where uncertainty simply derives from lack of information about the occurrence of a particular event – with a certain known (or at least knowable) probability distribution, and
2. Strong substantive uncertainty (analogous to Knight and Keynes “uncertainty”) is all cases involving unknown events or the impossibility, even in principle, of defining the probability distributions of the events themselves.

### Types of Uncertainty

Uncertainty estimation usually includes the estimation of the uncertainty of the output parameters by estimating the uncertainty of the input parameters. This is done by estimating a probability distribution of the error. Hence, it is pretty much “straight forward” as long as the input parameters have values. However, the uncertainty of a model may not only be estimated via the parameters, there may also be uncertainty in the structure of a model, e.g. which variable and parameters are important in the model.

Adopting the distinction between parametric- and structural uncertainty (Kyläheiko et al., 2002) we can further specify model uncertainty:

1. Parametric; uncertainty or imperfect knowledge about the parameters in the decision model, and
2. Structural (epistemic); uncertainty or imperfect knowledge about the structure of the model.

Combining the above we can describe the types of risk and uncertainty facing both the decision maker and the decision support model as in the following picture:

The purpose is then to solve weak substantive parametric and structural uncertainty using good methods and models. The model will constitute a mix of facts1 (certain values), risks with known (objective) probability distributions, uncertainties given by subjective probability distributions and a script of the firm’s operations.

### Modeling

However, models will always have some structural uncertainty – even if it would be possible to remove all by introducing more and more variable and relations. Occam’s Razor can usually be applied with good results; select the model that introduces the fewest assumptions and postulates the fewest entities while still sufficiently answering the question. Borrowing from multidimensional scaling the term ‘stress’ – as the violation done to the actual decision structure by removing parameters or variable from the model – we can visualize this by the following figure:

Reducing the dimensionality of the model will not necessarily reduce or move (distort) the endogenous variables event space since correlation exists between variable omitted and variable kept in the model and – depending on estimation methods – the standard errors of estimated relationships will increase, maintaining the original model variability.

### Strategy

Maybe the world and the uncertainties we face haven’t changed all that much as a result of the financial crisis, but our perception of risks has. That means there is a real opportunity to rethink the way we make strategic decisions, the way we plan under uncertainty. (Courtney, McKinsey Quarterly, Dec. 2008)

The development of strategy requires the courage to accept uncertainty. Strategists must accept that they will not have all of the information and not see the full spectrum of possible events, yet be committed to create and implement strategy. The uncertainty that exists is not only a product of not having complete information and being able to predict future events, it also is a product of the events generated by dynamic and thinking competitors.

By its nature, uncertainty invariably involves the estimation and acceptance of risk. Risk is equally common to action and inaction. Risk may be related to gain; greater potential gain often requires greater risk. However, we should clearly understand that the acceptance of risk does not equate to the imprudent willingness to gamble the entire likelihood of success on improbable events.

One important step in the direction of better and more informed decision making is the removal of procedural uncertainty by having good models capable of framing the environment of the circumstances under which the decisions are made – giving the best possible analysis.

It is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail. (Maslow, 1966)

S@R carries out thorough analysis of companies risk and uncertainties with aim of producing good decision support tools. Making sure the client takes a huge step forward from scenario analysis.

### References

A fresh look at strategy under uncertainty: An interview, McKinsey Quarterly, December 2008.

Camerer, C. & Weber, M., (1992). Recent Developments in Modelling Preferences: Uncertainty and Ambiguity, Journal of Risk and Uncertainty, Springer, vol. 5(4), 325-70.

Courtney, H., (2001). 20/20 Foresight. Boston: Harvard Business School Press.

Courtney, H. G., Kirkland, J., & Viguerie, P. S., (1997). Strategy Under Uncertainty. Harvard Business Review, 75(6), 67-79.

Dequech, D., (2000), Fundamental Uncertainty and Ambiguity, Eastern Economic Journal, 26(1), 41-60.

Dosi, G & Egidi, M, (1991). Substantive and Procedural Uncertainty: An Exploration of Economic Behaviours in Changing Environments, Journal of Evolutionary Economics, Springer, 1(2), 145-68.

Frost, R., (1916). Mountain interval. Henry Holt And Company.

Keynes, J., (2004). A Treatise on Probability. New York: Dover Publications.

Knight, F. (1921). Risk, Uncertainty and Profit. Boston: Houghton Mifflin.

Kylaheiko K., Sandstrom J. & Virkkunen V., (2002). Dynamic capability view in terms of real options. International Journal of Production Economics, Volume 80 (1), 65-83(19).

Maslow, A., (1966). The Psychology of Science. South Bend: Gateway Editions, Ltd.

### Endnotes

1. In a simulation the opening balance is usually considered as certain, but the balance sheet often contains highly uncertain items. In fact auditors should give interval estimates for the most critical items in the yearly balance report []