Info-gap decision theory

Criticism

A general criticism of non-probabilistic decision rules, discussed in detail at decision theory: alternatives to probability theory, is that optimal decision rules (formally, admissible decision rules) can always be derived by probabilistic methods, with a suitable utility function and prior distribution (this is the statement of the complete class theorems), and thus that non-probabilistic methods such as info-gap are unnecessary and do not yield new or better decision rules.

A more general criticism of decision making under uncertainty is the impact of outsized, unexpected events, ones that are not captured by the model. This is discussed particularly in black swan theory, and info-gap, used in isolation, is vulnerable to this, as are a fortiori all decision theories that use a fixed universe of possibilities, notably probabilistic ones.

Sniedovich raises two points to info-gap decision theory, one substantive, one scholarly:

1. the info-gap uncertainty model is flawed and oversold

One should consider the range of possibilities, not its subsets. Sniedovich argues that info-gap decision theory is therefore a "voodoo decision theory."

2. info-gap is maximin

Ben-Haim states (Ben-Haim 1999, pp. 271–2) that "robust reliability is emphatically not a [min-max] worst-case analysis". Note that Ben-Haim compares info-gap to minimax, while Sniedovich considers it a case of maximin.

Sniedovich has challenged the validity of info-gap theory for making decisions under severe uncertainty. Sniedovich notes that the info-gap robustness function is "local" to the region around ${\displaystyle {\displaystyle \tilde{u}}}$, where ${\displaystyle {\displaystyle \tilde{u}}}$ is likely to be substantially in error.

Maximin

Symbolically, max ${\displaystyle \alpha }$ assuming min (worst-case) outcome, or maximin.

In other words, while it is not a maximin analysis of outcome over the universe of uncertainty, it is a maximin analysis over a properly construed decision space.

Ben-Haim argues that info-gap's robustness model is not min-max/maximin analysis because it is not worst-case analysis of outcomes; it is a satisficing model, not an optimization model – a (straightforward) maximin analysis would consider worst-case outcomes over the entire space which, since uncertainty is often potentially unbounded, would yield an unbounded bad worst case.

Stability radius

Sniedovich has shown that info-gap's robustness model is a simple stability radius model, namely a local stability model of the generic form

${\displaystyle \hat{\rho }(\tilde{p}):=max\{\rho \ge 0:p\in P(s),\mathrm{\forall }p\in B(\rho ,\tilde{p})\}}$

where ${\displaystyle B(\rho ,\tilde{p})}$ denotes a ball of radius ${\displaystyle \rho }$ centered at ${\displaystyle \tilde{p}}$ and ${\displaystyle P(s)}$ denotes the set of values of ${\displaystyle p}$ that satisfy pre-determined stability conditions.

In other words, info-gap's robustness model is a stability radius model characterized by a stability requirement of the form ${\displaystyle r_c\le R(q,p)}$. Since stability radius models are designed for the analysis of small perturbations in a given nominal value of a parameter, Sniedovich argues that info-gap's robustness model is unsuitable for the treatment of severe uncertainty characterized by a poor estimate and a vast uncertainty space.

Discussion

Satisficing and bounded rationality

It is correct that the info-gap robustness function is local, and has restricted quantitative value in some cases. However, a major purpose of decision analysis is to provide focus for subjective judgments. That is, regardless of the formal analysis, a framework for discussion is provided. Without entering into any particular framework, or characteristics of frameworks in general, discussion follows about proposals for such frameworks.

Simon introduced the idea of bounded rationality. Limitations on knowledge, understanding, and computational capability constrain the ability of decision makers to identify optimal choices. Simon advocated satisficing rather than optimizing: seeking adequate (rather than optimal) outcomes given available resources. Schwartz, Conlisk and others discuss extensive evidence for the phenomenon of bounded rationality among human decision makers, as well as for the advantages of satisficing when knowledge and understanding are deficient. The info-gap robustness function provides a means of implementing a satisficing strategy under bounded rationality. For instance, in discussing bounded rationality and satisficing in conservation and environmental management, Burgman notes that "Info-gap theory ... can function sensibly when there are 'severe' knowledge gaps." The info-gap robustness and opportuneness functions provide "a formal framework to explore the kinds of speculations that occur intuitively when examining decision options." Burgman then proceeds to develop an info-gap robust-satisficing strategy for protecting the endangered orange-bellied parrot. Similarly, Vinot, Cogan and Cipolla discuss engineering design and note that "the downside of a model-based analysis lies in the knowledge that the model behavior is only an approximation to the real system behavior. Hence the question of the honest designer: how sensitive is my measure of design success to uncertainties in my system representation? ... It is evident that if model-based analysis is to be used with any level of confidence then ... [one must] attempt to satisfy an acceptable sub-optimal level of performance while remaining maximally robust to the system uncertainties." They proceed to develop an info-gap robust-satisficing design procedure for an aerospace application.

Alternatives

Of course, decision in the face of uncertainty is nothing new, and attempts to deal with it have a long history. A number of authors have noted and discussed similarities and differences between info-gap robustness and minimax or worst-case methods . Sniedovich has demonstrated formally that the info-gap robustness function can be represented as a maximin optimization, and is thus related to Wald's minimax theory. Sniedovich has claimed that info-gap's robustness analysis is conducted in the neighborhood of an estimate that is likely to be substantially wrong, concluding that the resulting robustness function is equally likely to be substantially wrong.

On the other hand, the estimate is the best one has, so it is useful to know if it can err greatly and still yield an acceptable outcome. This critical question clearly raises the issue of whether robustness (as defined by info-gap theory) is qualified to judge whether confidence is warranted, and how it compares to methods used to inform decisions under uncertainty using considerations not limited to the neighborhood of a bad initial guess. Answers to these questions vary with the particular problem at hand. Some general comments follow.

Sensitivity analysis

Sensitivity analysis – how sensitive conclusions are to input assumptions – can be performed independently of a model of uncertainty: most simply, one may take two different assumed values for an input and compares the conclusions. From this perspective, info-gap can be seen as a technique of sensitivity analysis, though by no means the only.

Robust optimization

The robust optimization literature provides methods and techniques that take a global approach to robustness analysis. These methods directly address decision under severe uncertainty, and have been used for this purpose for more than thirty years now. Wald's Maximin model is the main instrument used by these methods.

The principal difference between the Maximin model employed by info-gap and the various Maximin models employed by robust optimization methods is in the manner in which the total region of uncertainty is incorporated in the robustness model. Info-gap takes a local approach that concentrates on the immediate neighborhood of the estimate. In sharp contrast, robust optimization methods set out to incorporate in the analysis the entire region of uncertainty, or at least an adequate representation thereof. In fact, some of these methods do not even use an estimate.

Comparative analysis

Classical decision theory, offers two approaches to decision-making under severe uncertainty, namely maximin and Laplaces' principle of insufficient reason (assume all outcomes equally likely); these may be considered alternative solutions to the problem info-gap addresses.

Further, as discussed at decision theory: alternatives to probability theory, probabilists, particularly Bayesians probabilists, argue that optimal decision rules (formally, admissible decision rules) can always be derived by probabilistic methods (this is the statement of the complete class theorems), and thus that non-probabilistic methods such as info-gap are unnecessary and do not yield new or better decision rules.

Maximin

As attested by the rich literature on robust optimization, maximin provides a wide range of methods for decision making in the face of severe uncertainty.

Indeed, as discussed in criticism of info-gap decision theory, info-gap's robustness model can be interpreted as an instance of the general maximin model.

Bayesian analysis

As for Laplaces' principle of insufficient reason, in this context it is convenient to view it as an instance of Bayesian analysis.

The essence of the Bayesian analysis is applying probabilities for different possible realizations of the uncertain parameters. In the case of Knightian (non-probabilistic) uncertainty, these probabilities represent the decision maker's "degree of belief" in a specific realization.

In our example, suppose there are only five possible realizations of the uncertain revenue to allocation function. The decision maker believes that the estimated function is the most likely, and that the likelihood decreases as the difference from the estimate increases. Figure 11 exemplifies such a probability distribution.

Now, for any allocation, one can construct a probability distribution of the revenue, based on his prior beliefs. The decision maker can then choose the allocation with the highest expected revenue, with the lowest probability for an unacceptable revenue, etc.

The most problematic step of this analysis is the choice of the realizations probabilities. When there is an extensive and relevant past experience, an expert may use this experience to construct a probability distribution. But even with extensive past experience, when some parameters change, the expert may only be able to estimate that ${\displaystyle A}$ is more likely than ${\displaystyle B}$, but will not be able to reliably quantify this difference. Furthermore, when conditions change drastically, or when there is no past experience at all, it may prove to be difficult even estimating whether ${\displaystyle A}$ is more likely than ${\displaystyle B}$.

Nevertheless, methodologically speaking, this difficulty is not as problematic as basing the analysis of a problem subject to severe uncertainty on a single point estimate and its immediate neighborhood, as done by info-gap. And what is more, contrary to info-gap, this approach is global, rather than local.

Still, it must be stressed that Bayesian analysis does not expressly concern itself with the question of robustness.

Bayesian analysis raises the issue of learning from experience and adjusting probabilities accordingly. In other words, decision is not a one-stop process, but profits from a sequence of decisions and observations.