|
Scenario Assessment
65
terrorist attacks, probabilities can be derived from insurance and reinsurance tables.
If they are not available, an option is to use the implicit probability in the prices
of catastrophe bonds. For scenarios inspired by large events observed in external
databases, the frequency assessed can be related to the frequency observed in the
external database, or adjusted based on the similarity judgment method. Other
scenarios are the worst-case version of the crystallization of risks assessed during the
RCSA exercise (see previous chapters). The frequency assessment should then be
aligned to the results produced during the RCSA exercise.
R A N G E O F S C E N A R I O A S S E S S M E N T T E C H N I Q U E S
Regulators are well aware of the difficulty of scenario assessments. Given the
important role scenarios play in the measurement of capital for operational risk, it is
particularly important to get quantifications as reliable and as justifiable as possible.
Regulators request a high level of repeatability for the process used to generate
scenario data, through consistent preparation and application of the qualitative and
quantitative results. Firms are asked to reduce subjectivity and biases as much
as possible in the process. This is achieved by basing assumptions on empirical
evidence, explaining the rationale behind the level at which a scenario is analyzed, and
maintaining good quality documents on the assumptions and process for delivering
scenario assessment.
This section reviews some of the most common techniques used for scenario
assessment in financial services. The practice has evolved significantly over the
last few years and now offers better, more structured and reliable techniques. For
a comprehensive discussion on scenario analysis and assessment techniques, see
Hassani (2016).
1
E x p e r t J u d g m e n t
In its most unstructured version, expert-based scenario assessment is limited to inter-
viewing subject matter experts related to the scenario in question. A typical question
is: “What would be the worst impact generated by an internal (or external) fraud at a
probability of 1 in 40 years? And of 1 in 100 years?” This method is quick and inex-
pensive. However, results are highly dependent on the selection of experts, and even
with the support of preparatory documents, it does little to mitigate behavioral biases
in estimation. This method was common in the industry ten years ago, but because it
lacks structure and reliability, it is now increasingly less acceptable to regulators and
so is fading.
1
Hassani, B. (2016)
Scenario Analysis in Risk Management
, Springer.
66
RISK ASSESSMENT
S y s t e m a t i c E s t i m a t i o n a n d M i t i g a t i o n o f B i a s
This form of expert analysis uses more structured questions and benchmarks, based on
experience and similar events, to help reduce estimation biases. The following cases
present examples of assessment biases and how to mitigate them in the context of
scenario assessment.
Availability bias and recency bias: this occurs when recent events are perceived
as more likely than earlier events. For example, pandemic scenarios prolif-
erated after the bird flu and swine flu outbreaks of recent years; and terrorist
attack scenarios were common after 2001, before disappearing from the lists
of many institutions and then reappearing in late 2016, after the wave of ter-
ror attacks in Europe. To counteract recency bias, it helps to use longer time
spans of data, with the length depending on the type of risk environment. Long
data spans are appropriate for natural events, destruction of physical assets,
pandemics, fraud and other types of stable incident. Fast-evolving risks, par-
ticularly technology-based risks such as cyberattacks, will require smaller
time spans to be relevant.
Anchoring, confirmation and group polarization bias: anchoring is the tendency to
focus on the first piece of information that we encounter (such as a number),
anchoring it high or low and allowing it to influence other judgments. Con-
firmation bias is the tendency to believe anything that confirms our existing
views and opinions – a psychological effect used by politicians, marketers
and journalists. Group polarization bias appears when people exhibit more
extreme views as a group than they would as individuals. History reveals many
tragic consequences of this bias. One way of addressing these different biases
is to introduce a first round of “silent estimates” amongst participants, so that
the first person to comment doesn’t anchor the rest of the group’s assessments.
Workshop participants are asked to answer questions in writing without con-
sulting each other. The answers are then revealed and discussed. During the
debate, the workshop facilitator must allow everyone to express and defend
their opinion freely, without being dominated by the loudest or most opin-
ionated person in the room. Often, a small group of subject matter experts is
better than a large group of semi-informed managers, who can add noise to
the estimates. The “wisdom of the crowd” is not recommended in complex
topics such as scenario assessments.
Estimation biases are ingrained in human psychology and are unlikely to be com-
pletely eliminated. Being aware of our own limitations is an essential step toward mit-
igating bias. Specific training courses are available in expert elicitation and calibration
Scenario Assessment
67
techniques for those who want to study those questions further.
2
Overall, it is better to
support judgment with as much data as is available and to use the right specialists for
each scenario.
D e l p h i M e t h o d : P o o l i n g E x p e r t J u d g m e n t s
The Delphi technique has been used for decades in opinion surveys. It is a relatively
easy and inexpensive technique to run and is well suited to scenario assessment. The
method has the following steps:
Do'stlaringiz bilan baham: |
