Composite Risk Index Methodology

This document provides a technical description of the methodology used to calculate the Composite Risk Index (CRI).

The CRI represents a cumulative risk value aggregating all reported, assessed and severity classified safety-related incidents to form an index. This measure of risk exposure is based on probability and severity that considers the human perception of equivalent risk.

Note that CRI can be calculated for any given entity, provider or the State.

Figure 1 shows all components of proposed CRI index.

Composite Risk Index components.

Figure 1: Composite Risk Index components.

Acronyms and terminology

Table 1: Acronyms and terminology
TermDefinition
CRIComposite Risk Index
CRInormComposite Risk Index normalised by flight hours
RIRunway Incursion
UPAUnauthorised Penetration of Airspace
SMISeparation Minima Infringement
TECHATM Specific / Technical occurrences

Methodology

As a proxy of safety risk within certain airspace, State, or any Entity, the concept of exposure to risk, based on reported / historical safety information was proposed.

In order to calculate the composite risk, each historical, reported occurrence had to have assigned severity (using Risk Assessment Tool methodology) and probability (calculated from historical safety data).

Modelling can be performed using any source of safety occurrences data, as long as the input satisfies the minimum data requirements.

The CRI calculations are based on the following information:

  • Accidents
  • Operation occurrences (high/medium risk incidents with Severity A to C):
    • runway incursions (RI)
    • separation minima infringements (SMI)
    • unauthorized penetrations of airspace (UPA)
  • Other operational occurrences
  • Technical occurrences (high/medium risk incidents with Severity AA/A to C)

A detailed data description can be found at the end of this page.

To take into account the local conditions within each Entity, exposure data, i.e. the total number of flight hours within each Entity is used as an additional input (CRI normalised results).

The process is visualised in Figure 2.

Composite Risk Index process.

Figure 2: Composite Risk Index process.

Occurrences with no severity classification

All severity unclassified/not assessed events (Severity Category D) are distributed into groups A to E, based on historical distribution (determined using the all available safety data).

The probability of occurrence being assigned to a specific severity category is calculated using historical data, separately for each State/Entity in order to simulate/take into consideration the local conditions.

Figure 3 shows how unassessed occurences (severity D) are distributed into other severity groups.

Distribution of unassessed occurences (severity D) into other severity groups.

Figure 3: Distribution of unassessed occurences (severity D) into other severity groups.

Estimated numbers of occurrences

The formula how the total numbers of specific type of occurrence were determined using the distribution based on probability of unclassified occurrences is presented below.

\[ \text{No_occ_est}_i = ((\text{No_D}_i * P_i) + \text{No_occ}_i) \]

where

\(\text{No_occ_est}_i\)
estimated number of safety occurrence type \(i\)
\(\text{No_D}_i\)
number of reported occurrences of Severity Classification D (not determined) for occurrence type \(i\)
\(P_i\)
probability of safety occurrence type \(i\)
\(\text{No_D}_i\)
number of reported safety occurrences of type \(i\)

The probability of each type of occurrence is calculated using the simple principle:

\[ P_i = \frac{\text{No_occ}_i}{\text{Total No}_{{occ}_i}} \]

\(\text{No_occ}_i\)
number of reported safety occurrences type \(i\) in a year
\(\text{Total No}_{{occ}_i}\)
total number of reported safety occurrences type \(i\) in a group in all years

CRI calculation using human perception of risk

Using the estimated total number of occurrences for each type and adding to it the human perception of their risk, it is possible to calculate the CRI for each Entity separately. The simple formula to calculate the CRI is presented below.

\[\begin{align} \text{CRI}_i & = \text{No}_{\text{occ_est_i}} * w_i \\ \text{CRI} & = \sum_{i}^{} \frac{\text{CRI}_i}{\text{Total No}_{{occ}}} \end{align}\]

where

\(\text{CRI}_i\)
CRI index for occurrence type \(i\)
\(w_i\)
weight (based on severity and human perception) assigned to specific type of safety occurrence
\(\text{Total No}_{{occ}}\)
total number of all occurrences in a year

In the formula above the Weights added to each equation represent the additional human perception of the risk for the specific event so that the CRI can at the end consider the human perception of equivalent risk.

Introduction of Weights to express the severity of the event allows their description in non-monetary terms which have a meaningful and understandable explanation in human perception.

Each weight value for specific Severity category is determined by using the optimisation technique, with the aim to select a combination of weights that will not disturb the computation of the CRI from year to year if significant changes in reporting are introduced. In other words, the optimisation will find which combination of weights result in the lowest standard deviation of CRI values between the years for each Entity.

Due to a large number of variables involved and the enormous number of combinations possible, the optimisation and selection of the Weights combination is done in stages:

  1. Selection of Weights for accident, all OPS occurrences and TECH occurrences

  2. Selection of Weights for OPS occurrences based on their type (RI, SMI, UPA, OTHER), taking into consideration overall OPS Weight determined in Step 1

  3. Selection of Weights for OPS and TECH occurrences based on their severity (AA/A, B, C), taking into consideration overall OPS Weight determined in Step 2

In addition, each type of weight selection has predefined weight ranges, to allow for an incremental Severity classification order, based on human perception of risk (from accident to Severity C incident, i.e. from high risk to low risk). In other words, each range has an expectation value associated with it. See further details in the methodology report (Commission 2019)

CRI normalisation

To allow the applicability of the CRI to airspaces with different traffic levels, the CRI is normalised by flight hours for each Entity. The CRInorm is calculated based on the following formula:

CRInorm =CRI/(Number of Flight Hours)

Data

For CRI calculation, each historical, reported occurrence besides classification, has to have assigned severity and probability.

Modelling of Weights can be performed using any source of safety occurrences data, as long as the input satisfies the minimum data requirements as in the table below. Note, that safety data, should include occurrence severity, reported and calculated using severity classification of the Risk Assessment Methodology.

VariableDescription
Entitythe name of Entity within State, ANSP, or department
accidentsafety occurrence where: a person is fatally or seriously injured; the aircraft sustains damage or structural failure; the aircraft is missing or is completely inaccessible”
total_no_of_ops_occurrencesthe sum of safety occurrences (incidents) related to OPS
ri_arunway incursions severity A
smi_aseparation minima infringments severity A
upa_aunauthorised penetration of airspace severity A
ri_brunway incursions severity B
smi_bseparation minima infringments severity B
upa_bunauthorised penetration of airspace severity B
ri_crunway incursions severity C
smi_cseparation minima infringments severity C
upa_cunauthorised penetration of airspace severity C
ri_drunway incursions severity D
smi_dseparation minima infringments severity D
upa_dunauthorised penetration of airspace severity D
ri_erunway incursions severity E
smi_eseparation minima infringments severity E
upa_eunauthorised penetration of airspace severity E
total_no_of_tech_occurrencesthe sum of safety occurrences related to technical reasons
t_aatechnical occurrences severity AA
t_atechechnical occurrences severity A
t_btechchnical occurrences severity B
t_ctechechnical occurrences severity C
t_dtechechnical occurrences severity D
t_etechechnical occurrences severity E
total_number_of_all_reported_occurrencessum of all OPS and TECH occurences
yearyear
flight_hourssum of flight hours in Entity for year n

Severity

The classification scheme for safety occurrences in ATM specifies six severity categories for ATM related occurrences impacting the safe operations of the aircraft (as per ESARR2).

SeverityDescription
Aserious incident
Bmajor incident
Csignificant incident
Dnot determined
Eno safety effect

Frequency

The RAT classification scheme specifies five qualitative frequency categories (repeatability). However, these values are not commonly reported through the AST. Each State/Entity in principle should develop their own quantitative boundaries, which should consider national traffic volumes and specific operating conditions of the national ATM system.

Therefore, the occurrence probability is calculated using historical data from all available years separately for each Entity in order to simulate/take into consideration the local conditions.

Further Information

For further information please refer also to the detailed report explaining the development of the methodology (Commission 2019).

Working example

Script calculating the CRI using the explained methodology and sample data are available for testing purposes here.

Please feel free to use sample data provided to get familiar with the methodology.

User guide how to set up the environment can be found at ‘README.pdf’ file within repository.

References

Commission, Performance Review. 2019. “Composite Risk Index (CRI) - Methodology.” EUROCONTROL. https://ansperformance.eu/library/Composite_Risk_Index_methodology.pdf.