Introduction: The Limitations of Quantitative Risk Assessments (QRA)
Quantitative Risk Assessments (QRA) have become a staple in industries like oil and gas, aviation, healthcare, and finance. This systemic approach uses mathematical models to estimate risks and minimize the likelihood of hazardous events. However, real-world incidents like refinery explosions, rail fires, and chemical spills highlight significant gaps in QRA methodologies. These shortcomings often stem from an over-reliance on historical data, simplified assumptions, and the exclusion of critical variables, leading to a mismatch between estimated risks and actual outcomes.
This article explores real-world examples of QRA failures, analyzes why these shortcomings occur, and offers insights into how QRAs can be made more effective in risk management.
Understanding QRA: A Methodology with Gaps
QRA is designed to quantify the likelihood of incidents and help organizations implement mitigation measures to maintain risks within acceptable limits. Using large data sets, QRA attempts to predict the frequency of adverse events, such as fires or explosions, and their potential impacts.
Despite its widespread adoption, QRAs often fail to capture the complexity of real-world systems, resulting in risk predictions that underestimate the likelihood of catastrophic events. For example:
- Rail Accidents: U.S. Department of Transport’s acceptable risk criteria for hazardous material incidents suggest that events causing multiple fatalities should be extremely rare. Yet, in recent years, rail accidents involving hazardous materials have occurred more frequently than predicted, such as the chemical spill and fire in Ohio in 2023.
- Tunnel Fires: The Channel Tunnel between the U.K. and France, which operates under a QRA, has experienced several major fires (1996, 2006, 2008, 2012, and 2015), highlighting the gap between predicted and actual risk levels.
- Refinery Explosions: Incidents like the BP Texas City Refinery explosion in 2005, which killed 15 people and injured 170, demonstrate that QRAs often underestimate the role of organizational and human errors.
Key Shortcomings in QRA Methodologies
1. Over-Reliance on Historical Data
QRAs often depend heavily on past incident data without accounting for unique operating conditions, evolving technologies, and organizational complexities. Historical data may fail to reflect present-day realities, leading to underestimation of risks.
2. Simplified Assumptions
Many QRAs use overly simplified models that do not adequately capture the complexity of modern industrial systems. This can lead to unrealistic predictions, as seen in the BP Texas City Refinery explosion.
3. Exclusion of Critical Factors
QRAs often overlook the role of human error, organizational deficiencies, and systemic issues. For instance, many models fail to incorporate the likelihood of cascading failures or interdependencies within large systems.
4. Frequency Mismatches
Events classified as “once in a lifetime” or “extremely rare” based on QRA are occurring with greater regularity, revealing inaccuracies in the predicted frequency of incidents.
Real-World Implications: A Pattern of Underestimation
Oil and Gas Industry
- QRAs in the oil and gas sector predict fire and explosion incidents to occur once every 10 years per refinery. However, real-world data shows these events happening more frequently, often with devastating consequences.
Transportation Sector
- Rail incidents involving hazardous materials and major tunnel fires provide clear evidence that QRAs fail to account for all variables influencing risk.
BP Texas City Refinery Explosion
- The 2005 disaster underscored critical QRA flaws, including reliance on outdated historical data and simplified operational models. The assessment failed to capture the likelihood of human and organizational errors, which played a key role in the explosion.
How to Make QRAs More Effective
1. Incorporate Qualitative Insights
While quantitative data is crucial, integrating qualitative information, such as expert knowledge and organizational assessments, can provide a more holistic view of risks.
2. Account for Human and Organizational Errors
Future QRAs must factor in human and organizational deficiencies, which are often overlooked but play a significant role in catastrophic events.
3. Improve Modeling Techniques
Using advanced computational models and simulations can help QRAs better capture the complexity of modern operations, including interdependencies and cascading failure modes.
4. Regular Updates and Scenario Testing
QRAs should be updated frequently to reflect changes in technology, operations, and external conditions. Scenario testing and stress-testing models against worst-case scenarios can also help identify vulnerabilities.
5. Combine Historical and Predictive Data
A balanced approach that combines historical data with predictive analytics can improve the accuracy of risk assessments and make them more relevant to current and future conditions.
Conclusion: Using QRA as a Tool, Not a Solution
Quantitative Risk Assessments remain a valuable tool for identifying and mitigating risks, but they should not be relied upon as the sole solution. The examples highlighted—ranging from refinery explosions to rail and tunnel fires—demonstrate the limitations of current QRA methodologies.
To make QRAs more effective, industries must move beyond a purely quantitative approach and integrate qualitative insights, advanced modeling, and regular updates. By addressing these shortcomings, QRAs can better approximate real-world risks and help organizations mitigate hazards more effectively.
SPE members can access the complete paper on the SPE Health, Safety, Environment, and Sustainability Technical Discipline page for free from 30 January to 12 February.