Credibility challenges for flood risk models
In closing of my previous post, I mentioned that stylized facts are not the sole property of the social sciences. The geophysical sciences has plenty. (My guess is they exist throughout the sciences as a whole).
A recent study by Trigg et al in ERL on flood models demonstrate the production of stylized facts. The article, "The credibility challenge for global fluvial flood risk analysis" (funded by the Willis Re network that I mentioned earlier here) demonstrates that vastly different measures of flood risk are obtained depending on the model used. The abstract is as follows,
Quantifying flood hazard is an essential component of resilience planning, emergency response, and mitigation, including insurance. Traditionally undertaken at catchment and national scales, recently, efforts have intensified to estimate flood risk globally to better allow consistent and equitable decision making. Global flood hazard models are now a practical reality, thanks to improvements in numerical algorithms, global datasets, computing power, and coupled modelling frameworks. Outputs of these models are vital for consistent quantification of global flood risk and in projecting the impacts of climate change. However, the urgency of these tasks means that outputs are being used as soon as they are made available and before such methods have been adequately tested. To address this, we compare multi-probability flood hazard maps for Africa from six global models and show wide variation in their flood hazard, economic loss and exposed population estimates, which has serious implications for model credibility. While there is around 30%–40% agreement in flood extent, our results show that even at continental scales, there are significant differences in hazard magnitude and spatial pattern between models, notably in deltas, arid/semi-arid zones and wetlands. This study is an important step towards a better understanding of modelling global flood hazard, which is urgently required for both current risk and climate change projections.
The authors note that many flood risk models now exist but,
However, to date, all global flood hazard models have had limited validation against observed flood flows or extents. Partly, this is because they are different to other more local scale models in this field, and so cannot draw on a rich heritage of previous testing methods, but mainly it is due to the difficulty of undertaking validation comprehensively over such large spatial scales, particularly in data scarce areas where risk products are most needed.
The author's discussion provides various instances of substantial model disagreement. For instance,
- At a 1-in-25 year return period, this flooded area ranges from 3% to 8.3% of the continent and for a 1-in-1000 year return period, 4.2%–10.5%, depending upon the model.
- Exposure analysis by country also shows big differences between the model results (figure 4(c)), for example, Egypt ranging from approximately 1%–50%, depending on the model.
However, Trigg et al make a substantial assumption about the relationship between scientific advancement and political decision making- highlighted in bold below.
With the frequency and magnitude of flood disasters projected to increase due to both climate change and growing population exposure [3, 4], flooding is one of the key societal challenges for this century. In order to address this challenge, knowledge of the expected flood hazard for a given probability is required for risk reduction. ...Some countries have made significant progress in this regard, due to greater wealth, political will and more comprehensive data availability. However, fluvial (river) flood risk for much of the world is still 'unmapped', and even where mapping exists, it often uses different and inconsistent methodologies or datasets across countries and regions. This lack of consistent risk information makes global and national efforts to reduce risk and increase resilience as well as high level planning and decision making, particularly challenging. (emphasis mine)
That is, in order to make risk policy flood risk must become certain. This assumption has plagued the climate science community for a number of decades and may be part of the reason climate policy stalled on various occasions. The final word on risk measures has many practical challenges such as, data limitations, times scales under consideration and the nature of science as a practice as forever identifying uncertainty. Making scientific certainty for future outcomes a requisite to action ensure no action is possible- at least not on any meaningful times scale. It is only in recent years that scientific certainty is a requisite to decision making. Nations have long done more with far less scientific information than is currently available.
When we here complaints about measures of risk being wrong or needing to account for "scientific advancements" doing so depends on subjective judgements about the state of scientific knowledge and whose problem one wishes to solve and choice of model that best represent one's own view of risk.