Figures
Abstract
Satellite-based precipitation estimates products, CMORPH and PERSIANN-CCS, were evaluated with a dense rain gauge network over Beijing and adjacent regions for an extremely heavy precipitation event on July 21 2012. CMORPH and PEERSIANN-CSS misplaced the region of greatest rainfall accumulation, and failed to capture the spatial pattern of precipitation, evidenced by a low spatial correlation coefficient (CC). CMORPH overestimated the daily accumulated rainfall by 22.84% while PERSIANN-CCS underestimated by 72.75%. In the rainfall center, both CMORPH and PERSIANN-CCS failed to capture the temporal variation of the rainfall, and underestimated rainfall amounts by 43.43% and 87.26%, respectively. Based on our results, caution should be exercised when using CMORPH and PERSIANN-CCS as input for monitoring and forecasting floods in Beijing urban areas, and the potential for landslides in the mountainous zones west and north of Beijing.
Citation: Chen S, Liu H, You Y, Mullens E, Hu J, Yuan Y, et al. (2014) Evaluation of High-Resolution Precipitation Estimates from Satellites during July 2012 Beijing Flood Event Using Dense Rain Gauge Observations. PLoS ONE 9(4): e89681. https://doi.org/10.1371/journal.pone.0089681
Editor: Guy J.-P. Schumann, NASA Jet Propulsion Laboratory, United States of America
Received: August 20, 2013; Accepted: January 22, 2014; Published: April 1, 2014
Copyright: © 2014 Chen et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This work was supported by the R & D Special Foundation for Public Welfare Industry (meteorology) of China under Grant No. GYHY201306040; the National Natural Science Foundation of China with Grant No. 40905019, 41105098, 41275030; the State Key Laboratory of Severe Weather Open Fund 2013LASW–A09. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
On July 21, 2012, Beijing experienced one of the heaviest rain events in the past 60 years. The heavy rainfall triggered flash flooding and landslides, which killed 79 people and caused US $2 billion in direct economic losses, destroying at least 8,200 homes in the city and affecting more than 1.6 million people [1], [2]. A Study by [3] indicated that this extreme event was the result of a frontal system and mid-troposphere disturbance. Sang et al. [4] pointed out that topographic effects and natural climate variability, coupled with a changing climate system, may have contributed to the severity of this precipitation event. Additionally, low standards for mountain torrents monitoring and control for medium and small rivers in the effected regions contributed to the high-impact flooding. The inhomogeneous precipitation, both spatially and temporally, as well as excessive runoff due to increasing impervious surface areas, contributed to severe waterlogging in the urban area in Beijing.
Satellite-based quantitative precipitation estimates (QPE) products are widely applied to hydrologic modeling, hazards monitoring and climate research due to their global coverage and spatial continuity [5], [6], [7], [8], [9], [10]. Thermal Infrared (IR) and passive microwave (PMW) sensors are the most widely used instruments to quantitatively estimate rainfall on the ground. The physical bases of rainfall estimation from both IR and PMW sensors imagery are well explored and explained by many previous works [11], [12], [13]. Rainfall estimation from IR sensors depends on the assumption that the surface rainfall can be inferred from analyzing the cloud top characteristics, i.e., the cloud top temperature. That is, where the clouds which can reach highest altitude, and therefore have the lowest cloud top temperature are the most probably to precipitate. On the other hand, passive microwave measurements provide more physically direct link between hydrometeors in the atmosphere and radiances observed from a satellite. While being different in details, the PMW algorithms estimate rainfall rates fundamentally under the same principle: translating the scattering signature caused by ice water and/or emission signature caused by liquid water into a surface rainfall rate. Although the PMW sensors have clear advantages for rainfall estimation since they provide more information from the hydrometers themselves in the air, the IR sensors usually have much higher spatio-temporal resolution. Therefore, many recent rainfall algorithms utilize brightness temperatures from both PMW and IR sensors to achieve better rainfall estimation accuracy. Algorithms which combine both the advantages of PMW and IR were developed to estimate the rainfall [14], [15], [16], [17], [18], for example, the IR-dominated Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks(ANN) [16] and later the PERSIANN-Cloud Classification System (PERSIANN-CSS) [18], the National Oceanic and Atmospheric Administration's(NOAA) Climate Prediction Center(CPC) morphing technique(CMORPH) [14]. Studies by Dinku et al. [19] and [20] shows that the CMORPH and PERSIANN-CCS underestimated the rainfall by more than 30% in extreme precipitation events.
This study aimed to evaluate the performance of algorithms PERSIANN-CCS and CMORPH with a relatively dense rain gauge network over Beijing and the surrounding regions for the July 21, 2012 extreme precipitation event. The paper is organized as follows. Section 2 describes the study region and the evaluated rainfall products. Section 3 provides an analysis of spatial characteristics and error quantification for the rainfall from 00:00AM on July 21 to 00:00PM on July 22 2012. The total rainfall accumulation, the temporal variation of mean hourly rainfall, and rainfall contingency as a function of rainfall intensity are described in this section. A summary of analysis results and conclusions are provided in Section 4.
Study Region and Data
2.1 Study region
The study region, including Hebei province, Beijing and Tianjin, spans from 36°′N to 42.5°N in latitude and from 113.5°08′E to 120°E in longitude (Fig. 1). The east of study region terminates at the Bohai Sea, the western at the Taihang mountain ridge, and the northern at the Yanshan mountain ridge.
(b) Gauge distribution and topographic features in Beijing, Tianjin and Hebei province.
2.2 Data
Dense Automatic Weather Stations (AWS) gauge network observations from 2401 stations (Figure 1b) are used as the ‘ground truth’. The gauges are part of an enhanced dense gauge network over China, providing hourly real-time weather observations for authorized natural hazard monitoring departments and weather service sectors. An analysis by Ren [21] shows that the average rainfall error of gauge observations is 6.52% [32], which is much smaller than satellite-based precipitation estimates. The satellite-based quantitative precipitation estimates (QPE) products include hourly PERSIANN-CCS with a spatial resolution 0.04° and 3-hourly CMORPH with a spatial resolution 0.25°.
PERSIAN-CCS is an automated system for precipitation estimation from remotely sensed information using the artificial neural network, which extracts local and regional cloud features from infrared geostationary satellite imagery to estimate rainfall distribution. The system retrieves different rainfall rates at a given brightness temperature (Tb) and detects variable rain/no-rain IR thresholds for different cloud types [18], [22]. In this study, PERSIANN-CCS is resampled from 0.04° to 0.25° in order to compare with CMPORH at the same scale.
CMORPH has a quasi-global coverage (60°S-60°N in latitude) and is of very high-resolution (0.07278°, approximately 8 km, in latitude/longitude, half-hourly) [14]. Motion vectors derived from the half-hourly geostationary satellite IR imagery are used to extrapolate the passive microwave precipitation estimates. This technique improves the estimation of multi-hour precipitation accumulation, better than the simple averaging of available microwave-based estimates and other merging results that incorporate microwave and infrared information in the estimation [14]. Only the most recent one-month half-hourly/0.07278° CMORPH product could be publically accessed, with data during the July 21 Beijing flood event not available when this work was initiated in October of 2012. Therefore, the 3 hourly/0.25° CMORPH product was used together with PERSIANN-CCS to compare with the gauge observations.
2.3 Statistics metrics
Statistic metrics include Relative bias (RB), root-mean-squared error (RMSE), correlation coefficient (CC), probability of detection (POD), false alarm ratio (FAR) and critical success index (CSI) are used in this study to evaluate the performance of all the rainfall algorithms. These metrics are defined as follows.(1)
(2)
(3)where RB and CC are dimensionless, RMSE is in mm. In Eq. (3), “Cov ()” refers to the covariance, and σ indicates the standard deviation. RB, when multiplied by 100, denotes the degree of overestimation or underestimation in percentage. All the above statistics were computed for each pixel location within the study region.
Table 1 provides the contingency table to compute the number of hits (A), the false alarm (B), and miss rate (C). Statistics of POD, FAR and CSI were computed with the following equations [19]:(4)
(5)
(6)
Analysis of Rainfall Products
3.1 Accumulated rainfall
Rainfall estimates from all the products were accumulated from 00:00AM on 21 July 2012 to 00:00AM on 22 July 2012 to yield a two-day total rainfall as shown in Figure 2. The gauge daily observations were interpolated over the study domain based on Kriging interpolation skill, which is embedded in the Interactive Data Language (IDL, Version 8.2). The exponential model and default variogram model were used for the interpolation. The interpolated result (Figure 2(a)) indicates the maximum in accumulated rainfall extending from southwest of Beijing northeastward all the way to the adjacent Beijing and Hebei province, and eastward to center of Tianjin. CMORPH and PERSIANN-CCS misplaced this precipitation maximum. It is also worth noting that PERSIANN-CCS underestimated the rainfall by a wide margin, with maximum rainfall less than 200 mm.
(b) CMORPH accumulated precipitation (c) PERSIANN-CCS accumulated precipitation. (d) Scatter plots of gauge vs. CMORPH accumulated rainfall. (e) Scatter plot of gauge vs. PERSIANN-CCS accumulated rainfall. The red letter “A” in (a) indicates the rainfall center.
For quantitative comparison, only the grids that cover two and more rain gauges were used for comparison. Grids containing less than two gauges were discarded. Such grid-based comparison technique has been applied in many previous studies [23], [24], [25], [26], [27]. A total of 345 grids were available for direct comparison with a 0.25°×0.25° spatial resolution for the large region including Hebei, Beijing and Tianjin, and 30 grids for Beijing. Figure 2d–e show that both CMORPH and PERSIANN-CCS have low CC with gauge-based observations, which indicates CMORPH and PERSIANN-CCS failed to capture the spatial rainfall pattern. The index RB implies CMORPH overestimated the rainfall by about 6.14% but PERSIANN-CCS significantly underestimated the rainfall by 65.29%. Figure 3d indicates the cloud top in the region of highest accumulated precipitation was above 220 k except for the 11th hour. This suggests the IR-based QPE retrieve technique has inherent shortage that using IR brightness temperatures to estimate precipitation will yield a large error when the convective cloud has a higher brightness temperature than the commonly used temperature threshold [28], [29]. For CMORPH, the misplacement of the most intense rainfall might be due to: 1) Precipitation that is not observed, as it forms and dissipates over an area outside of passive microwave overpasses mosaics; or 2) Ice areas assigned zero rainfall estimates by the snow screening process in the CMORPH algorithm [14], [20]. This result is consistent with the findings in Hirpa et al. [30], Zhang [20] and Chen et al. [27] over complex terrain during heavy precipitation events.
(d) Time series mean brightness temperature at rainfall center shown in Figure 1d in the scale.
3.2 Precipitation time-series
Figure 3 shows time series plots of hourly area-average accumulated rainfall for gauge and satellite products for the large domain including Hebei, Beijing and Tianjin (HBT), and Beijing and the rainfall center indicated by the red letter A in Figure 2a. In Figure 3a, the gauge observed rainfall over HBT generally increased from the first hour to the peak occurring at the 20th hour and then decreased dramatically to less than 1 mm/h by the 24th hour. CMORPH captured the peak of rainfall and generally replicated the temporal evolution observed by the gauge. PERSIANN-CCS produced a peak in rainfall one hour ahead of the gauge but substantially underestimated the amounts throughout the event. The statistics show that CMORPH had a high CC of 0.92 and low RB (4.80%), while PERSIANN-CCS had a slightly lower CC (0.86) but a highly negative RB (−66.69%). For the Beijing area (Figure 3b), CMORPH and particularly PERSIANN-CCS failed to capture the rainfall peak and temporal variation, evidenced by the negative CC. Furthermore, CMORPH generally overestimated the rainfall, while PERISANN-CCS underestimated by −72.75%. For the rainfall maximum in the rainfall center (Figure 3c), CMORPH and PERSIANN-CCS performed poorly in capturing the rainfall peak and temporal variation, underestimating total accumulation by 42.26% and 85.37%, respectively.
It was noted that in the region of highest rainfall accumulation the gauge-observed very heavy rainfall (>15 mm/h) began at the 9th hour and lasted to the 15th hour. This rainfall intensity exceeded the lower bound of the rainfall intensity-duration threshold (Intensity = 12.45 mm/h Duration = 0.42 h) for landslides proposed by Hong et al. [31]. Several mountain torrents, floods, and landslides took place in these areas (http://www.youtube.com/watch?v=X1gzzUipevg; http://video.lnd.com.cn/htm/2012-07/25/content_2422846.htm), and 38 people were killed in Fangshan County, located in the rainfall maximum. Compared with gauge observation, the most intense rainfall observed by both CMORPH and PERISANN-CCS did not exceed 12.45 mm/h. Therefore, the landslide and debris would not have triggered if CMORPH and PERISANN-CCS were used as input to the landslide model proposed by Hong et al. [31].
3.3 Probability distributions
Information on the precipitation rate distribution, precipitation volume distribution, and the precipitation estimates' sensitivity as a function of precipitation rate may be revealed with Probability distribution functions (PDFs). PDFs offers insight into error dependence on precipitation rate, and the potential impact of this error on hydrological applications [32]. Figure 4 shows the hourly mean precipitation PDFs by occurrence (PDFc) and cumulative distribution of rainfall rate by volume (CDFv) for the gauge and satellite-based observations. Figure 4a shows that CMORPH had the best agreement with gauge observations in terms of hourly mean precipitation occurrence, while PERSIANN-CCS had a higher percentage of light rainfall rates (<1 mm/h), and low percentages for high rainfall rate (>5 mm/h). Figure 4b shows that more than 90% of the total rainfall accumulation derived from PERSIANN-CSS was contributed by rainfall rates less than 10 mm/h. The distributions for CMORPH and PERSIANN-CSS were similar for rainfall rates less than 22 mm/h; however, the gauge observations had more contribution from rainfall rates greater than 22 mm/h, especially relative to PERSIANN-CSS. This implies that CMORPH and PERSIANN-CCS could not resolve the intense rainfall rates associated with this event. This result is consistent with the results of Chen et al. [27].
3.4 Contingency information
Figure 5 provides contingency information in terms of POD, CSI and FAR. Figure 5a shows PERSIANN-CCS has low POD and CSI for rainfall rates greater than 15 mm/h, while CMORPH has low POD and CSI when rainfall rates exceed 25 mm/h. This result provides further evidence that PERSIANN-CCS and CMORPH generally perform poorly in the detection of very high rainfall rates, and is consistent with information revealed by the PDFs shown in Figure 4.
The rainfall threshold used is from 1 to 30 with interval of 1/h.
Summary and Conclusions
The primary objective of this study was to quantify the performance of the popular satellite-based QPE products CMORPH and PERSIANN-CCS for an extreme precipitation event over Beijing and surrounding regions, using a dense gauge network as ground truth. The satellite-based QPE products have been evaluated in terms of accumulated rainfall, time series rainfall, probability distribution and contingency metrics. Our results identified the following:
- Both CMORPH and PERSIANN-CCS were unable to capture the spatial rainfall pattern (CC<0.6, Figure 2) and misplaced the region of greatest precipitation intensity/accumulation.
- CMORPH captured the temporal variation of rainfall with high CC (0.92) over the large area HBT but could not resolve the temporal variation of rainfall over Beijing and the rainfall maximum (Figure 3a–c).
- PERSIANN-CCS underestimated the rainfall by more than 60% over the large area HBT, Beijing and rainfall center, and failed to resolve the temporal variation (Figure 2d–e, Figure 3a–c).
- PERSIANN-CCS was unable to detect high rainfall rate (>15 mm/h), and CMORPH had low probabilities of high rainfall rate (>25 mm/h) (Figure 4, 5).
This study suggests that satellite-based QPE products CMORPH and PERSIANN-CCS demonstrated poor performance in very heavy rainfall events, similar results can be found in studies of Dinku et al. (2010) [19], and Zhang (2012) as well as Chen et al. [27]. These two products tend to have limitations in terms of resolution and accuracy, especially for this type of extreme mid-latitude precipitation. Caution should be applied when CMORPH and PERSIANN-CCS are utilized for hydrological modeling and natural hazards (e.g. landslide) monitoring, because the data used to drive risk model cascades often form the dominant source of uncertainty within such model systems [33], [34]. However, we can envision that future Satellite-based QPE algorithms will be improved for hydrological and meteorological applications such as flood and landslide monitoring and forecasting, notably in the expected launch of Global Precipitation Measuring (GPM) mission in 2014 with dual-frequency radar onboard and better spatiotemporal coverage over the globe.
Acknowledgments
Thanks are given to Dr. Yudong Tian of NASA's Goddard Flight Center for his help in data processing for this study.
Author Contributions
Conceived and designed the experiments: SC. Performed the experiments: SC HL. Analyzed the data: HL Y. Yuan MH LH YL XZ. Wrote the paper: SC HL Y. You EM JH Y. Yuan MH LH YL XZ GT YH.
References
- 1. Zhang D-L, Lin Y, Zhao P, Yu X, Wang S, et al. (2013) The Beijing extreme rainfall of 21 July 2012: “Right results” but for wrong reasons. Geophysical Research Letters
- 2. Wang K, Wang L, Wei Y-M, Ye M (2013) Beijing storm of July 21, 2012: observations and reflections. Natural Hazards 1–6.
- 3.
Grumm RH (2012) Beijing flood of 21 July 2012. Available: http://nws.met.psu.edu/severe/2012/21Jul2012.pdf. Accessed: 30 Nov 2012.
- 4. Sang YF, Wang Z, Liu C (2012) What factors are responsible for the Beijing storm? Natural Hazards 1–2.
- 5. Change IPOC (2007) Climate change 2007: The physical science basis. Agenda 6: 07.
- 6. Creutin JD, Borga M (2003) Radar hydrology modifies the monitoring of flash-flood hazard. Hydrological processes 17: 1453–1456.
- 7. Hong Y, Adler R (2007) Towards an early-warning system for global landslides triggered by rainfall and earthquake. International journal of remote sensing 28: 3713–3719.
- 8. Hong Y, Adler RF, Huffman G (2007) An experimental global prediction system for rainfall-triggered landslides using satellite remote sensing and geospatial datasets. Geoscience and Remote Sensing, IEEE Transactions on 45: 1671–1680.
- 9. Wang J, Hong Y, Li L, Gourley JJ, Khan SI, et al. (2011) The coupled routing and excess storage (CREST) distributed hydrological model. Hydrological Sciences Journal 56: 84–98.
- 10. Li L, Ngongondo CS, Xu CY, Gong L (2012) Comparison of the global TRMM and WFD precipitation datasets in driving a large-scale hydrological model in southern Africa.
- 11. Barrett E, Beaumont M (1994) Satellite rainfall monitoring: An overview. Remote Sensing Reviews 11: 23–48.
- 12. You Y, Liu G, Wang Y, Cao J (2011) On the sensitivity of Tropical Rainfall Measuring Mission (TRMM) Microwave Imager channels to overland rainfall. Journal of Geophysical Research 116: D12203.
- 13. Kummerow C, Olson WS, Giglio L (1996) A simplified scheme for obtaining precipitation and vertical hydrometeor profiles from passive microwave sensors. Geoscience and Remote Sensing, IEEE Transactions on 34: 1213–1232.
- 14. Joyce RJ, Janowiak JE, Arkin PA, Xie P (2004) CMORPH: A method that produces global precipitation estimates from passive microwave and infrared data at high spatial and temporal resolution. Journal of Hydrometeorology 5: 487–503.
- 15. Hsu K, Gao X, Sorooshian S, Gupta HV (1997) Precipitation estimation from remotely sensed information using artificial neural networks. Journal of Applied Meteorology 36: 1176–1190.
- 16. Sorooshian S, Hsu KL, Xiaogang G, Gupta HV, Imam B, et al. (2000) Evaluation of PERSIANN system satellite-based estimates of tropical rainfall. Bulletin of the American Meteorological Society 81: 2035–2046.
- 17. Huffman GJ, Bolvin DT (2007) TRMM and other data precipitation data set documentation. Laboratory for Atmospheres, NASA Goddard Space Flight Center and Science Systems and Applications
- 18. Hong Y, Hsu KL, Sorooshian S, Gao X (2004) Precipitation estimation from remotely sensed imagery using an artificial neural network cloud classification system. Journal of Applied Meteorology 43: 1834–1853.
- 19.
Dinku T, Connor SJ, Ceccato P (2010) Comparison of CMORPH and TRMM-3B42 over mountainous regions of Africa and South America: New York: Springer-Verlag.
- 20. Zhang X (2012) Using NWP Analysis in Satellite Rainfall Estimation of Heavy Precipitation Events over Complex Terrain: University of Connecticut.
- 21. Ren Z, Wang G, Zou F, Zhang H (2003) The Research of Precipitation Measurement Errors in China. Acta Meteorologica Sinica 61: 621–627.
- 22. Gourley JJ, Hong Y, Flamig ZL, Li L, Wang J (2010) Intercomparison of rainfall estimates from radar, satellite, gauge, and combinations for a season of record rainfall. Journal of Applied Meteorology and Climatology 49: 437–452.
- 23. Adler RF, Huffman GJ, Chang A, Ferraro R, Xie PP, et al. (2003) The version-2 global precipitation climatology project (GPCP) monthly precipitation analysis (1979-present). Journal of Hydrometeorology 4: 1147–1167.
- 24. Chiu LS, Liu Z, Vongsaard J, Morain S, Budge A, et al. (2006) Comparison of TRMM and water district rain rates over New Mexico. Advances in Atmospheric Sciences 23: 1–13.
- 25. Chokngamwong R, Chiu LS (2008) Thailand daily rainfall and comparison with TRMM products. Journal of Hydrometeorology 9: 256–266.
- 26. Nicholson SE, Some B, McCollum J, Nelkin E, Klotter D, et al. (2003) Validation of TRMM and other rainfall estimates with a high-density gauge dataset for West Africa. Part II: Validation of TRMM rainfall products. Journal of Applied Meteorology 42: 1355–1368.
- 27. Chen S, Hong Y, Cao Q, Kirstetter PE, Gourley JJ, et al. (2012) Performance Evaluation of Radar and Satellite Rainfalls for Typhoon Morakot over Taiwan: Are Remote-sensing Products Ready for Gauge Denial Scenario of Extreme Events? Journal of Hydrology
- 28. Tuttle JD, Carbone RE, Arkin PA (2008) Comparison of ground-based radar and geosynchronous satellite climatologies of warm-season precipitation over the United States. Journal of Applied Meteorology and Climatology 47: 3264–3270.
- 29. Negri AJ, Adler RF (1993) An intercomparison of three satellite infrared rainfall techniques over Japan and surrounding waters. Journal of Applied Meteorology 32: 357–373.
- 30. Hirpa FA, Gebremichael M, Hopson T (2010) Evaluation of high-resolution satellite precipitation products over very complex terrain in Ethiopia. Journal of Applied Meteorology and Climatology 49: 1044–1051.
- 31. Hong Y, Adler RF, Negri A, Huffman GJ (2007) Flood and landslide applications of near real-time satellite rainfall products. Natural Hazards 43: 285–294.
- 32. Tian Y, Peters-Lidard CD, Adler RF, Kubota T, Ushio T (2010) Evaluation of GSMaP precipitation estimates over the contiguous United States. Journal of Hydrometeorology 11: 566–574.
- 33. Kay A, Davies H, Bell V, Jones R (2009) Comparison of uncertainty sources for climate change impacts: flood frequency in England. Climatic Change 92: 41–63.
- 34. McMillan HK, Brasington J (2008) End-to-end flood risk assessment: A coupled model cascade with uncertainty estimation. Water Resources Research 44: W03419.