GPM-API: A Python Interface to Access the Global Precipitation Measurement Mission Satellites Open Data Archive
Category
Contribute
Institutions
EPFL
Data type
Satellite data
Field
Large-scale hydrology
Researchers
Gionata Ghiggi
Description
The Global Precipitation Measurement Mission (GPM) data archive currently sizes several petabytes and includes satellite data records that extend back to 1987. This extensive archive is the result of contributions from two spaceborne radars and a fleet of 35 passive microwave (PMW) sensors that form the so-called GPM constellation. The data are organized into various product levels, encompassing raw and calibrated observations (Level 1), intermediate geophysical retrieval products (Level 2), and integrated datasets from multiple satellites (Level 3). The GPM-API is a Python package designed to ease the challenges of downloading, processing, analyzing, and visualizing the vast and diverse data included within the GPM data archive. The software has been designed to drastically increase scientist productivity, accelerate model prototyping and development as well as foster research reproducibility. Within the context of the ORD project we finalized the software to provide a seamless and user-friendly programming interface to (1) selectively download GPM products (2) efficiently locate these files on local storage, (3) readily access products into xarray for analysis (4) perform efficient on-disk and distributed data processing with Dask, (5) visualize product variables according to their specific characteristics, (6) identify precipitation events and phenomena within the dataset, and (7) extract spatial and temporal slices for the development of new retrieval algorithms or the efficient training of machine learning models. Additionally, the GPM-API facilitates the conversion of the original datasets into Climate and Forecast (CF) conventions compliant analysis-ready cloud-optimized (ARCO) Zarr format, with significant benefits in terms of storage requirements and read access performance. All package functionalities have been rigorously tested and extensively documented to enhance software reliability, usability and facilitate future maintenance. By increasing the accessibility of the GPM data archive, GPM-API democratizes access to global precipitation data for research, education and practical applications, as well as promotes collaboration within the scientific community. The GPM-API software is accessible at https://github.com/ghiggi/gpm_api.