Add new bucket utilities

.github/ISSUE_TEMPLATE/bug_report.yaml

@@ -41,7 +41,7 @@ body:
       description: |
         examples:
           - **OS**: Ubuntu 20.04
-          - **Python**: 3.8.2
+          - **Python**: 3.11.2
       value: |
         - OS:
         - python:

.github/workflows/lint.yaml

@@ -8,7 +8,7 @@ jobs:
       - uses: actions/checkout@v4
       - uses: actions/setup-python@v5
         with:
-          python-version: "3.8"
+          python-version: "3.10"
       - name: Ruff
         if: success() || failure()
         uses: pre-commit/[email protected]

.github/workflows/tests.yaml

@@ -14,11 +14,11 @@ jobs:
       fail-fast: false
       matrix:
         os: [ubuntu-latest]
-        python-version: ["3.9", "3.10", "3.11", "3.12"]
+        python-version: ["3.10", "3.11", "3.12", "3.13"]
         experimental: [false]
         include:
           - os: ubuntu-latest
-            python-version: "3.12"
+            python-version: "3.13"
             experimental: true
           - os: ubuntu-latest
             python-version: "3.11"

.github/workflows/tests_windows.yaml

@@ -14,7 +14,7 @@ jobs:
       fail-fast: false
       matrix:
         os: [windows-latest]
-        python-version: ["3.9", "3.10", "3.11", "3.12"]
+        python-version: ["3.10", "3.11", "3.12", "3.13"]
         experimental: [false]
     steps:
       - uses: actions/checkout@v4

CODE_OF_CONDUCT.md

@@ -0,0 +1,130 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socioeconomic status,
+nationality, personal appearance, race, caste, color, religion, or sexual
+identity and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+- Demonstrating empathy and kindness toward other people
+- Being respectful of differing opinions, viewpoints, and experiences
+- Giving and gracefully accepting constructive feedback
+- Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+- Focusing on what is best not just for us as individuals, but for the overall community
+
+Examples of unacceptable behavior include:
+
+- The use of sexualized language or imagery, and sexual attention or advances of any kind
+- Trolling, insulting or derogatory comments, and personal or political attacks
+- Public or private harassment
+- Publishing others' private information, such as a physical or email address,
+  without their explicit permission
+- Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official e-mail address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+\[INSERT CONTACT METHOD\].
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series of
+actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or permanent
+ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior, harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within the
+community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.1, available at
+[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
+
+Community Impact Guidelines were inspired by
+[Mozilla's code of conduct enforcement ladder][mozilla coc].
+
+For answers to common questions about this code of conduct, see the FAQ at
+[https://www.contributor-covenant.org/faq][faq]. Translations are available at
+[https://www.contributor-covenant.org/translations][translations].
+
+[faq]: https://www.contributor-covenant.org/faq
+[homepage]: https://www.contributor-covenant.org
+[mozilla coc]: https://github.com/mozilla/diversity
+[translations]: https://www.contributor-covenant.org/translations
+[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html

README.md

@@ -4,7 +4,7 @@
 | ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | Deployment        | [![PyPI](https://badge.fury.io/py/gpm_api.svg?style=flat)](https://pypi.org/project/gpm_api/) [![Conda](https://img.shields.io/conda/vn/conda-forge/gpm-api.svg?logo=conda-forge&logoColor=white&style=flat)](https://anaconda.org/conda-forge/gpm-api)                                                                                                                                                                                                                                                                                                                                                                                                                       |
 | Activity          | [![PyPI Downloads](https://img.shields.io/pypi/dm/gpm_api.svg?label=PyPI%20downloads&style=flat)](https://pypi.org/project/gpm_api/) [![Conda Downloads](https://img.shields.io/conda/dn/conda-forge/gpm-api.svg?label=Conda%20downloads&style=flat)](https://anaconda.org/conda-forge/gpm-api)                                                                                                                                                                                                                                                                                                                                                                               |
-| Python Versions   | [![Python Versions](https://img.shields.io/badge/Python-3.9%20%203.10%20%203.11%20%203.12-blue?style=flat)](https://www.python.org/downloads/)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
+| Python Versions   | [![Python Versions](https://img.shields.io/badge/Python-3.10%20%203.11%20%203.12%20%203.13-blue?style=flat)](https://www.python.org/downloads/)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
 | Supported Systems | [![Linux](https://img.shields.io/github/actions/workflow/status/ghiggi/gpm_api/.github/workflows/tests.yaml?label=Linux&style=flat)](https://github.com/ghiggi/gpm_api/actions/workflows/tests.yaml) [![macOS](https://img.shields.io/github/actions/workflow/status/ghiggi/gpm_api/.github/workflows/tests.yaml?label=macOS&style=flat)](https://github.com/ghiggi/gpm_api/actions/workflows/tests.yaml) [![Windows](https://img.shields.io/github/actions/workflow/status/ghiggi/gpm_api/.github/workflows/tests_windows.yaml?label=Windows&style=flat)](https://github.com/ghiggi/gpm_api/actions/workflows/tests_windows.yaml)                                            |
 | Project Status    | [![Project Status](https://www.repostatus.org/badges/latest/active.svg?style=flat)](https://www.repostatus.org/#active)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
 | Build Status      | [![Tests](https://github.com/ghiggi/gpm_api/actions/workflows/tests.yaml/badge.svg?style=flat)](https://github.com/ghiggi/gpm_api/actions/workflows/tests.yaml) [![Lint](https://github.com/ghiggi/gpm_api/actions/workflows/lint.yaml/badge.svg?style=flat)](https://github.com/ghiggi/gpm_api/actions/workflows/lint.yaml) [![Docs](https://readthedocs.org/projects/gpm_api/badge/?version=latest&style=flat)](https://gpm-api.readthedocs.io/en/latest/)                                                                                                                                                                                                                  |
@@ -95,7 +95,9 @@ A GPM granule can be opened in python using:
 ```python
 import gpm
 
-ds = gpm.open_granule(<path_to_granule>)
+ds = gpm.open_granule_dataset(<path_to_granule>)
+# or
+dt = gpm.open_granule_datatree(<path_to_granule>)
 
 ```
 

ci/environment.yaml

@@ -31,7 +31,6 @@ dependencies:
   - trame-vtk
   - trame-vuetify
   - trollsift
-  - xarray-datatree
-  - xarray
+  - xarray>=2025.01.0
   - xoak
   - ximage

ci/environment_latest.yaml

@@ -31,6 +31,5 @@ dependencies:
   - trame-vuetify
   - trollsift
   - xarray
-  - xarray-datatree
   - xoak
   - ximage

docs/environment.yaml

@@ -8,7 +8,7 @@ dependencies:
   - pandoc
   - pip
   - polars
-  - python>=3.9
+  - python>=3.10
   - pyvista
   - sphinx-book-theme==1.1.0
   - sphinx-gallery

docs/source/00_introduction.rst

@@ -586,6 +586,7 @@ IMERG Precipitation
 The Integrated Multi-satellite Retrievals for GPM (IMERG) is an advanced algorithm designed
 to generate a high resolution precipitation product every 30 minutes, covering the full globe (up to version 6,  a quasi-global area from 60°S to 60°N)
 with a spatial resolution of 0.1° x 0.1°.
+The latest IMERG product covers the time period from January 1998 to present.
 
 By leveraging measurements from Geostationary (GEO) IR imagers and the :ref:`GPM PMW sensors constellation <gpm_constellation>`,
 IMERG provides a "best-estimate" of 30-minute average precipitation rates.
@@ -606,8 +607,11 @@ While IMERG Early uses only forward propagation in time (extrapolation in time),
 
 When PMW data are too sparse, IMERG incorporates precipitation estimates derived from GEO IR imagery through a weighted Kalman filter.
 
-IR-based precipitation estimates are derived from the
-`Climate Prediction Center (CPC) Merged 4-km Global IR data product <https://disc.gsfc.nasa.gov/datasets/GPM_MERGIR_1/summary>`_.
+IR-based precipitation estimates are derived from the NOAA Climate Data Record (CDR) of Gridded Satellite Data from the ISCCP B1 (GridSat-B1) IR dataset for
+all timesteps between January 1998 and February 7, 2002, 20:00:00 UTC.
+Starting from February 7, 2002, 20:00:00 UTC, IR-based precipitation estimates
+are obtained from the `NOAA Climate Prediction Center (CPC) Merged 4-km Global IR data product <https://disc.gsfc.nasa.gov/datasets/GPM_MERGIR_1/summary>`_.
+
 This dataset composites infrared (IR) brightness temperature measurements from numerous geostationary sensors over their operational periods,
 including European (METEOSAT-5/7/8/9/10/11), Japanese (GMS-5, MTSat-1R/2, Himawari-8/9), and U.S. (GOES-8/9/10/11/12/13/14/15/16/17/18) satellites,
 every 30 minutes between 60°N/S.
@@ -931,10 +935,14 @@ Please also note that on Google Earth Engine are available the version 6 of `GSM
 and `IMERG <https://developers.google.com/earth-engine/datasets/catalog/NASA_GPM_L3_IMERG_V06>`_.
 
 GSMaP can be visualized on the `JAXA Global Rainfall Watch <https://sharaku.eorc.jaxa.jp/GSMaP/index.htm>`_,
-while IMERG on the `GPM IMERG Global Viewer <https://gpm.nasa.gov/data/visualization/global-viewer>`_ and the `EOSDIS WorldView Portal <https://worldview.earthdata.nasa.gov/?v=-235.13866988428558,-76.35016978404038,104.5800850894752,96.99821113230026&l=Reference_Labels_15m(hidden),Reference_Features_15m(hidden),Coastlines_15m,IMERG_Precipitation_Rate,VIIRS_NOAA20_CorrectedReflectance_TrueColor(hidden),VIIRS_SNPP_CorrectedReflectance_TrueColor(hidden),MODIS_Aqua_CorrectedReflectance_TrueColor(hidden),MODIS_Terra_CorrectedReflectance_TrueColor&lg=true&t=2024-02-08-T03%3A43%3A10Z>`_.
+while IMERG on
+`the RAIN-Global Viewer (Regional Animations of IMERG in Near-realtime - Global Edition) <https://storm.pps.eosdis.nasa.gov/storm/outreach/RAIN-Global.html>`_ ,
+the `GPM IMERG Global Viewer <https://gpm.nasa.gov/data/visualization/global-viewer>`_ and
+the `EOSDIS WorldView Portal <https://worldview.earthdata.nasa.gov/?v=-235.13866988428558,-76.35016978404038,104.5800850894752,96.99821113230026&l=Reference_Labels_15m(hidden),Reference_Features_15m(hidden),Coastlines_15m,IMERG_Precipitation_Rate,VIIRS_NOAA20_CorrectedReflectance_TrueColor(hidden),VIIRS_SNPP_CorrectedReflectance_TrueColor(hidden),MODIS_Aqua_CorrectedReflectance_TrueColor(hidden),MODIS_Terra_CorrectedReflectance_TrueColor&lg=true&t=2024-02-08-T03%3A43%3A10Z>`_.
 
 The `GES DISC Interactive Online Visualization ANd aNalysis Infrastructure (Giovanni) <https://giovanni.gsfc.nasa.gov/giovanni/>`_ also provides quick access to analysis of IMERG products.
 
+
 .. _useful_resources:
 
 Useful Resources

docs/source/02_installation.rst

@@ -286,7 +286,7 @@ To perform spaceborne-ground radar calibration and validation, install
 
 .. code-block:: bash
 
-   conda install -c conda-forge xradar wradlib pyart
+   conda install -c conda-forge xradar wradlib arm_pyart
 
 
 Run GPM-API on Jupyter Notebooks

docs/source/03_quickstart.rst

@@ -238,11 +238,17 @@ Open the data
 Within the GPM-API, the name *granule* is used to refer to a single file,
 while the name *dataset* is used to refer to a collection of granules.
 
-GPM-API enables to open single or multiple granules into an ``xarray.Dataset``, an object designed for working with labeled multi-dimensional arrays.
+GPM-API enables to open single or multiple granules into ``xarray.Dataset`` or ``xarray.DataTree`` objects,
+which are designed for working with labeled multi-dimensional arrays.
 
-The ``gpm.open_granule(filepath)`` opens a single file into xarray by providing the path of the file of interest.
+The ``gpm.open_granule_dataset(filepath)`` opens a single file into a ``xarray.Dataset`` object
+by providing the path of the file (granule) of interest. This function open a single sensor ``scan_mode``.
 
-The ``gpm.open_dataset`` function enables to open a collection of granules over a period of interest.
+The ``gpm.open_granule_datatree(filepath)`` opens a single file into a ``xarray.DataTree`` object
+by providing the path of the file (granule) of interest. This function open all sensors ``scan_modes``.
+
+The ``gpm.open_dataset`` and ``gpm.open_datatree`` functions enables to open a collection of granules
+ over a period of interest.
 
 The following example shows how to download and open a dataset over a specific time period:
 
@@ -281,6 +287,19 @@ The following example shows how to download and open a dataset over a specific t
     # Plot a specific variable of the dataset
     ds["precipRateNearSurface"].gpm.plot_map()
 
+    # As alternative you can open all scan modes into a datatree
+    dt = gpm.open_datatree(
+        product=product,
+        product_type=product_type,
+        version=version,
+        start_time=start_time,
+        end_time=end_time,
+    )
+    # Then retrieve the dataset from the wished scan mode and plot
+    ds = dt["FS"].to_dataset()
+    ds["precipRateNearSurface"].gpm.plot_map()
+
+
 
 You are now ready to explore the various :ref:`tutorials <tutorials>` available in the documentation and learn more about the GPM-API functionalities.
 

gpm/__init__.py

@@ -40,9 +40,9 @@
     define_configs,
     read_configs,
 )
-from gpm.dataset.dataset import open_dataset  # noqa
-from gpm.dataset.datatree import open_datatree  # noqa
-from gpm.dataset.granule import open_granule  # noqa
+from gpm.dataset.dataset import open_dataset, open_datatree  # noqa
+from gpm.dataset.datatree import open_raw_datatree  # noqa
+from gpm.dataset.granule import open_granule, open_granule_dataset, open_granule_datatree  # noqa
 from gpm.io.download import download_archive as download  # noqa
 from gpm.io.download import (  # noqa
     download_daily_data,
@@ -67,6 +67,7 @@
     check_regular_time,
     check_valid_geolocation,
 )
+from gpm.utils.pmw import available_pmw_frequencies  # noqa
 
 # from gpm.utils.manipulations import convert_from_decibel as idecibel
 # from gpm.utils.manipulations import convert_to_decibel as decibel