Getting started

The xdatasets library enables users to effortlessly access a vast collection of earth observation datasets that are compatible with xarray formats.

The library adopts an opinionated approach to data querying and caters to the specific needs of certain user groups, such as hydrologists, climate scientists, and engineers. One of the functionalities of xdatasets is the ability to extract data at a specific location or within a designated region, such as a watershed or municipality, while also enabling spatial and temporal operations.

To use xdatasets, users must employ a query. For instance, a straightforward query to extract the variables t2m (2m temperature) and tp (Total precipitation) from the era5_reanalysis_single_levels dataset at two geographical positions (Montreal and Toronto) could be as follows:

query = {
    "datasets": {"era5_reanalysis_single_levels": {'variables': ["t2m", "tp"]}},
    "space": {
        "clip": "point", # bbox, point or polygon
        "geometry": {'Montreal' : (45.508888, -73.561668),
                     'Toronto' : (43.651070, -79.347015)
                    }
    }
}

An example of a more complex query would look like the one below.

Note Don’t worry! Below, you’ll find additional examples that will assist in understanding each parameter in the query, as well as the possible combinations.

This query calls the same variables as above. However, instead of specifying geographical positions, a GeoPandas.DataFrame is used to provide features (such as shapefiles or geojson) for extracting data within each of them. Each polygon is identified using the unique identifier Station, and a spatial average is computed within each one (aggregation: True). The dataset, initially at an hourly time step, is converted into a daily time step while applying one or more temporal aggregations for each variable as prescribed in the query. xdatasets ultimately returns the dataset for the specified date range and time zone.

query = {
    "datasets": {"era5_reanalysis_single_levels": {'variables': ["t2m", "tp"]}},
    "space": {
        "clip": "polygon", # bbox, point or polygon
        "averaging": True, # spatial average of the variables within each polygon
        "geometry": gdf,
        "unique_id": "Station" # unique column name in geodataframe
    },
    "time": {
        "timestep": "D",
        "aggregation": {"tp": np.nansum,
                        "t2m": [np.nanmax, np.nanmin]},

        "start": '2000-01-01',
        "end": '2020-05-31',
        "timezone": 'America/Montreal',
    },
}

Query climate datasets

In order to use xdatasets, you must import at least xdatasets, pandas, geopandas, and numpy. Additionally, we import pathlib to interact with files.

import os

os.environ["USE_PYGEOS"] = "0"
from pathlib import Path

import geopandas as gpd

# Visualization
import hvplot.pandas  # noqa
import hvplot.xarray  # noqa-
import numpy as np
import pandas as pd
import panel as pn  # noqa

import xdatasets as xd

/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/clisops/core/regrid.py:42: UserWarning: xarray version >= 2023.03.0 is not supported for regridding operations with cf-time indexed arrays. Please use xarray version < 2023.03.0. For more information, see: https://github.com/pydata/xarray/issues/7794.
  warnings.warn(
/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

Clip by points (sites)

To begin with, we need to create a dictionary of sites and their corresponding geographical coordinates.

sites = {
    "Montreal": (45.508888, -73.561668),
    "New York": (40.730610, -73.935242),
    "Miami": (25.761681, -80.191788),
}

We will then extract the tp (total precipitation) and t2m (2m temperature) from the era5_reanalysis_single_levels dataset for the designated sites. Afterward, we will convert the time step to daily and adjust the timezone to Eastern Time. Finally, we will limit the temporal interval.

Before proceeding with this first query, let’s quickly outline the role of each parameter:

• datasets: A dictionary where datasets serve as keys and desired variables as values.

• space: A dictionary that defines the necessary spatial operations to apply on user-supplied geographic features.

• time: A dictionary that defines the necessary temporal operations to apply on the datasets

For more information on each parameter, consult the API documentation.

This is what the requested query looks like :

query = {
    "datasets": "era5_reanalysis_single_levels",
    "space": {"clip": "point", "geometry": sites},  # bbox, point or polygon
    "time": {
        "timestep": "D",  # http://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#offset-aliases
        "aggregation": {"tp": np.nansum, "t2m": np.nanmean},
        "start": "1959-01-01",
        "timezone": "America/Montreal",
    },
}
xds = xd.Query(**query)

Temporal operations: processing tp with era5_reanalysis_single_levels: 100%|██████████| 2/2 [00:05<00:00,  2.78s/it]

By accessing the data attribute, you can view the data obtained from the query. It’s worth noting that the variable name tp has been updated to tp_nansum to reflect the reduction operation (np.nansum) that was utilized to convert the time step from hourly to daily. Likewise, t2m was updated to t2m_nanmean.

xds.data

<xarray.Dataset>
Dimensions:      (spatial_agg: 1, timestep: 1, site: 3, time: 23621, source: 1)
Coordinates:
  * spatial_agg  (spatial_agg) object 'point'
  * timestep     (timestep) object 'D'
    latitude     (site) float64 45.5 40.75 25.75
    longitude    (site) float64 -73.5 -74.0 -80.25
  * site         (site) <U8 'Montreal' 'New York' 'Miami'
  * time         (time) datetime64[ns] 1959-01-01 1959-01-02 ... 2023-09-02
  * source       (source) <U29 'era5_reanalysis_single_levels'
Data variables:
    t2m_nanmean  (spatial_agg, timestep, time, site) float32 260.7 ... 301.0
    tp_nansum    (spatial_agg, timestep, time, site) float32 0.0005895 ... 0....
Attributes: (12/30)
    GRIB_NV:                                  0
    GRIB_Nx:                                  1440
    GRIB_Ny:                                  721
    GRIB_cfName:                              unknown
    GRIB_cfVarName:                           t2m
    GRIB_dataType:                            an
    ...                                       ...
    GRIB_totalNumber:                         0
    GRIB_typeOfLevel:                         surface
    GRIB_units:                               K
    long_name:                                2 metre temperature
    standard_name:                            unknown
    units:                                    K

xarray.Dataset

• Dimensions:
  • spatial_agg: 1
  • timestep: 1
  • site: 3
  • time: 23621
  • source: 1
• Coordinates: (7)
  • spatial_agg
    (spatial_agg)
    object
    'point'

    array(['point'], dtype=object)

  • timestep
    (timestep)
    object
    'D'

    array(['D'], dtype=object)

  • latitude
    (site)
    float64
    45.5 40.75 25.75

    long_name :

    latitude

    standard_name :

    latitude

    stored_direction :

    decreasing

    units :

    degrees_north

    array([45.5 , 40.75, 25.75])

  • longitude
    (site)
    float64
    -73.5 -74.0 -80.25

    long_name :

    longitude

    standard_name :

    longitude

    units :

    degrees_east

    array([-73.5 , -74.  , -80.25])

  • site
    (site)
    <U8
    'Montreal' 'New York' 'Miami'

    cf_role :

    timeseries_id

    array(['Montreal', 'New York', 'Miami'], dtype='<U8')

  • time
    (time)
    datetime64[ns]
    1959-01-01 ... 2023-09-02

    array(['1959-01-01T00:00:00.000000000', '1959-01-02T00:00:00.000000000',
           '1959-01-03T00:00:00.000000000', ..., '2023-08-31T00:00:00.000000000',
           '2023-09-01T00:00:00.000000000', '2023-09-02T00:00:00.000000000'],
          dtype='datetime64[ns]')

  • source
    (source)
    <U29
    'era5_reanalysis_single_levels'

    array(['era5_reanalysis_single_levels'], dtype='<U29')

• Data variables: (2)
  • t2m_nanmean
    (spatial_agg, timestep, time, site)
    float32
    260.7 272.3 297.4 ... 293.4 301.0

    GRIB_NV :

    0

    GRIB_Nx :

    1440

    GRIB_Ny :

    721

    GRIB_cfName :

    unknown

    GRIB_cfVarName :

    t2m

    GRIB_dataType :

    an

    GRIB_gridDefinitionDescription :

    Latitude/Longitude Grid

    GRIB_gridType :

    regular_ll

    GRIB_iDirectionIncrementInDegrees :

    0.25

    GRIB_iScansNegatively :

    0

    GRIB_jDirectionIncrementInDegrees :

    0.25

    GRIB_jPointsAreConsecutive :

    0

    GRIB_jScansPositively :

    0

    GRIB_latitudeOfFirstGridPointInDegrees :

    90.0

    GRIB_latitudeOfLastGridPointInDegrees :

    -90.0

    GRIB_longitudeOfFirstGridPointInDegrees :

    -180.0

    GRIB_longitudeOfLastGridPointInDegrees :

    179.75

    GRIB_missingValue :

    3.4028234663852886e+38

    GRIB_name :

    2 metre temperature

    GRIB_numberOfPoints :

    1038240

    GRIB_paramId :

    167

    GRIB_shortName :

    2t

    GRIB_stepType :

    instant

    GRIB_stepUnits :

    1

    GRIB_totalNumber :

    0

    GRIB_typeOfLevel :

    surface

    GRIB_units :

    K

    long_name :

    2 metre temperature

    standard_name :

    unknown

    units :

    K

    array([[[[260.7333 , 272.34067, 297.35284],
             [273.03668, 278.3229 , 297.80728],
             [274.65717, 274.93417, 297.66565],
             ...,
             [288.21964, 294.25046, 302.4256 ],
             [289.48007, 292.71133, 301.95346],
             [291.96652, 293.4351 , 300.99768]]]], dtype=float32)

  • tp_nansum
    (spatial_agg, timestep, time, site)
    float32
    0.0005895 0.002121 ... 0.0 0.003412

    long_name :

    Total precipitation

    units :

    m

    array([[[[5.8954954e-04, 2.1205675e-03, 4.0095672e-04],
             [1.6930103e-02, 1.8317699e-02, 5.2976608e-04],
             [2.9563904e-05, 0.0000000e+00, 1.6922951e-03],
             ...,
             [4.7683716e-07, 0.0000000e+00, 1.2983322e-02],
             [0.0000000e+00, 0.0000000e+00, 5.9204102e-03],
             [0.0000000e+00, 0.0000000e+00, 3.4122467e-03]]]], dtype=float32)

• Indexes: (5)
  • spatial_agg
    PandasIndex

    PandasIndex(Index(['point'], dtype='object', name='spatial_agg'))

  • timestep
    PandasIndex

    PandasIndex(Index(['D'], dtype='object', name='timestep'))

  • site
    PandasIndex

    PandasIndex(Index(['Montreal', 'New York', 'Miami'], dtype='object', name='site'))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['1959-01-01', '1959-01-02', '1959-01-03', '1959-01-04',
                   '1959-01-05', '1959-01-06', '1959-01-07', '1959-01-08',
                   '1959-01-09', '1959-01-10',
                   ...
                   '2023-08-24', '2023-08-25', '2023-08-26', '2023-08-27',
                   '2023-08-28', '2023-08-29', '2023-08-30', '2023-08-31',
                   '2023-09-01', '2023-09-02'],
                  dtype='datetime64[ns]', name='time', length=23621, freq='D'))

  • source
    PandasIndex

    PandasIndex(Index(['era5_reanalysis_single_levels'], dtype='object', name='source'))

• Attributes: (30)

  GRIB_NV :

  0

  GRIB_Nx :

  1440

  GRIB_Ny :

  721

  GRIB_cfName :

  unknown

  GRIB_cfVarName :

  t2m

  GRIB_dataType :

  an

  GRIB_gridDefinitionDescription :

  Latitude/Longitude Grid

  GRIB_gridType :

  regular_ll

  GRIB_iDirectionIncrementInDegrees :

  0.25

  GRIB_iScansNegatively :

  0

  GRIB_jDirectionIncrementInDegrees :

  0.25

  GRIB_jPointsAreConsecutive :

  0

  GRIB_jScansPositively :

  0

  GRIB_latitudeOfFirstGridPointInDegrees :

  90.0

  GRIB_latitudeOfLastGridPointInDegrees :

  -90.0

  GRIB_longitudeOfFirstGridPointInDegrees :

  -180.0

  GRIB_longitudeOfLastGridPointInDegrees :

  179.75

  GRIB_missingValue :

  3.4028234663852886e+38

  GRIB_name :

  2 metre temperature

  GRIB_numberOfPoints :

  1038240

  GRIB_paramId :

  167

  GRIB_shortName :

  2t

  GRIB_stepType :

  instant

  GRIB_stepUnits :

  1

  GRIB_totalNumber :

  0

  GRIB_typeOfLevel :

  surface

  GRIB_units :

  K

  long_name :

  2 metre temperature

  standard_name :

  unknown

  units :

  K

title = f"Comparison of total precipitation across three cities in North America from \
{xds.data.time.dt.year.min().values} to {xds.data.time.dt.year.max().values}"

xds.data.sel(
    timestep="D",
    source="era5_reanalysis_single_levels",
).hvplot(
    title=title,
    x="time",
    y="tp_nansum",
    grid=True,
    width=750,
    height=450,
    by="site",
    legend="top",
    widget_location="bottom",
)

title = f"Comparison of 2m temperature across three cities in North America from \
{xds.data.time.dt.year.min().values} to {xds.data.time.dt.year.max().values}"

xds.data.sel(
    timestep="D",
    source="era5_reanalysis_single_levels",
).hvplot(
    title=title,
    x="time",
    y="t2m_nanmean",
    grid=True,
    width=750,
    height=450,
    by="site",
    legend="top",
    widget_location="bottom",
)

Clip on polygons with no averaging in space

First, let’s explore specific polygon features. With xdatasets, you can access geographical datasets, such as watershed boundaries linked to streamflow stations. These datasets follow a nomenclature where they are named after the hydrological dataset, with "_polygons" appended. For example, if the hydrological dataset is named deh, its corresponding watershed boundaries dataset will be labeled deh_polygons. The query below retrieves all polygons for the deh_polygons dataset.

gdf = xd.Query(
    **{
        "datasets": "deh_polygons"
}).data

gdf

As the data is loaded into memory, the process of loading all polygons may take some time. To expedite this, we recommend employing filters, as illustrated below. It’s important to note that the filters are consistent for both hydrological and corresponding geographical datasets. Consequently, only watershed boundaries associated with existing hydrological data will be returned.

import xdatasets as xd

gdf = xd.Query(
    **{
        "datasets": {
            "deh_polygons": {
                "id": ["0421*"],
            }
        }
    }
).data.reset_index()

gdf

	Station	Superficie	geometry
0	042102	623.479187	POLYGON ((-78.57120 46.70742, -78.57112 46.707...
1	042103	579.479614	POLYGON ((-78.49014 46.64514, -78.49010 46.645...

Let’s examine the geographic locations of the polygon features.

gdf.hvplot(
    geo=True,
    tiles="ESRI",
    color="Station",
    alpha=0.8,
    width=750,
    height=450,
    legend="top",
    hover_cols=["Station", "Superficie"],
)

The following query seeks the variables t2m and tp from the era5_reanalysis_single_levels dataset, covering the period between January 1, 1959, and September 30, 1961, for the three polygons mentioned earlier. It is important to note that as aggregation is set to False, no spatial averaging will be conducted, and a mask (raster) will be returned for each polygon.

query = {
    "datasets": {"era5_reanalysis_single_levels": {"variables": ["t2m", "tp"]}},
    "space": {
        "clip": "polygon",  # bbox, point or polygon
        "averaging": False,  # spatial average of the variables within each polygon
        "geometry": gdf,
        "unique_id": "Station",  # unique column name in geodataframe
    },
    "time": {
        "start": "1959-01-01",
        "end": "1963-08-31",
    },
}

xds = xd.Query(**query)

Spatial operations: processing polygon 042102 with era5_reanalysis_single_levels: : 0it [00:00, ?it/s]/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/aux.py:230: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  bnds_tmp = xr.DataArray(data=np.zeros((ds.dims[var],2))*np.nan,
/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/aux.py:230: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  bnds_tmp = xr.DataArray(data=np.zeros((ds.dims[var],2))*np.nan,
/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/core.py:253: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  (ds_bnds,) = (xr.broadcast(ds.isel({d:0 for d in [k for k in ds.dims.keys() if k not in ['lat','lon','bnds']]}).
/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/core.py:272: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  gdf_pixels = gpd.GeoDataFrame(pd.DataFrame({v:[None]*ds_bnds.dims['loc']
Spatial operations: processing polygon 042103 with era5_reanalysis_single_levels: : 1it [00:00,  5.12it/s]/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/aux.py:230: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  bnds_tmp = xr.DataArray(data=np.zeros((ds.dims[var],2))*np.nan,
/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/aux.py:230: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  bnds_tmp = xr.DataArray(data=np.zeros((ds.dims[var],2))*np.nan,
/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/core.py:253: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  (ds_bnds,) = (xr.broadcast(ds.isel({d:0 for d in [k for k in ds.dims.keys() if k not in ['lat','lon','bnds']]}).
/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/core.py:272: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  gdf_pixels = gpd.GeoDataFrame(pd.DataFrame({v:[None]*ds_bnds.dims['loc']
Spatial operations: processing polygon 042103 with era5_reanalysis_single_levels: : 2it [00:00,  5.18it/s]

By accessing the data attribute, you can view the data obtained from the query. For each variable, the dimensions of time, latitude, longitude, and Station (the unique ID) are included. In addition, there is another variable called weights that is returned. This variable specifies the weight that should be assigned to each pixel if spatial averaging is conducted over a mask (polygon).

xds.data

<xarray.Dataset>
Dimensions:    (Station: 2, time: 40896, latitude: 3, longitude: 2, source: 1)
Coordinates:
  * latitude   (latitude) float64 46.25 46.5 46.75
  * longitude  (longitude) float64 -78.5 -78.25
  * time       (time) datetime64[ns] 1959-01-01 ... 1963-08-31T23:00:00
  * Station    (Station) object '042102' '042103'
  * source     (source) <U29 'era5_reanalysis_single_levels'
Data variables:
    t2m        (Station, time, latitude, longitude) float32 260.3 259.9 ... nan
    tp         (Station, time, latitude, longitude) float32 nan nan ... 0.0 nan
    weights    (Station, latitude, longitude) float64 0.01953 0.1244 ... nan
Attributes:
    Conventions:               CF-1.6
    history:                   2022-11-10 02:03:41 GMT by grib_to_netcdf-2.25...
    pangeo-forge:inputs_hash:  c4e1de94d7bedf0a63629db8fa0633c03b7e266149e97f...
    pangeo-forge:recipe_hash:  66be9c20b44c1a0fca83c2dd2a6f147aecc5be14590f1f...
    pangeo-forge:version:      0.9.4

xarray.Dataset

• Dimensions:
  • Station: 2
  • time: 40896
  • latitude: 3
  • longitude: 2
  • source: 1
• Coordinates: (5)
  • latitude
    (latitude)
    float64
    46.25 46.5 46.75

    long_name :

    latitude

    standard_name :

    latitude

    stored_direction :

    decreasing

    units :

    degrees_north

    array([46.25, 46.5 , 46.75])

  • longitude
    (longitude)
    float64
    -78.5 -78.25

    long_name :

    longitude

    standard_name :

    longitude

    units :

    degrees_east

    array([-78.5 , -78.25])

  • time
    (time)
    datetime64[ns]
    1959-01-01 ... 1963-08-31T23:00:00

    array(['1959-01-01T00:00:00.000000000', '1959-01-01T01:00:00.000000000',
           '1959-01-01T02:00:00.000000000', ..., '1963-08-31T21:00:00.000000000',
           '1963-08-31T22:00:00.000000000', '1963-08-31T23:00:00.000000000'],
          dtype='datetime64[ns]')

  • Station
    (Station)
    object
    '042102' '042103'

    cf_role :

    timeseries_id

    array(['042102', '042103'], dtype=object)

  • source
    (source)
    <U29
    'era5_reanalysis_single_levels'

    array(['era5_reanalysis_single_levels'], dtype='<U29')

• Data variables: (3)
  • t2m
    (Station, time, latitude, longitude)
    float32
    260.3 259.9 260.6 ... 288.2 nan

    GRIB_NV :

    0

    GRIB_Nx :

    1440

    GRIB_Ny :

    721

    GRIB_cfName :

    unknown

    GRIB_cfVarName :

    t2m

    GRIB_dataType :

    an

    GRIB_gridDefinitionDescription :

    Latitude/Longitude Grid

    GRIB_gridType :

    regular_ll

    GRIB_iDirectionIncrementInDegrees :

    0.25

    GRIB_iScansNegatively :

    0

    GRIB_jDirectionIncrementInDegrees :

    0.25

    GRIB_jPointsAreConsecutive :

    0

    GRIB_jScansPositively :

    0

    GRIB_latitudeOfFirstGridPointInDegrees :

    90.0

    GRIB_latitudeOfLastGridPointInDegrees :

    -90.0

    GRIB_longitudeOfFirstGridPointInDegrees :

    -180.0

    GRIB_longitudeOfLastGridPointInDegrees :

    179.75

    GRIB_missingValue :

    3.4028234663852886e+38

    GRIB_name :

    2 metre temperature

    GRIB_numberOfPoints :

    1038240

    GRIB_paramId :

    167

    GRIB_shortName :

    2t

    GRIB_stepType :

    instant

    GRIB_stepUnits :

    1

    GRIB_totalNumber :

    0

    GRIB_typeOfLevel :

    surface

    GRIB_units :

    K

    long_name :

    2 metre temperature

    standard_name :

    unknown

    units :

    K

    array([[[[260.29358, 259.86   ],
             [260.5924 , 260.2975 ],
             [260.56702,       nan]],
    
            [[260.35132, 259.83765],
             [260.66772, 260.365  ],
             [260.69897,       nan]],
    
            [[261.10132, 260.57007],
             [261.00952, 260.76147],
             [261.2439 ,       nan]],
    
            ...,
    
            [[289.57703, 289.35828],
             [289.0868 , 288.73132],
             [288.84265,       nan]],
    
            [[288.66583, 288.59357],
             [288.55255, 288.3084 ],
    ...
             [260.66772, 260.365  ],
             [260.69897,       nan]],
    
            [[261.10132, 260.57007],
             [261.00952, 260.76147],
             [261.2439 ,       nan]],
    
            ...,
    
            [[289.57703, 289.35828],
             [289.0868 , 288.73132],
             [288.84265,       nan]],
    
            [[288.66583, 288.59357],
             [288.55255, 288.3084 ],
             [288.32208,       nan]],
    
            [[288.4598 , 288.35825],
             [288.53207, 288.243  ],
             [288.1766 ,       nan]]]], dtype=float32)

  • tp
    (Station, time, latitude, longitude)
    float32
    nan nan nan nan ... 0.0 0.0 0.0 nan

    long_name :

    Total precipitation

    units :

    m

    array([[[[nan, nan],
             [nan, nan],
             [nan, nan]],
    
            [[nan, nan],
             [nan, nan],
             [nan, nan]],
    
            [[nan, nan],
             [nan, nan],
             [nan, nan]],
    
            ...,
    
            [[ 0.,  0.],
             [ 0.,  0.],
             [ 0., nan]],
    
            [[ 0.,  0.],
             [ 0.,  0.],
    ...
             [nan, nan],
             [nan, nan]],
    
            [[nan, nan],
             [nan, nan],
             [nan, nan]],
    
            ...,
    
            [[ 0.,  0.],
             [ 0.,  0.],
             [ 0., nan]],
    
            [[ 0.,  0.],
             [ 0.,  0.],
             [ 0., nan]],
    
            [[ 0.,  0.],
             [ 0.,  0.],
             [ 0., nan]]]], dtype=float32)

  • weights
    (Station, latitude, longitude)
    float64
    0.01953 0.1244 ... 0.05175 nan

    array([[[0.01953199, 0.12442214],
            [0.38660348, 0.42134397],
            [0.04809842,        nan]],
    
           [[0.01998101, 0.05901206],
            [0.4159409 , 0.45331767],
            [0.05174837,        nan]]])

• Indexes: (5)
  • latitude
    PandasIndex

    PandasIndex(Index([46.25, 46.5, 46.75], dtype='float64', name='latitude'))

  • longitude
    PandasIndex

    PandasIndex(Index([-78.5, -78.25], dtype='float64', name='longitude'))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['1959-01-01 00:00:00', '1959-01-01 01:00:00',
                   '1959-01-01 02:00:00', '1959-01-01 03:00:00',
                   '1959-01-01 04:00:00', '1959-01-01 05:00:00',
                   '1959-01-01 06:00:00', '1959-01-01 07:00:00',
                   '1959-01-01 08:00:00', '1959-01-01 09:00:00',
                   ...
                   '1963-08-31 14:00:00', '1963-08-31 15:00:00',
                   '1963-08-31 16:00:00', '1963-08-31 17:00:00',
                   '1963-08-31 18:00:00', '1963-08-31 19:00:00',
                   '1963-08-31 20:00:00', '1963-08-31 21:00:00',
                   '1963-08-31 22:00:00', '1963-08-31 23:00:00'],
                  dtype='datetime64[ns]', name='time', length=40896, freq=None))

  • Station
    PandasIndex

    PandasIndex(Index(['042102', '042103'], dtype='object', name='Station'))

  • source
    PandasIndex

    PandasIndex(Index(['era5_reanalysis_single_levels'], dtype='object', name='source'))

• Attributes: (5)

  Conventions :

  CF-1.6

  history :

  2022-11-10 02:03:41 GMT by grib_to_netcdf-2.25.1: /opt/ecmwf/mars-client/bin/grib_to_netcdf.bin -S param -o /cache/data7/adaptor.mars.internal-1668045821.5257316-3219-13-377ecf1c-2aeb-4f26-bdba-6033b97bd350.nc /cache/tmp/377ecf1c-2aeb-4f26-bdba-6033b97bd350-adaptor.mars.internal-1668045819.9198856-3219-21-tmp.grib

  pangeo-forge:inputs_hash :

  c4e1de94d7bedf0a63629db8fa0633c03b7e266149e97fe7645c350eada844d6

  pangeo-forge:recipe_hash :

  66be9c20b44c1a0fca83c2dd2a6f147aecc5be14590f1f4ed9eb3361f472cc9d

  pangeo-forge:version :

  0.9.4

Weights are much easier to comprehend visually, so let’s examine the weights returned for the station 042102. Notice that when selecting a single feature (Station 042102 in this case), the shape of our spatial dimensions is reduced to a 3x2 pixel area (longitude x latitude) that encompasses the entire feature.

station = "042102"

ds_station = xds.data.sel(Station=station)
ds_clipped = xds.bbox_clip(ds_station).squeeze()
ds_clipped

<xarray.Dataset>
Dimensions:    (time: 40896, latitude: 3, longitude: 2)
Coordinates:
  * latitude   (latitude) float64 46.25 46.5 46.75
  * longitude  (longitude) float64 -78.5 -78.25
  * time       (time) datetime64[ns] 1959-01-01 ... 1963-08-31T23:00:00
    Station    <U6 '042102'
    source     <U29 'era5_reanalysis_single_levels'
Data variables:
    t2m        (time, latitude, longitude) float32 260.3 259.9 ... 288.2 nan
    tp         (time, latitude, longitude) float32 nan nan nan ... 0.0 0.0 nan
    weights    (latitude, longitude) float64 0.01953 0.1244 ... 0.0481 nan
Attributes:
    Conventions:               CF-1.6
    history:                   2022-11-10 02:03:41 GMT by grib_to_netcdf-2.25...
    pangeo-forge:inputs_hash:  c4e1de94d7bedf0a63629db8fa0633c03b7e266149e97f...
    pangeo-forge:recipe_hash:  66be9c20b44c1a0fca83c2dd2a6f147aecc5be14590f1f...
    pangeo-forge:version:      0.9.4

xarray.Dataset

• Dimensions:
  • time: 40896
  • latitude: 3
  • longitude: 2
• Coordinates: (5)
  • latitude
    (latitude)
    float64
    46.25 46.5 46.75

    long_name :

    latitude

    standard_name :

    latitude

    stored_direction :

    decreasing

    units :

    degrees_north

    array([46.25, 46.5 , 46.75])

  • longitude
    (longitude)
    float64
    -78.5 -78.25

    long_name :

    longitude

    standard_name :

    longitude

    units :

    degrees_east

    array([-78.5 , -78.25])

  • time
    (time)
    datetime64[ns]
    1959-01-01 ... 1963-08-31T23:00:00

    array(['1959-01-01T00:00:00.000000000', '1959-01-01T01:00:00.000000000',
           '1959-01-01T02:00:00.000000000', ..., '1963-08-31T21:00:00.000000000',
           '1963-08-31T22:00:00.000000000', '1963-08-31T23:00:00.000000000'],
          dtype='datetime64[ns]')

  • Station
    ()
    <U6
    '042102'

    cf_role :

    timeseries_id

    array('042102', dtype='<U6')

  • source
    ()
    <U29
    'era5_reanalysis_single_levels'

    array('era5_reanalysis_single_levels', dtype='<U29')

• Data variables: (3)
  • t2m
    (time, latitude, longitude)
    float32
    260.3 259.9 260.6 ... 288.2 nan

    GRIB_NV :

    0

    GRIB_Nx :

    1440

    GRIB_Ny :

    721

    GRIB_cfName :

    unknown

    GRIB_cfVarName :

    t2m

    GRIB_dataType :

    an

    GRIB_gridDefinitionDescription :

    Latitude/Longitude Grid

    GRIB_gridType :

    regular_ll

    GRIB_iDirectionIncrementInDegrees :

    0.25

    GRIB_iScansNegatively :

    0

    GRIB_jDirectionIncrementInDegrees :

    0.25

    GRIB_jPointsAreConsecutive :

    0

    GRIB_jScansPositively :

    0

    GRIB_latitudeOfFirstGridPointInDegrees :

    90.0

    GRIB_latitudeOfLastGridPointInDegrees :

    -90.0

    GRIB_longitudeOfFirstGridPointInDegrees :

    -180.0

    GRIB_longitudeOfLastGridPointInDegrees :

    179.75

    GRIB_missingValue :

    3.4028234663852886e+38

    GRIB_name :

    2 metre temperature

    GRIB_numberOfPoints :

    1038240

    GRIB_paramId :

    167

    GRIB_shortName :

    2t

    GRIB_stepType :

    instant

    GRIB_stepUnits :

    1

    GRIB_totalNumber :

    0

    GRIB_typeOfLevel :

    surface

    GRIB_units :

    K

    long_name :

    2 metre temperature

    standard_name :

    unknown

    units :

    K

    array([[[260.29358, 259.86   ],
            [260.5924 , 260.2975 ],
            [260.56702,       nan]],
    
           [[260.35132, 259.83765],
            [260.66772, 260.365  ],
            [260.69897,       nan]],
    
           [[261.10132, 260.57007],
            [261.00952, 260.76147],
            [261.2439 ,       nan]],
    
           ...,
    
           [[289.57703, 289.35828],
            [289.0868 , 288.73132],
            [288.84265,       nan]],
    
           [[288.66583, 288.59357],
            [288.55255, 288.3084 ],
            [288.32208,       nan]],
    
           [[288.4598 , 288.35825],
            [288.53207, 288.243  ],
            [288.1766 ,       nan]]], dtype=float32)

  • tp
    (time, latitude, longitude)
    float32
    nan nan nan nan ... 0.0 0.0 0.0 nan

    long_name :

    Total precipitation

    units :

    m

    array([[[nan, nan],
            [nan, nan],
            [nan, nan]],
    
           [[nan, nan],
            [nan, nan],
            [nan, nan]],
    
           [[nan, nan],
            [nan, nan],
            [nan, nan]],
    
           ...,
    
           [[ 0.,  0.],
            [ 0.,  0.],
            [ 0., nan]],
    
           [[ 0.,  0.],
            [ 0.,  0.],
            [ 0., nan]],
    
           [[ 0.,  0.],
            [ 0.,  0.],
            [ 0., nan]]], dtype=float32)

  • weights
    (latitude, longitude)
    float64
    0.01953 0.1244 ... 0.0481 nan

    array([[0.01953199, 0.12442214],
           [0.38660348, 0.42134397],
           [0.04809842,        nan]])

• Indexes: (3)
  • latitude
    PandasIndex

    PandasIndex(Index([46.25, 46.5, 46.75], dtype='float64', name='latitude'))

  • longitude
    PandasIndex

    PandasIndex(Index([-78.5, -78.25], dtype='float64', name='longitude'))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['1959-01-01 00:00:00', '1959-01-01 01:00:00',
                   '1959-01-01 02:00:00', '1959-01-01 03:00:00',
                   '1959-01-01 04:00:00', '1959-01-01 05:00:00',
                   '1959-01-01 06:00:00', '1959-01-01 07:00:00',
                   '1959-01-01 08:00:00', '1959-01-01 09:00:00',
                   ...
                   '1963-08-31 14:00:00', '1963-08-31 15:00:00',
                   '1963-08-31 16:00:00', '1963-08-31 17:00:00',
                   '1963-08-31 18:00:00', '1963-08-31 19:00:00',
                   '1963-08-31 20:00:00', '1963-08-31 21:00:00',
                   '1963-08-31 22:00:00', '1963-08-31 23:00:00'],
                  dtype='datetime64[ns]', name='time', length=40896, freq=None))

• Attributes: (5)

  Conventions :

  CF-1.6

  history :

  2022-11-10 02:03:41 GMT by grib_to_netcdf-2.25.1: /opt/ecmwf/mars-client/bin/grib_to_netcdf.bin -S param -o /cache/data7/adaptor.mars.internal-1668045821.5257316-3219-13-377ecf1c-2aeb-4f26-bdba-6033b97bd350.nc /cache/tmp/377ecf1c-2aeb-4f26-bdba-6033b97bd350-adaptor.mars.internal-1668045819.9198856-3219-21-tmp.grib

  pangeo-forge:inputs_hash :

  c4e1de94d7bedf0a63629db8fa0633c03b7e266149e97fe7645c350eada844d6

  pangeo-forge:recipe_hash :

  66be9c20b44c1a0fca83c2dd2a6f147aecc5be14590f1f4ed9eb3361f472cc9d

  pangeo-forge:version :

  0.9.4

(
    (
        ds_clipped.t2m.isel(time=0).hvplot(
            title="The 2m temperature for pixels that intersect with the polygon on January 1, 1959",
            tiles="ESRI",
            geo=True,
            alpha=0.6,
            colormap="isolum",
            width=750,
            height=450,
        )
        * gdf[gdf.Station == station].hvplot(
            geo=True,
            width=750,
            height=450,
            legend="top",
            hover_cols=["Station", "Superficie"],
        )
    )
    + ds_clipped.weights.hvplot(
        title="The weights that should be assigned to each pixel when performing spatial averaging",
        tiles="ESRI",
        alpha=0.6,
        colormap="isolum",
        geo=True,
        width=750,
        height=450,
    )
    * gdf[gdf.Station == station].hvplot(
        geo=True,
        width=750,
        height=450,
        legend="top",
        hover_cols=["Station", "Superficie"],
    )
).cols(1)

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/IPython/core/formatters.py:974, in MimeBundleFormatter.__call__(self, obj, include, exclude)
    971     method = get_real_method(obj, self.print_method)
    973     if method is not None:
--> 974         return method(include=include, exclude=exclude)
    975     return None
    976 else:

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/holoviews/core/dimension.py:1286, in Dimensioned._repr_mimebundle_(self, include, exclude)
   1279 def _repr_mimebundle_(self, include=None, exclude=None):
   1280     """
   1281     Resolves the class hierarchy for the class rendering the
   1282     object using any display hooks registered on Store.display
   1283     hooks.  The output of all registered display_hooks is then
   1284     combined and returned.
   1285     """
-> 1286     return Store.render(self)

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/holoviews/core/options.py:1428, in Store.render(cls, obj)
   1426 data, metadata = {}, {}
   1427 for hook in hooks:
-> 1428     ret = hook(obj)
   1429     if ret is None:
   1430         continue

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/holoviews/ipython/display_hooks.py:287, in pprint_display(obj)
    285 if not ip.display_formatter.formatters['text/plain'].pprint:
    286     return None
--> 287 return display(obj, raw_output=True)

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/holoviews/ipython/display_hooks.py:258, in display(obj, raw_output, **kwargs)
    256 elif isinstance(obj, (Layout, NdLayout, AdjointLayout)):
    257     with option_state(obj):
--> 258         output = layout_display(obj)
    259 elif isinstance(obj, (HoloMap, DynamicMap)):
    260     with option_state(obj):

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/holoviews/ipython/display_hooks.py:149, in display_hook.<locals>.wrapped(element)
    147 try:
    148     max_frames = OutputSettings.options['max_frames']
--> 149     mimebundle = fn(element, max_frames=max_frames)
    150     if mimebundle is None:
    151         return {}, {}

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/holoviews/ipython/display_hooks.py:223, in layout_display(layout, max_frames)
    220     max_frame_warning(max_frames)
    221     return None
--> 223 return render(layout)

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/holoviews/ipython/display_hooks.py:76, in render(obj, **kwargs)
     73 if renderer.fig == 'pdf':
     74     renderer = renderer.instance(fig='png')
---> 76 return renderer.components(obj, **kwargs)

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/holoviews/plotting/renderer.py:396, in Renderer.components(self, obj, fmt, comm, **kwargs)
    393 embed = (not (dynamic or streams or self.widget_mode == 'live') or config.embed)
    395 if embed or config.comms == 'default':
--> 396     return self._render_panel(plot, embed, comm)
    397 return self._render_ipywidget(plot)

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/holoviews/plotting/renderer.py:403, in Renderer._render_panel(self, plot, embed, comm)
    401 doc = Document()
    402 with config.set(embed=embed):
--> 403     model = plot.layout._render_model(doc, comm)
    404 if embed:
    405     return render_model(model, comm)

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/panel/viewable.py:748, in Viewable._render_model(self, doc, comm)
    746 if comm is None:
    747     comm = state._comm_manager.get_server_comm()
--> 748 model = self.get_root(doc, comm)
    750 if self._design and self._design.theme.bokeh_theme:
    751     doc.theme = self._design.theme.bokeh_theme

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/panel/layout/base.py:311, in Panel.get_root(self, doc, comm, preprocess)
    307 def get_root(
    308     self, doc: Optional[Document] = None, comm: Optional[Comm] = None,
    309     preprocess: bool = True
    310 ) -> Model:
--> 311     root = super().get_root(doc, comm, preprocess)
    312     # ALERT: Find a better way to handle this
    313     if hasattr(root, 'styles') and 'overflow-x' in root.styles:

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/panel/viewable.py:666, in Renderable.get_root(self, doc, comm, preprocess)
    664 wrapper = self._design._wrapper(self)
    665 if wrapper is self:
--> 666     root = self._get_model(doc, comm=comm)
    667     if preprocess:
    668         self._preprocess(root)

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/panel/layout/base.py:177, in Panel._get_model(self, doc, root, parent, comm)
    175 root = root or model
    176 self._models[root.ref['id']] = (model, parent)
--> 177 objects, _ = self._get_objects(model, [], doc, root, comm)
    178 props = self._get_properties(doc)
    179 props[self._property_mapping['objects']] = objects

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/panel/layout/base.py:159, in Panel._get_objects(self, model, old_objects, doc, root, comm)
    157 else:
    158     try:
--> 159         child = pane._get_model(doc, root, model, comm)
    160     except RerenderError as e:
    161         if e.layout is not None and e.layout is not self:

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/panel/pane/holoviews.py:423, in HoloViews._get_model(self, doc, root, parent, comm)
    421     plot = self.object
    422 else:
--> 423     plot = self._render(doc, comm, root)
    425 plot.pane = self
    426 backend = plot.renderer.backend

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/panel/pane/holoviews.py:518, in HoloViews._render(self, doc, comm, root)
    515     if comm:
    516         kwargs['comm'] = comm
--> 518 return renderer.get_plot(self.object, **kwargs)

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/holoviews/plotting/bokeh/renderer.py:68, in BokehRenderer.get_plot(self_or_cls, obj, doc, renderer, **kwargs)
     61 @bothmethod
     62 def get_plot(self_or_cls, obj, doc=None, renderer=None, **kwargs):
     63     """
     64     Given a HoloViews Viewable return a corresponding plot instance.
     65     Allows supplying a document attach the plot to, useful when
     66     combining the bokeh model with another plot.
     67     """
---> 68     plot = super().get_plot(obj, doc, renderer, **kwargs)
     69     if plot.document is None:
     70         plot.document = Document() if self_or_cls.notebook_context else curdoc()

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/holoviews/plotting/renderer.py:240, in Renderer.get_plot(self_or_cls, obj, doc, renderer, comm, **kwargs)
    237     defaults = [kd.default for kd in plot.dimensions]
    238     init_key = tuple(v if d is None else d for v, d in
    239                      zip(plot.keys[0], defaults))
--> 240     plot.update(init_key)
    241 else:
    242     plot = obj

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/holoviews/plotting/plot.py:955, in DimensionedPlot.update(self, key)
    953 def update(self, key):
    954     if len(self) == 1 and key in (0, self.keys[0]) and not self.drawn:
--> 955         return self.initialize_plot()
    956     item = self.__getitem__(key)
    957     self.traverse(lambda x: setattr(x, '_updated', True))

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/holoviews/plotting/bokeh/plot.py:945, in LayoutPlot.initialize_plot(self, plots, ranges)
    942     continue
    944 shared_plots = list(passed_plots) if self.shared_axes else None
--> 945 subplots = subplot.initialize_plot(ranges=ranges, plots=shared_plots)
    946 nsubplots = len(subplots)
    948 modes = {sp.sizing_mode for sp in subplots
    949          if sp.sizing_mode not in (None, 'auto', 'fixed')}

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/holoviews/plotting/bokeh/plot.py:1094, in AdjointLayoutPlot.initialize_plot(self, ranges, plots)
   1092     else:
   1093         passed_plots = plots + adjoined_plots
-> 1094         adjoined_plots.append(subplot.initialize_plot(ranges=ranges, plots=passed_plots))
   1095 self.drawn = True
   1096 if not adjoined_plots: adjoined_plots = [None]

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/geoviews/plotting/bokeh/plot.py:106, in GeoPlot.initialize_plot(self, ranges, plot, plots, source)
    104 def initialize_plot(self, ranges=None, plot=None, plots=None, source=None):
    105     opts = {} if isinstance(self, HvOverlayPlot) else {'source': source}
--> 106     fig = super().initialize_plot(ranges, plot, plots, **opts)
    107     if self.geographic and self.show_bounds and not self.overlaid:
    108         from . import GeoShapePlot

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/holoviews/plotting/bokeh/element.py:2899, in OverlayPlot.initialize_plot(self, ranges, plot, plots)
   2897 if self.tabs:
   2898     subplot.overlaid = False
-> 2899 child = subplot.initialize_plot(ranges, plot, plots)
   2900 if isinstance(element, CompositeOverlay):
   2901     # Ensure that all subplots are in the same state
   2902     frame = element.get(key, None)

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/geoviews/plotting/bokeh/plot.py:106, in GeoPlot.initialize_plot(self, ranges, plot, plots, source)
    104 def initialize_plot(self, ranges=None, plot=None, plots=None, source=None):
    105     opts = {} if isinstance(self, HvOverlayPlot) else {'source': source}
--> 106     fig = super().initialize_plot(ranges, plot, plots, **opts)
    107     if self.geographic and self.show_bounds and not self.overlaid:
    108         from . import GeoShapePlot

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/holoviews/plotting/bokeh/element.py:1888, in ElementPlot.initialize_plot(self, ranges, plot, plots, source)
   1886 if self.autorange:
   1887     self._setup_autorange()
-> 1888 self._init_glyphs(plot, element, ranges, source)
   1889 if not self.overlaid:
   1890     self._update_plot(key, plot, style_element)

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/holoviews/plotting/bokeh/element.py:1829, in ElementPlot._init_glyphs(self, plot, element, ranges, source)
   1826 if isinstance(renderer, Renderer):
   1827     self.handles['glyph_renderer'] = renderer
-> 1829 self._postprocess_hover(renderer, source)
   1831 # Update plot, source and glyph
   1832 with abbreviated_exception():

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/geoviews/plotting/bokeh/plot.py:117, in GeoPlot._postprocess_hover(self, renderer, source)
    116 def _postprocess_hover(self, renderer, source):
--> 117     super()._postprocess_hover(renderer, source)
    118     hover = getattr(self.handles["plot"], "hover")
    119     hover = hover[0] if hover else None

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/holoviews/plotting/bokeh/raster.py:85, in RasterPlot._postprocess_hover(self, renderer, source)
     76     datetime_code = """
     77     if (value === -9223372036854776) {
     78         return "NaN"
   (...)
     82     }
     83     """
     84     for key in formatters:
---> 85         if formatters[key].lower() == "datetime":
     86             formatters[key] = CustomJSHover(code=datetime_code)
     88 hover.tooltips = tooltips

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/bokeh/core/has_props.py:367, in HasProps.__getattr__(self, name)
    364 if isinstance(descriptor, property): # Python property
    365     return super().__getattribute__(name)
--> 367 self._raise_attribute_error_with_matches(name, properties)

File ~/micromamba/envs/xdatasets/lib/python3.9/site-packages/bokeh/core/has_props.py:375, in HasProps._raise_attribute_error_with_matches(self, name, properties)
    372 if not matches:
    373     matches, text = sorted(properties), "possible"
--> 375 raise AttributeError(f"unexpected attribute {name!r} to {self.__class__.__name__}, {text} attributes are {nice_join(matches)}")

AttributeError: unexpected attribute 'lower' to CustomJSHover, possible attributes are args, code, js_event_callbacks, js_property_callbacks, name, subscribed_events, syncable or tags

:Layout
   .Overlay.I  :Overlay
      .WMTS.I     :WMTS   [Longitude,Latitude]
      .Image.I    :Image   [longitude,latitude]   (t2m)
      .Polygons.I :Polygons   [Longitude,Latitude]   (Station,Superficie)
   .Overlay.II :Overlay
      .WMTS.I     :WMTS   [Longitude,Latitude]
      .Image.I    :Image   [longitude,latitude]   (weights)
      .Polygons.I :Polygons   [Longitude,Latitude]   (Station,Superficie)

The two plots depicted above show the 2m temperature for each pixel that intersects with the polygon from Station 042102 and the corresponding weights to be applied to each pixel. In the lower plot, it is apparent that the majority of the polygon is situated in the central pixels, which results in those pixels having a weight of approximately 80%. It is evident that the two lower and the upper pixels have much less intersection with the polygon, which results in their respective weights being smaller (hover on the plot to verify the weights).

In various libraries, either all pixels that intersect with the geometries are kept, or only pixels with centers within the polygon are retained. However, as shown in the previous example, utilizing such methods can introduce significant biases in the final calculations.

Clip on polygons with averaging in space

The following query seeks the variables t2m and tp from the era5_reanalysis_single_levels and era5_land_reanalysis datasets, covering the period between January 1, 1950, to present, for the three polygons mentioned earlier. Note that when the aggregation parameter is set to True, spatial averaging takes place. In addition, the weighted mask (raster) described earlier will be applied to generate a time series for each polygon.

Additional steps are carried out in the process, including converting the original hourly time step to a daily time step. During this conversion, various temporal aggregations will be applied to each variable and a conversion to the local time zone will take place.

Note If users prefer to pass multiple dictionaries instead of a single large one, the following format is also considered acceptable.

datasets = {
    "era5_reanalysis_single_levels": {"variables": ["t2m", "tp"]},
    "era5_land_reanalysis": {"variables": ["t2m", "tp"]},
}
space = {
    "clip": "polygon",  # bbox, point or polygon
    "averaging": True,
    "geometry": gdf,  # 3 polygons
    "unique_id": "Station",
}
time = {
    "timestep": "D",
    "aggregation": {"tp": [np.nansum], "t2m": [np.nanmax, np.nanmin]},
    "start": "1950-01-01",
    "timezone": "America/Montreal",
}

xds = xd.Query(datasets=datasets, space=space, time=time)

/home/runner/work/xdatasets/xdatasets/xdatasets/workflows.py:51: UserWarning: "start_date" not found within input date time range. Defaulting to minimum time step in xarray object.
  ds = subset_time(ds, start_date=start_time, end_date=end_time)
Spatial operations: processing polygon 042102 with era5_reanalysis_single_levels: : 0it [00:00, ?it/s]/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/aux.py:230: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  bnds_tmp = xr.DataArray(data=np.zeros((ds.dims[var],2))*np.nan,
/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/aux.py:230: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  bnds_tmp = xr.DataArray(data=np.zeros((ds.dims[var],2))*np.nan,
/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/core.py:253: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  (ds_bnds,) = (xr.broadcast(ds.isel({d:0 for d in [k for k in ds.dims.keys() if k not in ['lat','lon','bnds']]}).
/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/core.py:272: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  gdf_pixels = gpd.GeoDataFrame(pd.DataFrame({v:[None]*ds_bnds.dims['loc']
Spatial operations: processing polygon 042103 with era5_reanalysis_single_levels: : 1it [00:00,  4.61it/s]/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/aux.py:230: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  bnds_tmp = xr.DataArray(data=np.zeros((ds.dims[var],2))*np.nan,
/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/aux.py:230: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  bnds_tmp = xr.DataArray(data=np.zeros((ds.dims[var],2))*np.nan,
/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/core.py:253: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  (ds_bnds,) = (xr.broadcast(ds.isel({d:0 for d in [k for k in ds.dims.keys() if k not in ['lat','lon','bnds']]}).
/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/core.py:272: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  gdf_pixels = gpd.GeoDataFrame(pd.DataFrame({v:[None]*ds_bnds.dims['loc']
Spatial operations: processing polygon 042103 with era5_reanalysis_single_levels: : 2it [00:00,  4.60it/s]
Temporal operations: processing tp with era5_reanalysis_single_levels: 100%|██████████| 2/2 [00:08<00:00,  4.03s/it]
Spatial operations: processing polygon 042102 with era5_land_reanalysis: : 0it [00:00, ?it/s]/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/aux.py:230: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  bnds_tmp = xr.DataArray(data=np.zeros((ds.dims[var],2))*np.nan,
/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/aux.py:230: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  bnds_tmp = xr.DataArray(data=np.zeros((ds.dims[var],2))*np.nan,
/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/core.py:253: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  (ds_bnds,) = (xr.broadcast(ds.isel({d:0 for d in [k for k in ds.dims.keys() if k not in ['lat','lon','bnds']]}).
/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/core.py:272: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  gdf_pixels = gpd.GeoDataFrame(pd.DataFrame({v:[None]*ds_bnds.dims['loc']
Spatial operations: processing polygon 042103 with era5_land_reanalysis: : 1it [00:00,  3.63it/s]/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/aux.py:230: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  bnds_tmp = xr.DataArray(data=np.zeros((ds.dims[var],2))*np.nan,
/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/aux.py:230: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  bnds_tmp = xr.DataArray(data=np.zeros((ds.dims[var],2))*np.nan,
/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/core.py:253: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  (ds_bnds,) = (xr.broadcast(ds.isel({d:0 for d in [k for k in ds.dims.keys() if k not in ['lat','lon','bnds']]}).
/home/runner/micromamba/envs/xdatasets/lib/python3.9/site-packages/xagg/core.py:272: FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.
  gdf_pixels = gpd.GeoDataFrame(pd.DataFrame({v:[None]*ds_bnds.dims['loc']
Spatial operations: processing polygon 042103 with era5_land_reanalysis: : 2it [00:00,  3.86it/s]
Temporal operations: processing tp with era5_land_reanalysis: 100%|██████████| 2/2 [00:08<00:00,  4.25s/it]

xds.data

<xarray.Dataset>
Dimensions:      (spatial_agg: 1, timestep: 1, Station: 2, time: 27060,
                  source: 2)
Coordinates:
  * spatial_agg  (spatial_agg) object 'polygon'
  * timestep     (timestep) object 'D'
  * Station      (Station) object '042102' '042103'
  * time         (time) datetime64[ns] 1950-01-01 1950-01-02 ... 2024-02-01
  * source       (source) <U29 'era5_land_reanalysis' 'era5_reanalysis_single...
Data variables:
    t2m_nanmax   (spatial_agg, timestep, Station, time, source) float64 272.8...
    t2m_nanmin   (spatial_agg, timestep, Station, time, source) float64 268.7...
    tp_nansum    (spatial_agg, timestep, Station, time, source) float64 0.000...
Attributes: (12/30)
    GRIB_NV:                                  0
    GRIB_Nx:                                  1440
    GRIB_Ny:                                  721
    GRIB_cfName:                              unknown
    GRIB_cfVarName:                           t2m
    GRIB_dataType:                            an
    ...                                       ...
    GRIB_totalNumber:                         0
    GRIB_typeOfLevel:                         surface
    GRIB_units:                               K
    long_name:                                2 metre temperature
    standard_name:                            unknown
    units:                                    K

xarray.Dataset

• Dimensions:
  • spatial_agg: 1
  • timestep: 1
  • Station: 2
  • time: 27060
  • source: 2
• Coordinates: (5)
  • spatial_agg
    (spatial_agg)
    object
    'polygon'

    array(['polygon'], dtype=object)

  • timestep
    (timestep)
    object
    'D'

    array(['D'], dtype=object)

  • Station
    (Station)
    object
    '042102' '042103'

    cf_role :

    timeseries_id

    array(['042102', '042103'], dtype=object)

  • time
    (time)
    datetime64[ns]
    1950-01-01 ... 2024-02-01

    array(['1950-01-01T00:00:00.000000000', '1950-01-02T00:00:00.000000000',
           '1950-01-03T00:00:00.000000000', ..., '2024-01-30T00:00:00.000000000',
           '2024-01-31T00:00:00.000000000', '2024-02-01T00:00:00.000000000'],
          dtype='datetime64[ns]')

  • source
    (source)
    <U29
    'era5_land_reanalysis' 'era5_rea...

    array(['era5_land_reanalysis', 'era5_reanalysis_single_levels'], dtype='<U29')

• Data variables: (3)
  • t2m_nanmax
    (spatial_agg, timestep, Station, time, source)
    float64
    272.8 nan 275.1 ... nan 275.6 nan

    GRIB_NV :

    0

    GRIB_Nx :

    1440

    GRIB_Ny :

    721

    GRIB_cfName :

    unknown

    GRIB_cfVarName :

    t2m

    GRIB_dataType :

    an

    GRIB_gridDefinitionDescription :

    Latitude/Longitude Grid

    GRIB_gridType :

    regular_ll

    GRIB_iDirectionIncrementInDegrees :

    0.25

    GRIB_iScansNegatively :

    0

    GRIB_jDirectionIncrementInDegrees :

    0.25

    GRIB_jPointsAreConsecutive :

    0

    GRIB_jScansPositively :

    0

    GRIB_latitudeOfFirstGridPointInDegrees :

    90.0

    GRIB_latitudeOfLastGridPointInDegrees :

    -90.0

    GRIB_longitudeOfFirstGridPointInDegrees :

    -180.0

    GRIB_longitudeOfLastGridPointInDegrees :

    179.75

    GRIB_missingValue :

    3.4028234663852886e+38

    GRIB_name :

    2 metre temperature

    GRIB_numberOfPoints :

    1038240

    GRIB_paramId :

    167

    GRIB_shortName :

    2t

    GRIB_stepType :

    instant

    GRIB_stepUnits :

    1

    GRIB_totalNumber :

    0

    GRIB_typeOfLevel :

    surface

    GRIB_units :

    K

    long_name :

    2 metre temperature

    standard_name :

    unknown

    units :

    K

    array([[[[[272.82241918,          nan],
              [275.11908597,          nan],
              [278.63031731,          nan],
              ...,
              [271.72325978,          nan],
              [274.66651182,          nan],
              [275.57841929,          nan]],
    
             [[272.81799986,          nan],
              [275.13434532,          nan],
              [278.58419256,          nan],
              ...,
              [271.71501781,          nan],
              [274.65089339,          nan],
              [275.58020039,          nan]]]]])

  • t2m_nanmin
    (spatial_agg, timestep, Station, time, source)
    float64
    268.7 nan 273.1 ... nan 273.1 nan

    GRIB_NV :

    0

    GRIB_Nx :

    1440

    GRIB_Ny :

    721

    GRIB_cfName :

    unknown

    GRIB_cfVarName :

    t2m

    GRIB_dataType :

    an

    GRIB_gridDefinitionDescription :

    Latitude/Longitude Grid

    GRIB_gridType :

    regular_ll

    GRIB_iDirectionIncrementInDegrees :

    0.25

    GRIB_iScansNegatively :

    0

    GRIB_jDirectionIncrementInDegrees :

    0.25

    GRIB_jPointsAreConsecutive :

    0

    GRIB_jScansPositively :

    0

    GRIB_latitudeOfFirstGridPointInDegrees :

    90.0

    GRIB_latitudeOfLastGridPointInDegrees :

    -90.0

    GRIB_longitudeOfFirstGridPointInDegrees :

    -180.0

    GRIB_longitudeOfLastGridPointInDegrees :

    179.75

    GRIB_missingValue :

    3.4028234663852886e+38

    GRIB_name :

    2 metre temperature

    GRIB_numberOfPoints :

    1038240

    GRIB_paramId :

    167

    GRIB_shortName :

    2t

    GRIB_stepType :

    instant

    GRIB_stepUnits :

    1

    GRIB_totalNumber :

    0

    GRIB_typeOfLevel :

    surface

    GRIB_units :

    K

    long_name :

    2 metre temperature

    standard_name :

    unknown

    units :

    K

    array([[[[[268.74626406,          nan],
              [273.08123498,          nan],
              [274.06608528,          nan],
              ...,
              [266.27550551,          nan],
              [269.12031197,          nan],
              [273.15670152,          nan]],
    
             [[268.79871227,          nan],
              [273.07942164,          nan],
              [274.05094243,          nan],
              ...,
              [266.2889379 ,          nan],
              [269.11621101,          nan],
              [273.13567238,          nan]]]]])

  • tp_nansum
    (spatial_agg, timestep, Station, time, source)
    float64
    0.0005841 nan ... 0.002068 nan

    long_name :

    Total precipitation

    units :

    m

    array([[[[[0.0005841 ,        nan],
              [0.00161747,        nan],
              [0.00372332,        nan],
              ...,
              [0.00068769,        nan],
              [0.00059458,        nan],
              [0.0020507 ,        nan]],
    
             [[0.00058223,        nan],
              [0.00160405,        nan],
              [0.00376379,        nan],
              ...,
              [0.00068897,        nan],
              [0.00060628,        nan],
              [0.00206809,        nan]]]]])

• Indexes: (5)
  • spatial_agg
    PandasIndex

    PandasIndex(Index(['polygon'], dtype='object', name='spatial_agg'))

  • timestep
    PandasIndex

    PandasIndex(Index(['D'], dtype='object', name='timestep'))

  • Station
    PandasIndex

    PandasIndex(Index(['042102', '042103'], dtype='object', name='Station'))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['1950-01-01', '1950-01-02', '1950-01-03', '1950-01-04',
                   '1950-01-05', '1950-01-06', '1950-01-07', '1950-01-08',
                   '1950-01-09', '1950-01-10',
                   ...
                   '2024-01-23', '2024-01-24', '2024-01-25', '2024-01-26',
                   '2024-01-27', '2024-01-28', '2024-01-29', '2024-01-30',
                   '2024-01-31', '2024-02-01'],
                  dtype='datetime64[ns]', name='time', length=27060, freq='D'))

  • source
    PandasIndex

    PandasIndex(Index(['era5_land_reanalysis', 'era5_reanalysis_single_levels'], dtype='object', name='source'))

• Attributes: (30)

  GRIB_NV :

  0

  GRIB_Nx :

  1440

  GRIB_Ny :

  721

  GRIB_cfName :

  unknown

  GRIB_cfVarName :

  t2m

  GRIB_dataType :

  an

  GRIB_gridDefinitionDescription :

  Latitude/Longitude Grid

  GRIB_gridType :

  regular_ll

  GRIB_iDirectionIncrementInDegrees :

  0.25

  GRIB_iScansNegatively :

  0

  GRIB_jDirectionIncrementInDegrees :

  0.25

  GRIB_jPointsAreConsecutive :

  0

  GRIB_jScansPositively :

  0

  GRIB_latitudeOfFirstGridPointInDegrees :

  90.0

  GRIB_latitudeOfLastGridPointInDegrees :

  -90.0

  GRIB_longitudeOfFirstGridPointInDegrees :

  -180.0

  GRIB_longitudeOfLastGridPointInDegrees :

  179.75

  GRIB_missingValue :

  3.4028234663852886e+38

  GRIB_name :

  2 metre temperature

  GRIB_numberOfPoints :

  1038240

  GRIB_paramId :

  167

  GRIB_shortName :

  2t

  GRIB_stepType :

  instant

  GRIB_stepUnits :

  1

  GRIB_totalNumber :

  0

  GRIB_typeOfLevel :

  surface

  GRIB_units :

  K

  long_name :

  2 metre temperature

  standard_name :

  unknown

  units :

  K

(
    xds.data[["t2m_nanmax", "t2m_nanmin"]]
    .squeeze()
    .hvplot(
        x="time",
        groupby=["Station", "source"],
        width=750,
        height=400,
        grid=True,
        widget_location="bottom",
    )
)

The resulting dataset can be explored for the total_precipitation (tp) data attribute :

(
    xds.data[["tp_nansum"]]
    .squeeze()
    .hvplot(
        x="time",
        groupby=["Station", "source"],
        width=750,
        height=400,
        grid=True,
        widget_location="bottom",
        color="blue",
    )
)

Bounding box (bbox) around polygons

The following query seeks the variable tp from the era5_land_reanalysis_dev dataset, covering the period between January 1, 1959, and December 31, 1970, for the bounding box that delimits the three polygons mentioned earlier.

Additional steps are carried out in the process, including converting to the local time zone.

query = {
    "datasets": {"era5_land_reanalysis": {"variables": ["tp"]}},
    "space": {
        "clip": "bbox",  # bbox, point or polygon
        "geometry": gdf,
    },
    "time": {
        "start": "1959-01-01",
        "end": "1970-12-31",
        "timezone": "America/Montreal",
    },
}


xds = xd.Query(**query)

xds.data

<xarray.Dataset>
Dimensions:    (time: 105192, latitude: 9, longitude: 11, source: 1)
Coordinates:
  * latitude   (latitude) float64 46.9 46.8 46.7 46.6 46.5 46.4 46.3 46.2 46.1
  * longitude  (longitude) float64 -78.9 -78.8 -78.7 -78.6 ... -78.1 -78.0 -77.9
  * time       (time) datetime64[ns] 1959-01-01 ... 1970-12-31T23:00:00
  * source     (source) <U20 'era5_land_reanalysis'
Data variables:
    tp         (time, latitude, longitude) float32 1.08e-07 1.08e-07 ... 0.0 0.0
Attributes:
    pangeo-forge:inputs_hash:  4c04980cf1e6af8df1d96abaacccdcdb7d2fcd2c8605dc...
    pangeo-forge:recipe_hash:  5fa22023abf774ec274a71534d060ea1643071fbcf580f...
    pangeo-forge:version:      0.9.4
    timezone:                  America/Montreal

xarray.Dataset

• Dimensions:
  • time: 105192
  • latitude: 9
  • longitude: 11
  • source: 1
• Coordinates: (4)
  • latitude
    (latitude)
    float64
    46.9 46.8 46.7 ... 46.3 46.2 46.1

    long_name :

    latitude

    standard_name :

    latitude

    stored_direction :

    decreasing

    units :

    degrees_north

    array([46.9, 46.8, 46.7, 46.6, 46.5, 46.4, 46.3, 46.2, 46.1])

  • longitude
    (longitude)
    float64
    -78.9 -78.8 -78.7 ... -78.0 -77.9

    long_name :

    longitude

    standard_name :

    longitude

    units :

    degrees_east

    array([-78.9, -78.8, -78.7, -78.6, -78.5, -78.4, -78.3, -78.2, -78.1, -78. ,
           -77.9])

  • time
    (time)
    datetime64[ns]
    1959-01-01 ... 1970-12-31T23:00:00

    array(['1959-01-01T00:00:00.000000000', '1959-01-01T01:00:00.000000000',
           '1959-01-01T02:00:00.000000000', ..., '1970-12-31T21:00:00.000000000',
           '1970-12-31T22:00:00.000000000', '1970-12-31T23:00:00.000000000'],
          dtype='datetime64[ns]')

  • source
    (source)
    <U20
    'era5_land_reanalysis'

    array(['era5_land_reanalysis'], dtype='<U20')

• Data variables: (1)
  • tp
    (time, latitude, longitude)
    float32
    1.08e-07 1.08e-07 ... 0.0 0.0

    GRIB_NV :

    0

    GRIB_Nx :

    1171

    GRIB_Ny :

    701

    GRIB_cfName :

    unknown

    GRIB_cfVarName :

    tp

    GRIB_dataType :

    fc

    GRIB_gridDefinitionDescription :

    Latitude/Longitude Grid

    GRIB_gridType :

    regular_ll

    GRIB_iDirectionIncrementInDegrees :

    0.1

    GRIB_iScansNegatively :

    0

    GRIB_jDirectionIncrementInDegrees :

    0.1

    GRIB_jPointsAreConsecutive :

    0

    GRIB_jScansPositively :

    0

    GRIB_latitudeOfFirstGridPointInDegrees :

    85.0

    GRIB_latitudeOfLastGridPointInDegrees :

    15.0

    GRIB_longitudeOfFirstGridPointInDegrees :

    -167.0

    GRIB_longitudeOfLastGridPointInDegrees :

    -50.0

    GRIB_missingValue :

    9999

    GRIB_name :

    Total precipitation

    GRIB_numberOfPoints :

    820871

    GRIB_paramId :

    228

    GRIB_shortName :

    tp

    GRIB_stepType :

    accum

    GRIB_stepUnits :

    1

    GRIB_totalNumber :

    0

    GRIB_typeOfLevel :

    surface

    GRIB_units :

    m

    long_name :

    Total precipitation

    standard_name :

    unknown

    units :

    m

    array([[[1.0803342e-07, 1.0803342e-07, 8.9406967e-08, ...,
             0.0000000e+00, 0.0000000e+00, 0.0000000e+00],
            [1.0803342e-07, 1.0803342e-07, 7.0780516e-08, ...,
             0.0000000e+00, 0.0000000e+00, 0.0000000e+00],
            [1.7136335e-07, 1.6018748e-07, 1.2665987e-07, ...,
             5.2154064e-08, 2.6077032e-08, 1.1175871e-08],
            ...,
            [2.5704503e-07, 2.1979213e-07, 1.8626451e-07, ...,
             1.0803342e-07, 1.0803342e-07, 9.3132257e-08],
            [3.9860606e-07, 3.5762787e-07, 3.0919909e-07, ...,
             1.2293458e-07, 1.0803342e-07, 8.9406967e-08],
            [9.7975135e-07, 7.9721212e-07, 6.4447522e-07, ...,
             1.5273690e-07, 1.1548400e-07, 7.8231096e-08]],
    
           [[2.0116568e-07, 7.4505806e-08, 3.7252903e-09, ...,
             0.0000000e+00, 0.0000000e+00, 0.0000000e+00],
            [2.7567148e-07, 1.0430813e-07, 1.0430813e-07, ...,
             0.0000000e+00, 0.0000000e+00, 0.0000000e+00],
            [7.6368451e-07, 5.6251884e-07, 4.1350722e-07, ...,
             2.3841858e-07, 2.3469329e-07, 1.9744039e-07],
    ...
            [0.0000000e+00, 0.0000000e+00, 0.0000000e+00, ...,
             0.0000000e+00, 0.0000000e+00, 0.0000000e+00],
            [0.0000000e+00, 0.0000000e+00, 0.0000000e+00, ...,
             0.0000000e+00, 0.0000000e+00, 0.0000000e+00],
            [0.0000000e+00, 0.0000000e+00, 0.0000000e+00, ...,
             0.0000000e+00, 0.0000000e+00, 0.0000000e+00]],
    
           [[0.0000000e+00, 0.0000000e+00, 0.0000000e+00, ...,
             0.0000000e+00, 0.0000000e+00, 0.0000000e+00],
            [0.0000000e+00, 0.0000000e+00, 0.0000000e+00, ...,
             0.0000000e+00, 0.0000000e+00, 0.0000000e+00],
            [0.0000000e+00, 0.0000000e+00, 0.0000000e+00, ...,
             0.0000000e+00, 0.0000000e+00, 0.0000000e+00],
            ...,
            [0.0000000e+00, 0.0000000e+00, 0.0000000e+00, ...,
             0.0000000e+00, 0.0000000e+00, 0.0000000e+00],
            [0.0000000e+00, 0.0000000e+00, 0.0000000e+00, ...,
             0.0000000e+00, 0.0000000e+00, 0.0000000e+00],
            [0.0000000e+00, 0.0000000e+00, 0.0000000e+00, ...,
             0.0000000e+00, 0.0000000e+00, 0.0000000e+00]]], dtype=float32)

• Indexes: (4)
  • latitude
    PandasIndex

    PandasIndex(Index([ 46.90000000000095,  46.80000000000095,  46.70000000000095,
           46.600000000000946, 46.500000000000945, 46.400000000000944,
            46.30000000000094,  46.20000000000094,  46.10000000000094],
          dtype='float64', name='latitude'))

  • longitude
    PandasIndex

    PandasIndex(Index([-78.90000000000501, -78.80000000000501, -78.70000000000502,
           -78.60000000000502, -78.50000000000503, -78.40000000000504,
           -78.30000000000504, -78.20000000000505, -78.10000000000505,
           -78.00000000000506, -77.90000000000506],
          dtype='float64', name='longitude'))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['1959-01-01 00:00:00', '1959-01-01 01:00:00',
                   '1959-01-01 02:00:00', '1959-01-01 03:00:00',
                   '1959-01-01 04:00:00', '1959-01-01 05:00:00',
                   '1959-01-01 06:00:00', '1959-01-01 07:00:00',
                   '1959-01-01 08:00:00', '1959-01-01 09:00:00',
                   ...
                   '1970-12-31 14:00:00', '1970-12-31 15:00:00',
                   '1970-12-31 16:00:00', '1970-12-31 17:00:00',
                   '1970-12-31 18:00:00', '1970-12-31 19:00:00',
                   '1970-12-31 20:00:00', '1970-12-31 21:00:00',
                   '1970-12-31 22:00:00', '1970-12-31 23:00:00'],
                  dtype='datetime64[ns]', name='time', length=105192, freq=None))

  • source
    PandasIndex

    PandasIndex(Index(['era5_land_reanalysis'], dtype='object', name='source'))

• Attributes: (4)

  pangeo-forge:inputs_hash :

  4c04980cf1e6af8df1d96abaacccdcdb7d2fcd2c8605dc76acd737bf010bc8b4

  pangeo-forge:recipe_hash :

  5fa22023abf774ec274a71534d060ea1643071fbcf580fb6c252a293d8b66aa7

  pangeo-forge:version :

  0.9.4

  timezone :

  America/Montreal

Let’s find out which day (24-hour period) was the rainiest in the entire region for the data retrieved in previous cell.

indexer = (
    xds.data.sel(source="era5_land_reanalysis")
    .tp.sum(["latitude", "longitude"])
    .rolling(time=24)
    .sum()
    .argmax("time")
    .values
)

xds.data.isel(time=indexer).time.dt.date.values.tolist()

datetime.date(1961, 9, 26)

Let’s visualise the evolution of the hourly precipitation during that day. Note that each image (raster) delimits exactly the bounding box required to cover all polygons in the query. Please note that for full interactivity, running the code in a Jupyter Notebook is necessary.

da = xds.data.tp.isel(time=slice(indexer - 24, indexer))
da = da.where(da > 0.0001, drop=True)

(da * 1000).squeeze().hvplot.quadmesh(
    width=750,
    height=450,
    geo=True,
    tiles="ESRI",
    groupby=["time"],
    legend="top",
    cmap="gist_ncar",
    widget_location="bottom",
    widget_type="scrubber",
    dynamic=False,
    clim=(0.01, 10),
)

Query hydrological datasets

Hydrological queries are still being tested and output format is likely to change. Stay tuned!

query = {"datasets": "deh"}
xds = xd.Query(**query)
xds.data

<xarray.Dataset>
Dimensions:        (id: 744, variable: 2, spatial_agg: 2, timestep: 1,
                    time_agg: 1, source: 1, time: 60631)
Coordinates: (12/15)
    drainage_area  (id) float32 dask.array<chunksize=(744,), meta=np.ndarray>
    end_date       (variable, id, spatial_agg, timestep, time_agg, source) datetime64[ns] dask.array<chunksize=(2, 744, 2, 1, 1, 1), meta=np.ndarray>
  * id             (id) object '010101' '010801' '010802' ... '104804' '120201'
    latitude       (id) float32 dask.array<chunksize=(744,), meta=np.ndarray>
    longitude      (id) float32 dask.array<chunksize=(744,), meta=np.ndarray>
    name           (id) object dask.array<chunksize=(744,), meta=np.ndarray>
    ...             ...
  * spatial_agg    (spatial_agg) object 'point' 'watershed'
    start_date     (variable, id, spatial_agg, timestep, time_agg, source) datetime64[ns] dask.array<chunksize=(2, 744, 2, 1, 1, 1), meta=np.ndarray>
  * time           (time) datetime64[ns] 1860-01-01 1860-01-02 ... 2025-12-31
  * time_agg       (time_agg) object 'mean'
  * timestep       (timestep) object 'D'
  * variable       (variable) object 'level' 'streamflow'
Data variables:
    level          (id, time, variable, spatial_agg, timestep, time_agg, source) float32 dask.array<chunksize=(1, 60631, 1, 1, 1, 1, 1), meta=np.ndarray>
    streamflow     (id, time, variable, spatial_agg, timestep, time_agg, source) float32 dask.array<chunksize=(1, 60631, 1, 1, 1, 1, 1), meta=np.ndarray>

xarray.Dataset

• Dimensions:
  • id: 744
  • variable: 2
  • spatial_agg: 2
  • timestep: 1
  • time_agg: 1
  • source: 1
  • time: 60631
• Coordinates: (15)
  • drainage_area
    (id)
    float32
    dask.array<chunksize=(744,), meta=np.ndarray>

    long_name :

    drainage_area

    units :

    km2

    Array Chunk Bytes 2.91 kiB 2.91 kiB Shape (744,) (744,) Dask graph 1 chunks in 2 graph layers Data type float32 numpy.ndarray

  • end_date
    (variable, id, spatial_agg, timestep, time_agg, source)
    datetime64[ns]
    dask.array<chunksize=(2, 744, 2, 1, 1, 1), meta=np.ndarray>

    Array Chunk Bytes 23.25 kiB 23.25 kiB Shape (2, 744, 2, 1, 1, 1) (2, 744, 2, 1, 1, 1) Dask graph 1 chunks in 2 graph layers Data type datetime64[ns] numpy.ndarray

  • id
    (id)
    object
    '010101' '010801' ... '120201'

    array(['010101', '010801', '010802', ..., '104803', '104804', '120201'],
          dtype=object)

  • latitude
    (id)
    float32
    dask.array<chunksize=(744,), meta=np.ndarray>

    long_name :

    latitude

    standard_name :

    latitude

    units :

    decimal_degrees

    Array Chunk Bytes 2.91 kiB 2.91 kiB Shape (744,) (744,) Dask graph 1 chunks in 2 graph layers Data type float32 numpy.ndarray

  • longitude
    (id)
    float32
    dask.array<chunksize=(744,), meta=np.ndarray>

    long_name :

    longitude

    standard_name :

    longitude

    units :

    decimal_degrees

    Array Chunk Bytes 2.91 kiB 2.91 kiB Shape (744,) (744,) Dask graph 1 chunks in 2 graph layers Data type float32 numpy.ndarray

  • name
    (id)
    object
    dask.array<chunksize=(744,), meta=np.ndarray>

    Array Chunk Bytes 5.81 kiB 5.81 kiB Shape (744,) (744,) Dask graph 1 chunks in 2 graph layers Data type object numpy.ndarray

  • province
    (id)
    object
    dask.array<chunksize=(744,), meta=np.ndarray>

    Array Chunk Bytes 5.81 kiB 5.81 kiB Shape (744,) (744,) Dask graph 1 chunks in 2 graph layers Data type object numpy.ndarray

  • regulated
    (id)
    object
    dask.array<chunksize=(744,), meta=np.ndarray>

    Array Chunk Bytes 5.81 kiB 5.81 kiB Shape (744,) (744,) Dask graph 1 chunks in 2 graph layers Data type object numpy.ndarray

  • source
    (source)
    object
    'Ministère de l’Environnement, d...

    array(['Ministère de l’Environnement, de la Lutte contre les changements climatiques, de la Faune et des Parcs'],
          dtype=object)

  • spatial_agg
    (spatial_agg)
    object
    'point' 'watershed'

    array(['point', 'watershed'], dtype=object)

  • start_date
    (variable, id, spatial_agg, timestep, time_agg, source)
    datetime64[ns]
    dask.array<chunksize=(2, 744, 2, 1, 1, 1), meta=np.ndarray>

    Array Chunk Bytes 23.25 kiB 23.25 kiB Shape (2, 744, 2, 1, 1, 1) (2, 744, 2, 1, 1, 1) Dask graph 1 chunks in 2 graph layers Data type datetime64[ns] numpy.ndarray

  • time
    (time)
    datetime64[ns]
    1860-01-01 ... 2025-12-31

    array(['1860-01-01T00:00:00.000000000', '1860-01-02T00:00:00.000000000',
           '1860-01-03T00:00:00.000000000', ..., '2025-12-29T00:00:00.000000000',
           '2025-12-30T00:00:00.000000000', '2025-12-31T00:00:00.000000000'],
          dtype='datetime64[ns]')

  • time_agg
    (time_agg)
    object
    'mean'

    array(['mean'], dtype=object)

  • timestep
    (timestep)
    object
    'D'

    array(['D'], dtype=object)

  • variable
    (variable)
    object
    'level' 'streamflow'

    array(['level', 'streamflow'], dtype=object)

• Data variables: (2)
  • level
    (id, time, variable, spatial_agg, timestep, time_agg, source)
    float32
    dask.array<chunksize=(1, 60631, 1, 1, 1, 1, 1), meta=np.ndarray>

    Array Chunk Bytes 688.32 MiB 236.84 kiB Shape (744, 60631, 2, 2, 1, 1, 1) (1, 60631, 1, 1, 1, 1, 1) Dask graph 2976 chunks in 2 graph layers Data type float32 numpy.ndarray

  • streamflow
    (id, time, variable, spatial_agg, timestep, time_agg, source)
    float32
    dask.array<chunksize=(1, 60631, 1, 1, 1, 1, 1), meta=np.ndarray>

    Array Chunk Bytes 688.32 MiB 236.84 kiB Shape (744, 60631, 2, 2, 1, 1, 1) (1, 60631, 1, 1, 1, 1, 1) Dask graph 2976 chunks in 2 graph layers Data type float32 numpy.ndarray

• Indexes: (7)
  • id
    PandasIndex

    PandasIndex(Index(['010101', '010801', '010802', '010901', '010902', '010903', '011001',
           '011002', '011003', '011201',
           ...
           '103704', '103714', '103715', '103801', '104001', '104401', '104801',
           '104803', '104804', '120201'],
          dtype='object', name='id', length=744))

  • source
    PandasIndex

    PandasIndex(Index(['Ministère de l’Environnement, de la Lutte contre les changements climatiques, de la Faune et des Parcs'], dtype='object', name='source'))

  • spatial_agg
    PandasIndex

    PandasIndex(Index(['point', 'watershed'], dtype='object', name='spatial_agg'))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['1860-01-01', '1860-01-02', '1860-01-03', '1860-01-04',
                   '1860-01-05', '1860-01-06', '1860-01-07', '1860-01-08',
                   '1860-01-09', '1860-01-10',
                   ...
                   '2025-12-22', '2025-12-23', '2025-12-24', '2025-12-25',
                   '2025-12-26', '2025-12-27', '2025-12-28', '2025-12-29',
                   '2025-12-30', '2025-12-31'],
                  dtype='datetime64[ns]', name='time', length=60631, freq=None))

  • time_agg
    PandasIndex

    PandasIndex(Index(['mean'], dtype='object', name='time_agg'))

  • timestep
    PandasIndex

    PandasIndex(Index(['D'], dtype='object', name='timestep'))

  • variable
    PandasIndex

    PandasIndex(Index(['level', 'streamflow'], dtype='object', name='variable'))

• Attributes: (0)

ds = (
    xd.Query(
        **{
            "datasets": {
                "deh": {
                    "id": ["020*"],
                    "regulated": ["Natural"],
                    "variables": ["streamflow"],
                }
            },
            "time": {"start": "1970-01-01", "minimum_duration": (15 * 365, "d")},
        }
    )
    .data.squeeze()
    .load()
)

ds

<xarray.Dataset>
Dimensions:        (id: 5, time: 20454)
Coordinates: (12/15)
    drainage_area  (id) float32 1.09e+03 647.0 59.8 626.0 1.2e+03
    end_date       (id) datetime64[ns] 2006-10-13 2024-02-03 ... 1996-08-13
  * id             (id) object '020302' '020404' '020502' '020602' '020802'
    latitude       (id) float32 48.77 48.81 48.98 48.98 49.2
    longitude      (id) float32 -64.52 -64.92 -64.43 -64.7 -65.29
    name           (id) object 'Saint' 'York' ... 'Dartmouth' 'Madeleine'
    ...             ...
    spatial_agg    <U9 'watershed'
    start_date     (id) datetime64[ns] 1989-08-12 1980-10-01 ... 1970-01-01
  * time           (time) datetime64[ns] 1970-01-01 1970-01-02 ... 2025-12-31
    time_agg       <U4 'mean'
    timestep       <U1 'D'
    variable       <U10 'streamflow'
Data variables:
    streamflow     (id, time) float32 nan nan nan nan nan ... nan nan nan nan

xarray.Dataset

• Dimensions:
  • id: 5
  • time: 20454
• Coordinates: (15)
  • drainage_area
    (id)
    float32
    1.09e+03 647.0 59.8 626.0 1.2e+03

    long_name :

    drainage_area

    units :

    km2

    array([1090. ,  647. ,   59.8,  626. , 1200. ], dtype=float32)

  • end_date
    (id)
    datetime64[ns]
    2006-10-13 ... 1996-08-13

    array(['2006-10-13T00:00:00.000000000', '2024-02-03T00:00:00.000000000',
           '1997-06-08T00:00:00.000000000', '2024-02-03T00:00:00.000000000',
           '1996-08-13T00:00:00.000000000'], dtype='datetime64[ns]')

  • id
    (id)
    object
    '020302' '020404' ... '020802'

    array(['020302', '020404', '020502', '020602', '020802'], dtype=object)

  • latitude
    (id)
    float32
    48.77 48.81 48.98 48.98 49.2

    long_name :

    latitude

    standard_name :

    latitude

    units :

    decimal_degrees

    array([48.769165, 48.806946, 48.982224, 48.97778 , 49.202778],
          dtype=float32)

  • longitude
    (id)
    float32
    -64.52 -64.92 -64.43 -64.7 -65.29

    long_name :

    longitude

    standard_name :

    longitude

    units :

    decimal_degrees

    array([-64.51583 , -64.916664, -64.42694 , -64.69972 , -65.29472 ],
          dtype=float32)

  • name
    (id)
    object
    'Saint' 'York' ... 'Madeleine'

    array(['Saint', 'York', 'Au Renard', 'Dartmouth', 'Madeleine'],
          dtype=object)

  • province
    (id)
    object
    'QC' 'QC' 'QC' 'QC' 'QC'

    array(['QC', 'QC', 'QC', 'QC', 'QC'], dtype=object)

  • regulated
    (id)
    object
    'Natural' 'Natural' ... 'Natural'

    array(['Natural', 'Natural', 'Natural', 'Natural', 'Natural'],
          dtype=object)

  • source
    ()
    <U102
    'Ministère de l’Environnement, d...

    array('Ministère de l’Environnement, de la Lutte contre les changements climatiques, de la Faune et des Parcs',
          dtype='<U102')

  • spatial_agg
    ()
    <U9
    'watershed'

    array('watershed', dtype='<U9')

  • start_date
    (id)
    datetime64[ns]
    1989-08-12 ... 1970-01-01

    array(['1989-08-12T00:00:00.000000000', '1980-10-01T00:00:00.000000000',
           '1977-07-10T00:00:00.000000000', '1970-10-01T00:00:00.000000000',
           '1970-01-01T00:00:00.000000000'], dtype='datetime64[ns]')

  • time
    (time)
    datetime64[ns]
    1970-01-01 ... 2025-12-31

    array(['1970-01-01T00:00:00.000000000', '1970-01-02T00:00:00.000000000',
           '1970-01-03T00:00:00.000000000', ..., '2025-12-29T00:00:00.000000000',
           '2025-12-30T00:00:00.000000000', '2025-12-31T00:00:00.000000000'],
          dtype='datetime64[ns]')

  • time_agg
    ()
    <U4
    'mean'

    array('mean', dtype='<U4')

  • timestep
    ()
    <U1
    'D'

    array('D', dtype='<U1')

  • variable
    ()
    <U10
    'streamflow'

    array('streamflow', dtype='<U10')

• Data variables: (1)
  • streamflow
    (id, time)
    float32
    nan nan nan nan ... nan nan nan nan

    array([[ nan,  nan,  nan, ...,  nan,  nan,  nan],
           [ nan,  nan,  nan, ...,  nan,  nan,  nan],
           [ nan,  nan,  nan, ...,  nan,  nan,  nan],
           [ nan,  nan,  nan, ...,  nan,  nan,  nan],
           [14.7, 14.3, 14. , ...,  nan,  nan,  nan]], dtype=float32)

• Indexes: (2)
  • id
    PandasIndex

    PandasIndex(Index(['020302', '020404', '020502', '020602', '020802'], dtype='object', name='id'))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['1970-01-01', '1970-01-02', '1970-01-03', '1970-01-04',
                   '1970-01-05', '1970-01-06', '1970-01-07', '1970-01-08',
                   '1970-01-09', '1970-01-10',
                   ...
                   '2025-12-22', '2025-12-23', '2025-12-24', '2025-12-25',
                   '2025-12-26', '2025-12-27', '2025-12-28', '2025-12-29',
                   '2025-12-30', '2025-12-31'],
                  dtype='datetime64[ns]', name='time', length=20454, freq=None))

• Attributes: (0)

query = {"datasets": "hydat"}
xds = xd.Query(**query)
xds.data

<xarray.Dataset>
Dimensions:        (data_type: 2, id: 7881, spatial_agg: 2, timestep: 1,
                    time_agg: 1, latitude: 2800, longitude: 4680, time: 59413)
Coordinates: (12/15)
  * data_type      (data_type) <U5 'flow' 'level'
    drainage_area  (id) float64 dask.array<chunksize=(10,), meta=np.ndarray>
    end_date       (id, data_type, spatial_agg, timestep, time_agg) object dask.array<chunksize=(7881, 2, 2, 1, 1), meta=np.ndarray>
  * id             (id) <U7 '01AA002' '01AD001' ... '11AF004' '11AF005'
  * latitude       (latitude) float64 85.0 84.97 84.95 ... 15.07 15.05 15.02
  * longitude      (longitude) float64 -167.0 -167.0 -166.9 ... -50.05 -50.02
    ...             ...
    source         (id) object dask.array<chunksize=(7881,), meta=np.ndarray>
  * spatial_agg    (spatial_agg) object 'point' 'watershed'
    start_date     (id, data_type, spatial_agg, timestep, time_agg) object dask.array<chunksize=(7881, 2, 2, 1, 1), meta=np.ndarray>
  * time           (time) datetime64[ns] 1860-01-01 1860-01-02 ... 2022-08-31
  * time_agg       (time_agg) <U4 'mean'
  * timestep       (timestep) <U3 'day'
Data variables:
    mask           (id, latitude, longitude) float64 dask.array<chunksize=(1, 500, 500), meta=np.ndarray>
    value          (id, time, data_type, spatial_agg, timestep, time_agg) float64 dask.array<chunksize=(10, 59413, 1, 1, 1, 1), meta=np.ndarray>

xarray.Dataset

• Dimensions:
  • data_type: 2
  • id: 7881
  • spatial_agg: 2
  • timestep: 1
  • time_agg: 1
  • latitude: 2800
  • longitude: 4680
  • time: 59413
• Coordinates: (15)
  • data_type
    (data_type)
    <U5
    'flow' 'level'

    array(['flow', 'level'], dtype='<U5')

  • drainage_area
    (id)
    float64
    dask.array<chunksize=(10,), meta=np.ndarray>

    Array Chunk Bytes 61.57 kiB 80 B Shape (7881,) (10,) Dask graph 789 chunks in 2 graph layers Data type float64 numpy.ndarray

  • end_date
    (id, data_type, spatial_agg, timestep, time_agg)
    object
    dask.array<chunksize=(7881, 2, 2, 1, 1), meta=np.ndarray>

    Array Chunk Bytes 246.28 kiB 246.28 kiB Shape (7881, 2, 2, 1, 1) (7881, 2, 2, 1, 1) Dask graph 1 chunks in 2 graph layers Data type object numpy.ndarray

  • id
    (id)
    <U7
    '01AA002' '01AD001' ... '11AF005'

    array(['01AA002', '01AD001', '01AD002', ..., '11AF002', '11AF004', '11AF005'],
          dtype='<U7')

  • latitude
    (latitude)
    float64
    85.0 84.97 84.95 ... 15.05 15.02

    axis :

    Y

    long_name :

    latitude

    standard_name :

    latitude

    units :

    degrees_north

    array([85.   , 84.975, 84.95 , ..., 15.075, 15.05 , 15.025])

  • longitude
    (longitude)
    float64
    -167.0 -167.0 ... -50.05 -50.02

    axis :

    X

    long_name :

    longitude

    standard_name :

    longitude

    units :

    degrees_east

    array([-167.   , -166.975, -166.95 , ...,  -50.075,  -50.05 ,  -50.025])

  • name
    (id)
    object
    dask.array<chunksize=(7881,), meta=np.ndarray>

    Array Chunk Bytes 61.57 kiB 61.57 kiB Shape (7881,) (7881,) Dask graph 1 chunks in 2 graph layers Data type object numpy.ndarray

  • province
    (id)
    object
    dask.array<chunksize=(7881,), meta=np.ndarray>

    Array Chunk Bytes 61.57 kiB 61.57 kiB Shape (7881,) (7881,) Dask graph 1 chunks in 2 graph layers Data type object numpy.ndarray

  • regulated
    (id)
    float64
    dask.array<chunksize=(10,), meta=np.ndarray>

    Array Chunk Bytes 61.57 kiB 80 B Shape (7881,) (10,) Dask graph 789 chunks in 2 graph layers Data type float64 numpy.ndarray

  • source
    (id)
    object
    dask.array<chunksize=(7881,), meta=np.ndarray>

    Array Chunk Bytes 61.57 kiB 61.57 kiB Shape (7881,) (7881,) Dask graph 1 chunks in 2 graph layers Data type object numpy.ndarray

  • spatial_agg
    (spatial_agg)
    object
    'point' 'watershed'

    array(['point', 'watershed'], dtype=object)

  • start_date
    (id, data_type, spatial_agg, timestep, time_agg)
    object
    dask.array<chunksize=(7881, 2, 2, 1, 1), meta=np.ndarray>

    Array Chunk Bytes 246.28 kiB 246.28 kiB Shape (7881, 2, 2, 1, 1) (7881, 2, 2, 1, 1) Dask graph 1 chunks in 2 graph layers Data type object numpy.ndarray

  • time
    (time)
    datetime64[ns]
    1860-01-01 ... 2022-08-31

    array(['1860-01-01T00:00:00.000000000', '1860-01-02T00:00:00.000000000',
           '1860-01-03T00:00:00.000000000', ..., '2022-08-29T00:00:00.000000000',
           '2022-08-30T00:00:00.000000000', '2022-08-31T00:00:00.000000000'],
          dtype='datetime64[ns]')

  • time_agg
    (time_agg)
    <U4
    'mean'

    array(['mean'], dtype='<U4')

  • timestep
    (timestep)
    <U3
    'day'

    array(['day'], dtype='<U3')

• Data variables: (2)
  • mask
    (id, latitude, longitude)
    float64
    dask.array<chunksize=(1, 500, 500), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 769.44 GiB 1.91 MiB Shape (7881, 2800, 4680) (1, 500, 500) Dask graph 472860 chunks in 2 graph layers Data type float64 numpy.ndarray

  • value
    (id, time, data_type, spatial_agg, timestep, time_agg)
    float64
    dask.array<chunksize=(10, 59413, 1, 1, 1, 1), meta=np.ndarray>

    Array Chunk Bytes 13.95 GiB 4.53 MiB Shape (7881, 59413, 2, 2, 1, 1) (10, 59413, 1, 1, 1, 1) Dask graph 3156 chunks in 2 graph layers Data type float64 numpy.ndarray

• Indexes: (8)
  • data_type
    PandasIndex

    PandasIndex(Index(['flow', 'level'], dtype='object', name='data_type'))

  • id
    PandasIndex

    PandasIndex(Index(['01AA002', '01AD001', '01AD002', '01AD003', '01AD004', '01AD005',
           '01AD008', '01AD009', '01AD012', '01AD013',
           ...
           '11AE012', '11AE013', '11AE014', '11AE015', '11AE016', '11AE018',
           '11AF001', '11AF002', '11AF004', '11AF005'],
          dtype='object', name='id', length=7881))

  • latitude
    PandasIndex

    PandasIndex(Index([              85.0,             84.975,  84.94999999999999,
            84.92499999999998,  84.89999999999998,  84.87499999999997,
            84.84999999999997,  84.82499999999996,  84.79999999999995,
            84.77499999999995,
           ...
            15.24999999998414, 15.224999999984135,  15.19999999998413,
           15.174999999984124, 15.149999999984118, 15.124999999984112,
           15.099999999984107, 15.074999999984101, 15.049999999984095,
            15.02499999998409],
          dtype='float64', name='latitude', length=2800))

  • longitude
    PandasIndex

    PandasIndex(Index([             -167.0,            -166.975,             -166.95,
           -166.92499999999998, -166.89999999999998, -166.87499999999997,
           -166.84999999999997, -166.82499999999996, -166.79999999999995,
           -166.77499999999995,
           ...
           -50.249999999973454,  -50.22499999997345,  -50.19999999997344,
            -50.17499999997344,  -50.14999999997343, -50.124999999973426,
            -50.09999999997342, -50.074999999973414,  -50.04999999997341,
             -50.0249999999734],
          dtype='float64', name='longitude', length=4680))

  • spatial_agg
    PandasIndex

    PandasIndex(Index(['point', 'watershed'], dtype='object', name='spatial_agg'))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['1860-01-01', '1860-01-02', '1860-01-03', '1860-01-04',
                   '1860-01-05', '1860-01-06', '1860-01-07', '1860-01-08',
                   '1860-01-09', '1860-01-10',
                   ...
                   '2022-08-22', '2022-08-23', '2022-08-24', '2022-08-25',
                   '2022-08-26', '2022-08-27', '2022-08-28', '2022-08-29',
                   '2022-08-30', '2022-08-31'],
                  dtype='datetime64[ns]', name='time', length=59413, freq=None))

  • time_agg
    PandasIndex

    PandasIndex(Index(['mean'], dtype='object', name='time_agg'))

  • timestep
    PandasIndex

    PandasIndex(Index(['day'], dtype='object', name='timestep'))

• Attributes: (0)