Skip to content

Cannot use xarrays own times for indexing #1240

@gerritholl

Description

@gerritholl

I need to get the first Δt from the start of my dataset, i.e. ds.sel(start_time, start_time + timedelta). However, due to pandas using M8[ns] but datetime.datetime not supporting this, the index gets converted to an int and indexing fails. Inspection tells me that by the time the index reaches pandas it is already an int. This is ultimately due to the numpy problem that timedelta64(0, 'ns').item() is an int, but it would be very nice if xarray had a workaround so that we can use indexing such as shown below.

In [282]: time = pd.date_range('2000-01-01', freq='H', periods=365 * 24)

In [283]: ds = xarray.Dataset({'foo': ('time', np.arange(365 * 24)), 'time': time})

In [284]: ds.sel(time=slice(ds["time"][0], ds["time"][10]))
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-284-a101e126e1b0> in <module>()
----> 1 ds.sel(time=slice(ds["time"][0], ds["time"][10]))

/dev/shm/gerrit/venv/stable-3.5/lib/python3.5/site-packages/xarray/core/dataset.py in sel(self, method, tolerance, drop, **indexers)
   1180         """
   1181         pos_indexers, new_indexes = indexing.remap_label_indexers(
-> 1182             self, indexers, method=method, tolerance=tolerance
   1183         )
   1184         result = self.isel(drop=drop, **pos_indexers)

/dev/shm/gerrit/venv/stable-3.5/lib/python3.5/site-packages/xarray/core/indexing.py in remap_label_indexers(data_obj, indexers, method, tolerance)
    286         else:
    287             idxr, new_idx = convert_label_indexer(index, label,
--> 288                                                   dim, method, tolerance)
    289             pos_indexers[dim] = idxr
    290             if new_idx is not None:

/dev/shm/gerrit/venv/stable-3.5/lib/python3.5/site-packages/xarray/core/indexing.py in convert_label_indexer(index, label, index_name, method, tolerance)
    183         indexer = index.slice_indexer(_try_get_item(label.start),
    184                                       _try_get_item(label.stop),
--> 185                                       _try_get_item(label.step))
    186         if not isinstance(indexer, slice):
    187             # unlike pandas, in xarray we never want to silently convert a slice

/dev/shm/gerrit/venv/stable-3.5/lib/python3.5/site-packages/pandas/tseries/index.py in slice_indexer(self, start, end, step, kind)
   1496 
   1497         try:
-> 1498             return Index.slice_indexer(self, start, end, step, kind=kind)
   1499         except KeyError:
   1500             # For historical reasons DatetimeIndex by default supports

/dev/shm/gerrit/venv/stable-3.5/lib/python3.5/site-packages/pandas/indexes/base.py in slice_indexer(self, start, end, step, kind)
   2995         """
   2996         start_slice, end_slice = self.slice_locs(start, end, step=step,
-> 2997                                                  kind=kind)
   2998 
   2999         # return a slice

/dev/shm/gerrit/venv/stable-3.5/lib/python3.5/site-packages/pandas/indexes/base.py in slice_locs(self, start, end, step, kind)
   3174         start_slice = None
   3175         if start is not None:
-> 3176             start_slice = self.get_slice_bound(start, 'left', kind)
   3177         if start_slice is None:
   3178             start_slice = 0

/dev/shm/gerrit/venv/stable-3.5/lib/python3.5/site-packages/pandas/indexes/base.py in get_slice_bound(self, label, side, kind)
   3113         # For datetime indices label may be a string that has to be converted
   3114         # to datetime boundary according to its resolution.
-> 3115         label = self._maybe_cast_slice_bound(label, side, kind)
   3116 
   3117         # we need to look up the label

/dev/shm/gerrit/venv/stable-3.5/lib/python3.5/site-packages/pandas/tseries/index.py in _maybe_cast_slice_bound(self, label, side, kind)
   1444 
   1445         if is_float(label) or isinstance(label, time) or is_integer(label):
-> 1446             self._invalid_indexer('slice', label)
   1447 
   1448         if isinstance(label, compat.string_types):

/dev/shm/gerrit/venv/stable-3.5/lib/python3.5/site-packages/pandas/indexes/base.py in _invalid_indexer(self, form, key)
   1282                         "indexers [{key}] of {kind}".format(
   1283                             form=form, klass=type(self), key=key,
-> 1284                             kind=type(key)))
   1285 
   1286     def get_duplicates(self):

TypeError: cannot do slice indexing on <class 'pandas.tseries.index.DatetimeIndex'> with these indexers [946684800000000000] of <class 'int'>

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions