dtype is ignored if _FillValue in encoding is provided for xr.to_zarr()?

[mliukis]

I am trying to save data of np.uint32 type to Zarr store:

import numpy as np
import os
import shutil
import xarray as xr

values = np.full((3,6), 50, dtype=np.uint32)

var_name = 'test'
ds = xr.Dataset(data_vars={var_name: xr.DataArray(data=values)})
print(f'dtype in xr.Dataset: {ds[var_name].dtype}')

# dtype of uint32 is not preserved when written to Zarr if _FillValue is provided: trying to force _FillValue to be of np.uint32 type does not help
encodings_dtype_fillvalue = {var_name: {'_FillValue': np.uint(0), 'dtype': np.uint32}}
encodings_dtype = {var_name: {'dtype': np.uint32}}

test_file = 'test.zarr'

for each_encoding in [encodings_dtype_fillvalue, encodings_dtype]:
    print(f'Using encoding: {each_encoding}')
    _ = ds.to_zarr(test_file, encoding=each_encoding, consolidated=True)

    # Read back in and verify the type
    with xr.open_zarr(test_file, decode_timedelta=False, consolidated=True) as ds_from_zarr:
        print(f'--->Type of data from zarr: {ds_from_zarr[var_name].dtype}')

    # Cleanup
    if os.path.exists(test_file):
        shutil.rmtree(test_file)

It seems that the dtype of the data is ignored on output when I provide _FillValue setting for encoding thus "transforming" data to float64 type instead of requested uint32 type. Please see example above which produces output:

dtype in xr.Dataset: uint32
Using encoding: {'test': {'_FillValue': 0, 'dtype': <class 'numpy.uint32'>}}
--->Type of data from zarr: float64
Using encoding: {'test': {'dtype': <class 'numpy.uint32'>}}
--->Type of data from zarr: uint32

Does anybody have a suggestions on how to specify the _FillValue without messing up the type of stored data in Zarr format? Or is it a bug in xarray?
Any help or suggestion is much appreciated!
Thanks in advance.

[dcherian]

Why are you specifying _FillValue for integer data?

In xarray, the missing_value and _FillValue are interpreted in the same way. So it is usually used as a sentinel value for NaNs in float data.

Here's the code:

xarray/xarray/coding/variables.py

Lines 165 to 170 in 8417f49

	if fv is not None:
	# Ensure _FillValue is cast to same dtype as data's
	encoding["_FillValue"] = dtype.type(fv)
	fill_value = pop_to(encoding, attrs, "_FillValue", name=name)
	if not pd.isnull(fill_value):
	data = duck_array_ops.fillna(data, fill_value)

[mliukis]

Thank you for the code snippet! I guess, we will just set no-data values to zeros and omit _FillValue in such case.

We usually specify _FillValue to populate uninitialized portions of the array. We have not had any issues with providing _FillValue to other than float data in NetCDF format.