Invalid annotation error when trying to read .nwb file

[Idavr]

Greetings everyone!
I hope someone here can help with solving a very basic issue.

Currently I am trying to read a .nwb file using NWBIO like this:

from pynwb import NWBFile, NWBHDF5IO
from neo.io import NWBIO

file = '/745606_20231213-probe0.nwb'

reader = NWBIO(file)
data_neo = reader.read()
print(data_neo)

/home/idavalik/anaconda3/envs/elephant/lib/python3.8/site-packages/hdmf/utils.py:668: UserWarning: Ignoring cached namespace 'hdmf-common' version 1.2.0 because version 1.8.0 is already loaded.
  return func(args[0], **pargs)
/home/idavalik/anaconda3/envs/elephant/lib/python3.8/site-packages/hdmf/utils.py:668: UserWarning: Ignoring cached namespace 'core' version 2.2.5 because version 2.6.0-alpha is already loaded.
 return func(args[0], **pargs)

Traceback (most recent call last):
  File "/home/idavalik/Documents/elephant/elephant/assembly_detection.py", line 18, in <module>
    data_neo = reader.read()
  File "/home/idavalik/Documents/elephant/elephant/neo/io/baseio.py", line 133, in read
    return self.read_all_blocks(lazy=lazy, **kargs)
  File "/home/idavalik/Documents/elephant/elephant/neo/io/nwbio.py", line 271, in read_all_blocks
    self._read_stimulus_group(lazy)
  File "/home/idavalik/Documents/elephant/elephant/neo/io/nwbio.py", line 390, in _read_stimulus_group
    self._read_timeseries_group("stimulus", lazy)
  File "/home/idavalik/Documents/elephant/elephant/neo/io/nwbio.py", line 351, in _read_timeseries_group
    segment = self._get_segment(block_name, segment_name)
  File "/home/idavalik/Documents/elephant/elephant/neo/io/nwbio.py", line 291, in _get_segment
    block = Block(name=block_name, **self.global_block_metadata)
  File "/home/idavalik/Documents/elephant/elephant/neo/core/block.py", line 96, in __init__
    super().__init__(name=name, description=description, file_origin=file_origin, **annotations)
  File "/home/idavalik/Documents/elephant/elephant/neo/core/container.py", line 204, in __init__
    super().__init__(name=name, description=description, file_origin=file_origin, **annotations)
  File "/home/idavalik/Documents/elephant/elephant/neo/core/baseneo.py", line 273, in __init__
    _check_annotations(annotations)
  File "/home/idavalik/Documents/elephant/elephant/neo/core/baseneo.py", line 49, in _check_annotations
    _check_annotations(element)
  File "/home/idavalik/Documents/elephant/elephant/neo/core/baseneo.py", line 54, in _check_annotations
    raise ValueError(f"Invalid annotation. Annotations of type {type(value)} are not" f"allowed")
ValueError: Invalid annotation. Annotations of type <class 'h5py._hl.dataset.Dataset'> are notallowed

Environment:

• OS: Linux/Ubuntu
• Python version: 3.8
• Neo version: 0.13.1.dev0
• NumPy version: 1.24.3
• pynwb version: 2.5.0

I saw some previous issues with reading .nwb files, but not with this exact ValueError. Any assistance in very much appreciated!

Best,

Idavr

[tjr1]

I also can't read any .nwb files with neo due to this error. @CodyCBakerPhD , might you have an idea on this one?

[Moritz-Alexander-Kern]

I encountered the same issue

The .nwb file is publicly available, here:
https://dandiarchive.org/dandiset/000473/0.230417.1502/files?location=sub-216301&page=1

Code to reproduce:

from neo.io import NWBIO

filepath = "sub-216301_ses-216301-20200521-probe0_ecephys+ogen.nwb"
io = NWBIO(f"{filepath}",'r')
block = io.read_block(block_index=1)

Output:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[3], line 3
      1 filepath = "sub-216301_ses-216301-20200521-probe0_ecephys+ogen.nwb"
      2 io = NWBIO(f"{filepath}",'r')
----> 3 block = io.read_block(block_index=1)

File ~/DANDI/.venv/lib/python3.10/site-packages/neo/io/nwbio.py:287, in NWBIO.read_block(self, lazy, block_index, **kargs)
    283 def read_block(self, lazy=False, block_index=0, **kargs):
    284     """
    285     Load the first block in the file.
    286     """
--> 287     return self.read_all_blocks(lazy=lazy)[block_index]

File ~/DANDI/.venv/lib/python3.10/site-packages/neo/io/nwbio.py:277, in NWBIO.read_all_blocks(self, lazy, **kwargs)
    275 self._blocks = {}
    276 self._read_acquisition_group(lazy=lazy)
--> 277 self._read_stimulus_group(lazy)
    278 self._read_units(lazy=lazy)
    279 self._read_epochs_group(lazy)

File ~/DANDI/.venv/lib/python3.10/site-packages/neo/io/nwbio.py:397, in NWBIO._read_stimulus_group(self, lazy)
    396 def _read_stimulus_group(self, lazy):
--> 397     self._read_timeseries_group("stimulus", lazy)

File ~/DANDI/.venv/lib/python3.10/site-packages/neo/io/nwbio.py:358, in NWBIO._read_timeseries_group(self, group_name, lazy)
    356     block_name = hierarchy["block"]
    357     segment_name = hierarchy["segment"]
--> 358 segment = self._get_segment(block_name, segment_name)
    359 if isinstance(timeseries, pynwb.misc.AnnotationSeries):
    360     event = EventProxy(timeseries, group_name)

File ~/DANDI/.venv/lib/python3.10/site-packages/neo/io/nwbio.py:297, in NWBIO._get_segment(self, block_name, segment_name)
    295     block = self._blocks[block_name]
    296 else:
--> 297     block = Block(name=block_name, **self.global_block_metadata)
    298     self._blocks[block_name] = block
    299 segment = None

File ~/DANDI/.venv/lib/python3.10/site-packages/neo/core/block.py:96, in Block.__init__(self, name, description, file_origin, file_datetime, rec_datetime, index, **annotations)
     83 def __init__(
     84     self,
     85     name=None,
   (...)
     91     **annotations,
     92 ):
     93     """
     94     Initialize a new :class:`Block` instance.
     95     """
---> 96     super().__init__(name=name, description=description, file_origin=file_origin, **annotations)
     98     self.file_datetime = file_datetime
     99     self.rec_datetime = rec_datetime

File ~/DANDI/.venv/lib/python3.10/site-packages/neo/core/container.py:204, in Container.__init__(self, name, description, file_origin, **annotations)
    200 def __init__(self, name=None, description=None, file_origin=None, **annotations):
    201     """
    202     Initialize a new :class:`Container` instance.
    203     """
--> 204     super().__init__(name=name, description=description, file_origin=file_origin, **annotations)

File ~/DANDI/.venv/lib/python3.10/site-packages/neo/core/baseneo.py:301, in BaseNeo.__init__(self, name, description, file_origin, **annotations)
    293 """
    294 This is the base constructor for all Neo objects.
    295 
   (...)
    298 :class:`BaseNeo` or the child class.
    299 """
    300 # create `annotations` for additional arguments
--> 301 _check_annotations(annotations)
    302 self.annotations = annotations
    304 # these attributes are recommended for all objects.

File ~/DANDI/.venv/lib/python3.10/site-packages/neo/core/baseneo.py:74, in _check_annotations(value)
     72         if not isinstance(key, str):
     73             raise TypeError(f"Annotations keys must be strings not of type {type(key)}")
---> 74         _check_annotations(element)
     75 elif isinstance(value, (list, tuple)):
     76     for element in value:

File ~/DANDI/.venv/lib/python3.10/site-packages/neo/core/baseneo.py:79, in _check_annotations(value)
     77         _check_annotations(element)
     78 elif not isinstance(value, ALLOWED_ANNOTATION_TYPES):
---> 79     raise ValueError(f"Invalid annotation. Annotations of type {type(value)} are not" f"allowed")

ValueError: Invalid annotation. Annotations of type <class 'hdmf.utils.StrDataset'> are notallowed

With:

OS: Ubuntu 22.04.4 LTS
Python 3.10.12

Package                   Version
------------------------- -----------
h5py                      3.11.0
hdmf                      3.14.0
neo                       0.13.1
numpy                     1.26.4
pynwb                     2.8.0
quantities                0.15.0

This is related to: NeuralEnsemble/elephant#627

[apdavison]

Hi everyone, sorry for the delay in starting to address this. Please try to read the file using the code in the linked pull request (#1502). That code allows me to read the test file from DANDI, but there are many warnings, so I suspect there are still problems to fix, and I will look into this further in a couple of weeks. All feedback is welcome!