Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nasty segfault with erroneous netcdf-4 data #444

Open
durack1 opened this issue Mar 16, 2022 · 0 comments
Open

Nasty segfault with erroneous netcdf-4 data #444

durack1 opened this issue Mar 16, 2022 · 0 comments

Comments

@durack1
Copy link
Member

durack1 commented Mar 16, 2022

Describe the bug
I am able to reproduce a segfault when attempting to open a poorly formed netcdf-4 file. I would expect cdms to throw an OSError or similar error, rather than segfaulting

To Reproduce
Steps to reproduce the behavior:
The below is a grab of an ipython console output:

In [5]: getGlobalAtts(f)

getGlobalAtts: badFile: /p/css03/esgf_publish/CMIP6/ScenarioMIP/CCCma/CanESM5/ssp126/r5i1p1f1/Omon/tauvo/gn/v20190306/tauvo_Omon_CanESM5_ssp126_r5i1p1f1_gn_201501-210012.nc
Error: NetCDF: HDF error

ncopen: filename "/p/css03/esgf_publish/CMIP6/ScenarioMIP/CCCma/CanESM5/ssp126/r5i1p1f1/Omon/tauvo/gn/v20190306/tauvo_Omon_CanESM5_ssp126_r5i1p1f1_gn_201501-210012.nc": NetCDF: HDF error
Out[5]: 
['/p/css03/esgf_publish/CMIP6/ScenarioMIP/CCCma/CanESM5/ssp126/r5i1p1f1/Omon/tauvo/gn/v20190306/tauvo_Omon_CanESM5_ssp126_r5i1p1f1_gn_201501-210012.nc',
 OSError('NetCDF: HDF error')]

In [6]: getGlobalAtts(f)

Output from spyder call 'get_namespace_view':
*** Error in `~/anaconda3/envs/cdms315spy515cart020/bin/python': free(): invalid pointer: 0x00007fd568424bf0 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x81329)[0x7fd570980329]
~anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(H5MM_xfree+0xf)[0x7fd4f9222dff]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(+0x1aed3a)[0x7fd4f9251d3a]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(+0x1aeeb2)[0x7fd4f9251eb2]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(H5O_msg_reset+0x74)[0x7fd4f9252ca4]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(H5G__link_release_table+0x58)[0x7fd4f91db368]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(H5G__dense_iterate+0x14f)[0x7fd4f91d1ecf]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(H5G__obj_iterate+0xd2)[0x7fd4f91e2b32]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(H5G_iterate+0x148)[0x7fd4f91d9b28]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(+0x16f5ab)[0x7fd4f92125ab]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(H5Literate+0xca)[0x7fd4f9213f4a]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../../libnetcdf.so.19(+0x945de)[0x7fd5022b85de]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../../libnetcdf.so.19(+0x976ba)[0x7fd5022bb6ba]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../../libnetcdf.so.19(NC_open+0x238)[0x7fd502254618]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../../libnetcdf.so.19(nc_open+0x15)[0x7fd5022547b5]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../../libnetcdf.so.19(ncopen+0x24)[0x7fd502273b14]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/Cdunif.cpython-39-x86_64-linux-gnu.so(+0x7689)[0x7fd5023b3689]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/Cdunif.cpython-39-x86_64-linux-gnu.so(+0x931a)[0x7fd5023b531a]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/Cdunif.cpython-39-x86_64-linux-gnu.so(+0xe67e)[0x7fd5023ba67e]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x15fd78)[0x558c65f90d78]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyObject_MakeTpCall+0x316)[0x558c65f776a6]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x534c)[0x558c66015f2c]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x1390d0)[0x558c65f6a0d0]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x1ac3c7)[0x558c65fdd3c7]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyObject_MakeTpCall+0x3c2)[0x558c65f77752]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x4e0c)[0x558c660159ec]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x1390d0)[0x558c65f6a0d0]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyFunction_Vectorcall+0x336)[0x558c65fd1366]
~/anaconda3/envs/cdms315spy515cart020/bin/python(PyObject_Call+0x106)[0x558c65f963a6]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyObject_MakeTpCall+0x316)[0x558c65f776a6]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x534c)[0x558c66015f2c]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x139900)[0x558c65f6a900]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyFunction_Vectorcall+0x336)[0x558c65fd1366]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x3b9)[0x558c66010f99]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x1390d0)[0x558c65f6a0d0]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalCodeWithName+0x47)[0x558c6604f667]
~/anaconda3/envs/cdms315spy515cart020/bin/python(PyEval_EvalCodeEx+0x39)[0x558c6604f6a9]
~/anaconda3/envs/cdms315spy515cart020/bin/python(PyEval_EvalCode+0x1b)[0x558c6604f6cb]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x2930ad)[0x558c660c40ad]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x19972e)[0x558c65fca72e]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x3b9)[0x558c66010f99]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x167055)[0x558c65f98055]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x5179)[0x558c66015d59]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x167055)[0x558c65f98055]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x5179)[0x558c66015d59]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x167055)[0x558c65f98055]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x19a579)[0x558c65fcb579]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x603)[0x558c660111e3]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyFunction_Vectorcall+0x19a)[0x558c65fd11ca]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x3b9)[0x558c66010f99]

Restarting kernel...

Expected behavior
I would expect and OSError to be thrown, and no segfault

Screenshots or traceback
If applicable, add screenshots/traceback to help explain your problem.

Desktop (please complete the following information):

  • OS: RHEL 7.9
  • Browser NA
  • Version cdms3.1.5

Environment Information

`conda info`

(cdms315spy515cart020) bash-4.2$ conda info
                                                                                                                                                                                                                    
     active environment : cdms315spy515cart020                                                                                                                                                                      
    active env location : ~/anaconda3/envs/cdms315spy515cart020                                                                                                                                         
            shell level : 3                                                                                                                                                                                         
       user config file : ~/.condarc                                                                                                                                                                    
 populated config files :                                                                                                                                                                                           
          conda version : 4.12.0                                                                                                                                                                                    
    conda-build version : 3.21.4                                                                                                                                                                                    
         python version : 3.8.8.final.0                                                                                                                                                                             
       virtual packages : __linux=3.10.0=0                                                                                                                                                                          
                          __glibc=2.17=0                                                                                                                                                                            
                          __unix=0=0                                                                                                                                                                                
                          __archspec=1=x86_64                                                                                                                                                                       
       base environment : ~/anaconda3  (writable)                                                                                                                                                       
      conda av data dir : ~/anaconda3/etc/conda                                                                                                                                                         
  conda av metadata url : None                                                                                                                                                                                      
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64                                                                                                                                              
                          https://repo.anaconda.com/pkgs/main/noarch                                                                                                                                                
                          https://repo.anaconda.com/pkgs/r/linux-64                                                                                                                                                 
                          https://repo.anaconda.com/pkgs/r/noarch                                                                                                                                                   
          package cache : ~/anaconda3/pkgs                                                                                                                                                              
                          ~/.conda/pkgs                                                                                                                                                                 
       envs directories : ~/anaconda3/envs                                                                                                                                                              
                          ~/.conda/envs                                                                                                                                                                 
               platform : linux-64                                                                                                                                                                                  
             user-agent : conda/4.12.0 requests/2.27.1 CPython/3.8.8 Linux/3.10.0-1160.42.2.el7.x86_64 rhel/7.9 glibc/2.17                                                                                          
                UID:GID : 40336:1026                                                                                                                                                                                
             netrc file : None                                                                                                                                                                                      
           offline mode : False

`conda config --show-sources`


`conda list --show-channel-urls`


Additional context
Add any other context about the problem here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant