i'm reading bunch of netcdf files using pupynere interface (linux). following code results in mmap error:
import numpy np import os, glob pupynere import netcdffile nc alts = [] vals = [] path='coll_mip' filter='*.nc' infile in glob.glob(os.path.join(path, filter)): curdata = nc(infile,'r') vals.append(curdata.variables['o3.mixing.ratio'][:]) alts.append(curdata.variables['altitude'][:]) curdata.close()
error:
$ python2.7 /mnt/grid/src/profile/contra.py traceback (most recent call last): file "/mnt/grid/src/profile/contra.py", line 15, in <module> file "/usr/lib/python2.7/site-packages/pupynere-1.0.13-py2.7.egg/pupynere.py", line 159, in __init__ file "/usr/lib/python2.7/site-packages/pupynere-1.0.13-py2.7.egg/pupynere.py", line 386, in _read file "/usr/lib/python2.7/site-packages/pupynere-1.0.13-py2.7.egg/pupynere.py", line 446, in _read_var_array mmap.error: [errno 24] many open files
interestingly, if comment 1 of append
commands (either do!) works! doing wrong? i'm closing file, right? somehow related python list. used different, inefficient approach before (always copying each element) worked.
ps: ulimit -n
yields 1024, program fails @ file number 498.
maybe related to, solution doesn't work me: numpy , memmap: [errno 24] many open files
my guess mmap.mmap call in pupynere holding file descriptor open (or creating new one). if this:
vals.append(curdata.variables['o3.mixing.ratio'][:].copy()) alts.append(curdata.variables['altitude'][:].copy())
Comments
Post a Comment