linux - python mmap.error: Too many open files. What's wrong? -


i'm reading bunch of netcdf files using pupynere interface (linux). following code results in mmap error:

import numpy np import os, glob pupynere import netcdffile nc alts = [] vals = [] path='coll_mip' filter='*.nc' infile in glob.glob(os.path.join(path, filter)):         curdata = nc(infile,'r')         vals.append(curdata.variables['o3.mixing.ratio'][:])         alts.append(curdata.variables['altitude'][:])         curdata.close() 

error:

$ python2.7 /mnt/grid/src/profile/contra.py traceback (most recent call last):   file "/mnt/grid/src/profile/contra.py", line 15, in <module>   file "/usr/lib/python2.7/site-packages/pupynere-1.0.13-py2.7.egg/pupynere.py", line 159, in __init__   file "/usr/lib/python2.7/site-packages/pupynere-1.0.13-py2.7.egg/pupynere.py", line 386, in _read   file "/usr/lib/python2.7/site-packages/pupynere-1.0.13-py2.7.egg/pupynere.py", line 446, in _read_var_array mmap.error: [errno 24] many open files 

interestingly, if comment 1 of append commands (either do!) works! doing wrong? i'm closing file, right? somehow related python list. used different, inefficient approach before (always copying each element) worked.

ps: ulimit -n yields 1024, program fails @ file number 498.

maybe related to, solution doesn't work me: numpy , memmap: [errno 24] many open files

my guess mmap.mmap call in pupynere holding file descriptor open (or creating new one). if this:

vals.append(curdata.variables['o3.mixing.ratio'][:].copy()) alts.append(curdata.variables['altitude'][:].copy()) 

Comments