Python: subprocess memory leak -


i want run serial program on multiple cores @ same time , need multiple time (in loop). use subprocess.popen distribute jobs on processors limiting number of jobs number of available processors. add jobs list , check poll() if jobs done, remove them list , continue submission until total number of jobs completed.

i have been looking on web , found couple of interesting scripts , came out adapted version:

nextproc = 0 processes = [] while (len(processes) < limitproc):     # here assume limitproc < ncores   input = filelist[nextproc]+'.in'      # filelist: list of input file   output = filelist[nextproc]+'.out'    # list of output file   cwd = pathlist[nextproc]              # list of paths   processes.append(subprocess.popen(['myprogram','-i',input,'-screen',output],cwd=cwd,bufsize=-1))   nextproc += 1   time.sleep(wait)  while (len(processes) > 0):                     # loop until processes done   time.sleep(wait)   in xrange(len(processes)-1, -1, -1):    # remove processes done (traverse backward)      if processes[i].poll() not none:       del processes[i]     time.sleep(wait)    while (len(processes) < limitproc) , (nextproc < maxprocesses):    # submit new processes     output = filelist[nextproc]+'.out'     input = filelist[nextproc]+'.in'     cwd = pathlist[nextproc]     processes.append(subprocess.popen(['myprogram','-i',input,'-screen',output],cwd=cwd,bufsize=-1))     nextproc += 1     time.sleep(wait)  print 'jobs done' 

i run script in loop , problem execution time increases 1 step another. here graph: http://i62.tinypic.com/2lk8f41.png

myprogram time execution constant. i'd glad if explain me causing leak.

thanks lot, begbi


Comments

Popular posts from this blog

javascript - RequestAnimationFrame not working when exiting fullscreen switching space on Safari -

Python ctypes access violation with const pointer arguments -