python 2.7 - Scons AttributeError: 'builtin_function_or_method' object has no attribute 'dispatch' -


i have sconstruct script instantiating object.this object internally calls method run multiprocessing module. example shown below

this object before calling function unpickles file , pass inputs multiprocessing module.

def run_scons(self,inpfile,outfile):          # unpickle input parameter         fid=open(inpfile,'rb')         input_data=pkls.load(fid)         my_results=[]         #run solver in loop         my_data in input_data:             work_ers=len(my_data)             pool = pool(processes=work_ers)             a_result=pool.map_async(my_solver, my_data)             pool.close()             pool.join()             my_results.append(a_result.get())         fid.close()          fid_out=open(outfile,'wb+')         pkls.dump(rot_full_results,fid_out) 

i following error when executing same function via scons.

pool = pool(processes=work_ers)   file "c:\python27\lib\multiprocessing\__init__.py", line 232, in pool     return pool(processes, initializer, initargs, maxtasksperchild)   file "c:\python27\lib\multiprocessing\pool.py", line 138, in __init__     self._setup_queues()   file "c:\python27\lib\multiprocessing\pool.py", line 232, in _setup_queues     .queues import simplequeue   file "c:\python27\lib\multiprocessing\queues.py", line 48, in <module>     multiprocessing.synchronize import lock, boundedsemaphore, semaphore, condition   file "c:\python27\lib\multiprocessing\synchronize.py", line 48, in <module>     multiprocessing.forking import assert_spawning, popen   file "c:\python27\lib\multiprocessing\forking.py", line 60, in <module>     class forkingpickler(pickler):   file "c:\python27\lib\multiprocessing\forking.py", line 61, in forkingpickler     dispatch = pickler.dispatch.copy() attributeerror: 'builtin_function_or_method' object has no attribute 'dispatch' scons: building terminated because of errors. 

after reading error, found out scons has hack around renames pickle module cpickle, whereas multiprocessing module looking cpickle , goes down. there way around this?

i using multiprocessing process 'job' files in parallel. doing in sconstruct file:

if getoption("run_jobs"):      my_jobs = # code produces list of job objects      ecode = build_support.mp_run_jobs(my_jobs)      exit(ecode) 

but ran exception above:

attributeerror: 'builtin_function_or_method' object has no attribute 'dispatch' 

on linux using 'scons_horrible_regression_test_hack' works:

$ scons_horrible_regression_test_hack=1 scons --run-jobs 

but don't want have export symbol on linux , windows , mac etc.

i did find workaround, delete pickle modules , re-import them this:

if getoption("run_jobs"):      # workaround scons.compat module renaming      import imp      del sys.modules['pickle']     del sys.modules['cpickle']      sys.modules['pickle'] = imp.load_module('pickle', *imp.find_module('pickle'))     sys.modules['cpickle'] = imp.load_module('cpickle', *imp.find_module('cpickle'))      import pickle     import cpickle      print "(pickle == cpickle) = ", (pickle == cpickle)      my_jobs = # code produces list of job objects      ecode = build_support.mp_run_jobs(my_jobs)      exit(ecode) 

now multiprocessing of jobs works expected!


Comments

Popular posts from this blog

javascript - RequestAnimationFrame not working when exiting fullscreen switching space on Safari -

Python ctypes access violation with const pointer arguments -