ó õùPc@s8dgZddlZddlZddlZddlZddlZddlmZmZm Z ddl m Z m Z dZ dZdZejƒZd„Zefed „Zdefd „ƒYZd efd „ƒYZd efd„ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZdS(tPooliÿÿÿÿN(tProcesst cpu_countt TimeoutError(tFinalizetdebugiiicCs t|ŒS(N(tmap(targs((s%..\python\lib\multiprocessing\pool.pytmapstar&sc Cs}|dks0t|ƒtkr*|dks0t‚|j}|j}t|dƒrn|jjƒ|j jƒn|dk r‡||Œnd}xÛ|dks®|rj||krjy |ƒ}Wn"t t fk rßt dƒPnX|dkrút dƒPn|\} } } } } yt | | | Žf}Wntk rI}t|f}nX|| | |fƒ|d7}qWt d|ƒdS(Nit_writers)worker got EOFError or IOError -- exitingsworker got sentinel -- exitingisworker exiting after %d tasks(tNonettypetinttAssertionErrortputtgetthasattrR tcloset_readertEOFErrortIOErrorRtTruet ExceptiontFalse(tinqueuetoutqueuet initializertinitargstmaxtasksRRt completedttasktjobtitfuncRtkwdstresultte((s%..\python\lib\multiprocessing\pool.pytworker-s40     !    cBseZdZeZddddd„Zd„Zd„Zd„Zd„Z did„Z dd„Z dd „Z dd „Z didd „Zddd „Zed „ƒZed„ƒZed„ƒZed„ƒZd„Zd„Zd„Zd„Zed„ƒZed„ƒZRS(sH Class which supports an async version of the `apply()` builtin c Cs|jƒtjƒ|_i|_t|_||_||_||_|dkr|y t ƒ}Wq|t k rxd}q|Xn|dk r§t |dƒ r§t dƒ‚n||_g|_|jƒtjdtjd|fƒ|_t|j_t|j_|jjƒtjdtjd|j|j|j|jfƒ|_t|j_t|j_|jjƒtjdtjd|j|j|jfƒ|_t|j_t|j_|jjƒt ||j!d|j|j"|j|j|j|j|j|jfddƒ|_#dS(Nit__call__sinitializer must be a callablettargetRt exitpriorityi($t _setup_queuestQueuet _taskqueuet_cachetRUNt_statet_maxtasksperchildt _initializert _initargsR RtNotImplementedErrorRt TypeErrort _processest_poolt_repopulate_poolt threadingtThreadRt_handle_workerst_worker_handlerRtdaemontstartt _handle_taskst _quick_putt _outqueuet _task_handlert_handle_resultst _quick_gett_result_handlerRt_terminate_poolt_inqueuet _terminate(tselft processesRRtmaxtasksperchild((s%..\python\lib\multiprocessing\pool.pyt__init__WsR                    $          cCswt}xjttt|jƒƒƒD]M}|j|}|jdk r"td|ƒ|jƒt }|j|=q"q"W|S(sCleanup after any worker processes which have exited due to reaching their specified lifetime. Returns True if any workers were cleaned up. scleaning up worker %dN( RtreversedtrangetlenR5texitcodeR RtjoinR(RGtcleanedR R%((s%..\python\lib\multiprocessing\pool.pyt_join_exited_workerss"  c Cs¦xŸt|jt|jƒƒD]}|jdtd|j|j|j|j |j fƒ}|jj |ƒ|j j ddƒ|_ t|_|jƒtdƒqWdS(s€Bring the number of pool processes up to the specified number, for use after reaping workers which have exited. R'RRt PoolWorkers added workerN(RLR4RMR5RR%RER?R0R1R/tappendtnametreplaceRR;R<R(RGR tw((s%..\python\lib\multiprocessing\pool.pyR6žs#   cCs|jƒr|jƒndS(sEClean up any exited workers and start replacements for them. N(RQR6(RG((s%..\python\lib\multiprocessing\pool.pyt_maintain_pool®s cCsPddlm}|ƒ|_|ƒ|_|jjj|_|jjj|_ dS(Ni(t SimpleQueue( tqueuesRXRER?R tsendR>RtrecvRB(RGRX((s%..\python\lib\multiprocessing\pool.pyR)´s   cCs.|jtkst‚|j|||ƒjƒS(s1 Equivalent of `apply()` builtin (R.R-R t apply_asyncR(RGR!RR"((s%..\python\lib\multiprocessing\pool.pytapply»scCs.|jtkst‚|j|||ƒjƒS(s/ Equivalent of `map()` builtin (R.R-R t map_asyncR(RGR!titerablet chunksize((s%..\python\lib\multiprocessing\pool.pyRÂsicsÝ|jtkst‚|dkrft|jƒ‰|jj‡‡fd†t|ƒDƒˆjfƒˆS|dksxt‚t j ˆ||ƒ}t|jƒ‰|jj‡fd†t|ƒDƒˆjfƒd„ˆDƒSdS(sZ Equivalent of `itertools.imap()` -- can be MUCH slower than `Pool.map()` ic3s0|]&\}}ˆj|ˆ|fifVqdS(N(t_job(t.0R tx(R#R!(s%..\python\lib\multiprocessing\pool.pys Ðsc3s0|]&\}}ˆj|t|fifVqdS(N(RaR(RbR Rc(R#(s%..\python\lib\multiprocessing\pool.pys ×scss"|]}|D] }|Vq qdS(N((Rbtchunktitem((s%..\python\lib\multiprocessing\pool.pys ÙsN( R.R-R t IMapIteratorR,R+Rt enumeratet _set_lengthRt _get_tasks(RGR!R_R`t task_batches((R#R!s%..\python\lib\multiprocessing\pool.pytimapÉs csÝ|jtkst‚|dkrft|jƒ‰|jj‡‡fd†t|ƒDƒˆjfƒˆS|dksxt‚t j ˆ||ƒ}t|jƒ‰|jj‡fd†t|ƒDƒˆjfƒd„ˆDƒSdS(sK Like `imap()` method but ordering of results is arbitrary ic3s0|]&\}}ˆj|ˆ|fifVqdS(N(Ra(RbR Rc(R#R!(s%..\python\lib\multiprocessing\pool.pys âsc3s0|]&\}}ˆj|t|fifVqdS(N(RaR(RbR Rc(R#(s%..\python\lib\multiprocessing\pool.pys éscss"|]}|D] }|Vq qdS(N((RbRdRe((s%..\python\lib\multiprocessing\pool.pys ësN( R.R-R tIMapUnorderedIteratorR,R+RRgRhRRi(RGR!R_R`Rj((R#R!s%..\python\lib\multiprocessing\pool.pytimap_unorderedÛs cCsV|jtkst‚t|j|ƒ}|jj|jd|||fgdfƒ|S(s> Asynchronous equivalent of `apply()` builtin N( R.R-R t ApplyResultR,R+RRaR (RGR!RR"tcallbackR#((s%..\python\lib\multiprocessing\pool.pyR\ís+csû|jtkst‚t|dƒs3t|ƒ}n|dkr}tt|ƒt|jƒdƒ\}}|r}|d7}q}nt|ƒdkr˜d}nt j |||ƒ}t |j |t|ƒ|ƒ‰|j j‡fd†t|ƒDƒdfƒˆS(s< Asynchronous equivalent of `map()` builtin t__len__iiic3s0|]&\}}ˆj|t|fifVqdS(N(RaR(RbR Rc(R#(s%..\python\lib\multiprocessing\pool.pys sN(R.R-R RtlistR tdivmodRMR5RRit MapResultR,R+RRg(RGR!R_R`RotextraRj((R#s%..\python\lib\multiprocessing\pool.pyR^ös ( cCsMx<|jjtkr>|jtkr>|jƒtjdƒqWtdƒdS(Ngš™™™™™¹?sworker handler exiting(R:R.R-RWttimetsleepR(tpool((s%..\python\lib\multiprocessing\pool.pyR9 s$ c Cs1tjƒ}x¶t|jdƒD]˜\}}d}x‚t|ƒD]P\}}|jratdƒPny||ƒWq>tk rtdƒPq>Xq>W|rtdƒ||dƒqqPqWtdƒy@tdƒ|j dƒtdƒx|D]} |dƒqðWWntk r"td ƒnXtd ƒdS( Niÿÿÿÿs'task handler found thread._state != RUNscould not put task on queuesdoing set_length()istask handler got sentinels/task handler sending sentinel to result handlers(task handler sending sentinel to workerss/task handler got IOError when sending sentinelsstask handler exiting( R7tcurrent_threadtiterRR RgR.RRR( t taskqueueRRRwtthreadttaskseqt set_lengthR Rtp((s%..\python\lib\multiprocessing\pool.pyR=s6             cCsòtjƒ}x¶y |ƒ}Wn"ttfk r@tdƒdSX|jrm|jtks_t‚tdƒPn|dkr‡tdƒPn|\}}}y||j ||ƒWqt k rÁqXqx¢|ri|jtkriy |ƒ}Wn"ttfk rtdƒdSX|dkr+tdƒqÈn|\}}}y||j ||ƒWqÈt k reqÈXqÈWt |dƒrÕtdƒy5x.t dƒD] }|j jƒs¬Pn|ƒq“WWqÕttfk rÑqÕXntdt|ƒ|jƒdS( Ns.result handler got EOFError/IOError -- exitings,result handler found thread._state=TERMINATEsresult handler got sentinels&result handler ignoring extra sentinelRs"ensuring that outqueue is not fulli s7result handler exiting: len(cache)=%s, thread._state=%s(R7RxRRRR.t TERMINATER R t_settKeyErrorRRLRtpollRM(RRtcacheR{RRR tobj((s%..\python\lib\multiprocessing\pool.pyRA9sX              ccsCt|ƒ}x0ttj||ƒƒ}|s1dS||fVqdS(N(Ryttuplet itertoolstislice(R!tittsizeRc((s%..\python\lib\multiprocessing\pool.pyRiss  cCstdƒ‚dS(Ns:pool objects cannot be passed between processes or pickled(R2(RG((s%..\python\lib\multiprocessing\pool.pyt __reduce__|scCsEtdƒ|jtkrAt|_t|j_|jjdƒndS(Ns closing pool(RR.R-tCLOSER:R+RR (RG((s%..\python\lib\multiprocessing\pool.pyRs    cCs-tdƒt|_t|j_|jƒdS(Nsterminating pool(RRR.R:RF(RG((s%..\python\lib\multiprocessing\pool.pyt terminateˆs   cCsntdƒ|jttfks%t‚|jjƒ|jjƒ|jjƒx|j D]}|jƒqVWdS(Ns joining pool( RR.R‹RR R:ROR@RCR5(RGR~((s%..\python\lib\multiprocessing\pool.pyROŽs    cCsWtdƒ|jjƒx9|jƒrR|jjƒrR|jjƒtjdƒqWdS(Ns7removing tasks from inqueue until task handler finishedi( Rt_rlocktacquiretis_aliveRR‚R[RuRv(Rt task_handlerR‰((s%..\python\lib\multiprocessing\pool.pyt_help_stuff_finish—s    c Csltdƒt|_t|_|jdƒtdƒ|j||t|ƒƒ|jƒspt|ƒdkspt‚t|_|jdƒ|rÙt |ddƒrÙtdƒx-|D]"} | j dkr°| j ƒq°q°Wntdƒ|j dƒtdƒ|j dƒ|rht |ddƒrhtd ƒx;|D]0} | jƒr1td | j ƒ| j ƒq1q1WndS( Nsfinalizing pools&helping task handler/workers to finishiRŒsterminating workerssjoining task handlerg}Ô%­I²Tsjoining result handlersjoining pool workersscleaning up worker %d(RRR.RR R‘RMRR RRNRŒROtpid( tclsRzRRRwtworker_handlerRtresult_handlerRƒR~((s%..\python\lib\multiprocessing\pool.pyRD s0     $           N((((t__name__t __module__t__doc__RR RJRQR6RWR)R]RRkRmR\R^t staticmethodR9R=RARiRŠRRŒROR‘t classmethodRD(((s%..\python\lib\multiprocessing\pool.pyRQs0  7        ':     RncBsDeZd„Zd„Zd„Zdd„Zdd„Zd„ZRS(cCsStjtjƒƒ|_tjƒ|_||_t|_ ||_ |||jeZd„Zd„Zdd„ZeZd„Zd„ZRS(cCsktjtjƒƒ|_tjƒ|_||_tj ƒ|_ d|_ d|_ i|_|||jRRB(RG((s%..\python\lib\multiprocessing\pool.pyR)ŠscCsW|jjƒz5|jjƒ|jjdg|ƒ|jjƒWd|jjƒXdS(N(t not_emptyRŽtqueuetcleartextendR t notify_allR¦(RRR‰((s%..\python\lib\multiprocessing\pool.pyR‘s   N(( R–R—tdummyRR RJR)R™R‘(((s%..\python\lib\multiprocessing\pool.pyR½ƒs (t__all__R7R*R†R°RutmultiprocessingRRRtmultiprocessing.utilRRR-R‹RtcountRžRR R%tobjectRRnRsRfRlR½(((s%..\python\lib\multiprocessing\pool.pyt s(        $ÿ{2,I