0Fie?&ddlmZddlZddlZddlZddlZddlZddlZddlZddl m Z ddl m Z ddl mZddlZddlmZddlmZddlZddlmZdd lmZdd lmZdd lmZdd lmZdd lm Z ddl!m"Z"ddl#m$Z$ddl%m&Z&m'Z'ddl(m)Z)m*Z*ej+dZ,ej-ddZ.ej/de0dej1de2dej3de.ddej3de.ddej3d e.dd!ej3d"dd#$ej3d%dd&$ej3d'e2d(d)ej3d*d+ddd,-ej3d.e2dd/ej3d0e2dd1ej3d2e2dd3ej3d4e2dd5ej3d6e2dd7ej3d8e4dd9ej3d:d;e2ddd<=ej3d>e2dd?ej3d@dAddBCej3dDddE$ej3dFe2dGdHej3dIde2dJKej3dLe2ddMej3dNe2ddOej3dPe2ddQej3dRe2dGdSej3dTe2ddUej3dVe2ddWej3dXe2dYddZ=ej3d[d\ddd]-ej3d^e2ddd_`ej1dadbej5e$cej3dde2ddde`ej3dfe2ddgej6dsdkZ7dtdqZ8e9drkr e7dSdS)u) annotationsN)Iterator)suppress)Any)valmap) TimeoutError) CPU_COUNT)Nanny)wait_for_signals)get_address_host_port) asyncio_run)get_loop_factory)nprocesses_nthreads)validate_preload_argv)enable_proctitle_on_childrenenable_proctitle_on_current) import_term parse_portszdistributed.dask_workerT)exists resolve_pathworker)ignore_unknown_options)namecontext_settings schedulerF)typerequiredz --tls-ca-filez'CA cert(s) file for TLS (in PEM format))rdefaulthelpz --tls-certz(certificate file for TLS (in PEM format)z --tls-keyz(private key file for TLS (in PEM format)z --worker-porta-Serving computation port, defaults to random. When creating multiple workers with --nworkers, a sequential range of worker ports may be used by specifying the first and last available ports like :. For example, --worker-port=3000:3026 will use ports 3000, 3001, ..., 3025, 3026.)rrz --nanny-porta%Serving nanny port, defaults to random. When creating multiple nannies with --nworkers, a sequential range of nanny ports may be used by specifying the first and last available ports like :. For example, --nanny-port=3000:3026 will use ports 3000, 3001, ..., 3025, 3026.z--dashboard-addressz:0z4Address on which to listen for diagnostics dashboardz--dashboard/--no-dashboard dashboardz+Launch the Dashboard [default: --dashboard])rrrz--listen-addressz_The address to which the worker binds. Example: tcp://0.0.0.0:9000 or tcp://:9000 for IPv4+IPv6z--contact-addressz}The address the worker advertises to the scheduler for communication with it and other workers. Example: tcp://127.0.0.1:9000z--hostzServing host. Should be an ip address that is visible to the scheduler and other workers. See --listen-address and --contact-address if you need different listen and contact addresses. See --interface.z --interfacez&Network interface like 'eth0' or 'ib0'z --protocolzProtocol like tcp, tls, or ucxz --nthreadszNumber of threads per process.z --nworkers n_workerszNumber of worker processes to launch. If negative, then (CPU_COUNT + 1 + nworkers) is used. Set to 'auto' to set nworkers and nthreads dynamically based on CPU_COUNT)rr show_defaultrz--namezA unique name for this worker like 'worker-1'. If used with --nworkers then the process number will be appended like name-0, name-1, name-2, ...z--memory-limitautoa- Bytes of memory per process that the worker can use. This can be: - an integer (bytes), note 0 is a special case for no memory management. - a float (fraction of total system memory). - a string (like 5GB or 5000M). - 'auto' for automatically computing the memory limit. )rr"rz--nanny/--no-nannyz@Start workers in nanny process for management [default: --nanny]z --pid-filezFile to write the process PIDz--local-directoryzDirectory to place worker files)rrrz --resourceszResources for task constraints like "GPU=2 MEM=10e9". Resources are applied separately to each worker process (only relevant when starting multiple worker processes with '--nworkers').z--scheduler-filezXFilename to JSON encoded scheduler information. Use with dask scheduler --scheduler-filez--death-timeoutz.Seconds to wait for a scheduler before closingz--dashboard-prefixzPrefix for the dashboardz --lifetimez6If provided, shut down the worker after this duration.z--lifetime-staggerz1Random amount by which to stagger lifetime valuesz--worker-classzdask.distributed.Workerz.Worker class used to instantiate workers from.z(--lifetime-restart/--no-lifetime-restartlifetime_restartzWhether or not to restart the worker after the lifetime lapses. This assumes that you are using the --lifetime and --nanny keywordsz --preloadzWModule that should be loaded by each worker process like "foo.bar" or "/path/to/foo.py")rmultipleis_eagerr preload_argv)nargsrcallbackz--preload-nannyzNModule that should be loaded by each nanny like "foo.bar" or "/path/to/foo.py"z--scheduler-sniz4Scheduler SNI (if different from scheduler hostname) worker_port str | None nanny_portc    !dtjdvrtjdtdt j\}}}t j|dz|dz|dzttdd|fd |fd |ffDd krt\ndntdkr tdzzdkr.td tjddkr0s.tdtjdr0|s.tdtjddkr0|r.tdtjd|sr0|r.tdtjd |r-t!|d\}t#|}dvrddrt!d\}}n|nU#t$$rH}tdt#|ztjdYd }~nd }~wwxYws tz rtt' d5}|t#t+jd d d n #1swxYwY fd}t/j| rW dd t7d D t9t:  nd t=|}t?||tAksJr|d<|d<|d<tB!n|!s1 s/tDj#$d d t%d!tKtLt$5t  d d d n #1swxYwYd"     !fd#} tO|tQ$nL#tRtTj)f$r3 st+d%tjdYnwxYwt+d&d S#t+d&wxYw)'z6Launch a Dask worker attached to an existing schedulerz dask-workerrz\dask-worker is deprecated and will be removed in a future release; use `dask worker` instead) stacklevelci|] \}}||| SN).0kvs ;lib/python3.11/site-packages/distributed/cli/dask_worker.py zmain..s/    Aq = 1 == tls_ca_filetls_worker_certtls_worker_keyr#NzVFailed to launch worker. Must specify --nworkers so that there's at least one process.zTFailed to launch worker. You cannot use the --no-nanny argument when n_workers > 1.zVFailed to launch worker. Must specify --listen-address when --contact-address is givenzPFailed to launch worker. You cannot specify --listen-address when n_workers > 1.zcFailed to launch worker. You cannot specify --listen-address when --worker-port or --host is given.T)strict:[]zFailed to launch worker. wcrtjrtjdSdSr4)ospathrremove)pid_filesr9 del_pid_filezmain..del_pid_fileds9w~~h'' $ (##### $ $r;, c3@K|]}|dVdS)=N)split)r6pairs r9 zmain..ls,??TC??????r; worker_class preload_nannylisten_addresszscheduler-addresszINeed to provide scheduler address like dask worker SCHEDULER_ADDRESS:8786Fc RK f dtDfd}fd}tj|}tj|}tj||gtjd{V\}}d|DdS)Nc g|]J\}} f dks  dkr n!t dzt|zd |KS)r0Nr$-) scheduler_filenthreads resourcessecuritycontact_addresshostr dashboard_addressr)str)r6i port_kwargs_ir[r r]r\kwargsr!rrXrYrrWsects r9 z%main..run..s   "!=! A -!# /#"3>>T\TRZZTYY_s1vv-         r;cpKtjd{VtjdDd{VdS)z-Wait for all nannies to initialize and finishNc3>K|]}|VdSr4)finished)r6ns r9rPzHmain..run..wait_for_nannies_to_finish..s*"A"AA1::<<"A"A"A"A"A"Ar;)asynciogather)nanniessr9wait_for_nannies_to_finishz5main..run..wait_for_nannies_to_finishsZ.'* * * * * * * *."A"A"A"A"AB B B B B B B B B Br;cKtd{Vdr"tjfdDd{VdSdS)zTWait for SIGINT or SIGTERM and close all nannies upon receiving one of those signalsNTc3LK|]}|ddVdS) zsignal-)timeoutreasonN)close)r6rhsignums r9rPzHmain..run..wait_for_signals_and_close..s<VVaggb1C61C1CgDDVVVVVVr;)r rirj)rsrknanny signal_fireds@r9wait_for_signals_and_closez5main..run..wait_for_signals_and_closes,--------FL nVVVVgVVV  r;) return_whenc6g|]}|Sr5)result)r6tasks r9rdz%main..run..s (((4(((r;) enumerateri create_taskwaitFIRST_COMPLETED)rlrvwait_for_signals_and_close_taskwait_for_nannies_to_finish_taskdone_rkr[r r]r\rar!rrtrX port_kwargsrYrrWrbrurcs @r9runzmain..runs^                "%.k$:$:#   ( C C C C C       +2*= & & ( (+ + '+2*= & & ( (+ + ' ,.M N/         a )(4((((((r;) loop_factoryzTimed out starting workerz End worker),sysargvwarningswarn FutureWarninggc get_threshold set_thresholdrrrintr loggererrorexitr r^ ValueErroropenwriterEgetpidatexitregisterreplacerNdictrfloatr_apportion_portslenr daskconfiggetr TypeErrorr rrriinfo)"rr\r,rSr[r.rXr!rtrrHrYr rWdashboard_prefixr<tls_certtls_keyr]rQrRrag0g1g2refrIrrrbrurcs"`` ` ```````` ` ` @@@@r9mainr(sRB ## j     !##JBBR!VR!VR!V,,,!!! """  K (  ) w '    CF133 88    NN 1}}MI- A~~ d     1}}U} b     ~  L     1}}}  F     t  Y       #+N4HHHGD!a&&Kd{{"4{{{  -*?4HHHFQ-O  03q669:::   * )& (C  &A GGC $$ % % % & & & & & & & & & & & & & & & $ $ $ $ $  %%%%%c3//5577 ??Y????? 5),,  |,,L"; IuMMK {  y ( ( ( ( !-~"/#1      KOO/ 6 6 > 1   )Z ( (4yyL2)2)2)2)2)2)2)2)2)2)2)2)2)2)2)2)2)2)2)2)h"CCEE(8(:(:;;;;; '. / 5 KK3 4 4 4    L!!!!! L!!!!sh/AH88 J >JJ +4K++K/2K/P$$P(+P(%Q,+S,AR52S4R55SS0rrtboolreturnlist[dict[str, Any]]c ntdfd }||}||}dtD}t|}d} d} | s| r t|\} } n#t$rt|}Y3wxYw | t|n#t$rd} YnwxYw | t|n#t$rd} YnwxYw| | g} |D]\} } | r| s1|rt d |d |d d t d |d d t| }t|d kr|d}|r0t| }t|d kr|d}||d}nd|i}| || S)zSpread out evenly --worker-port and/or --nanny-port ranges to the workers and nannies, avoiding overlap. Returns ======= List of kwargs to pass to the Worker or Nanny constructors sr-rIterator[int | None]c3Kt|}|dgdgfvrtD] }|dV dS|D]}|vr||V dS)Nr)rrangeadd)rportsrportr!seens r9 parse_uniquez&_apportion_ports..parse_uniquesA aS4&M ! !9%%  Ah    t##HHTNNNJJJ  r;cFg|]}ttfSr5)set)r6rs r9rdz$_apportion_ports..s1<<<<<<r;TFz(Not enough ports in range --worker_port z --nanny_port z for z workersr0r)rr,r)rr-rr) rriternext StopIterationrrsortedrappend)r,r.r!rtrworker_ports_iternanny_ports_iterr ports_itermore_wpsmore_npsworker_ports_i nanny_ports_irawpnpkwargs_irs ` @r9rrs 55D       % [11#|J//<< %i 0 0<<<EeJHH h ,0,<,< )NMM   eJ H      t$566 7 7 7 7   HHH     d#344 5 5 5 5   HHH  h"F).  %  ]   I{II$.II5>III !/{//$///(( r77a<<AB  $]++B2ww!||U "266HH|H h Ms6A22B B"B55 CC"C++ C:9C:__main__)r,r-r.r-) r,r-r.r-r!rrtrrr): __future__rrirrloggingrErrcollections.abcr contextlibrtypingrclicktlzrtornado.iolooprr dask.systemr distributedr distributed._signalsr distributed.commr distributed.compatibilityr distributed.configrdistributed.deploy.utilsrdistributed.preloadingrdistributed.proctitlerrdistributed.utilsrr getLoggerrPathpem_file_option_typecommandrargumentr^optionr UNPROCESSEDversion_optionrr__name__r5r;r9rs""""""  $$$$$$ '''''' !!!!!!111111222222111111//////88888888888876666666  4 5 5"uzDAAAHtt4/P/P/PQQQ #666    2     3     3    2  2  ?      6   j    $      T0XsD/Ola6VWWW   P     8        K lb7VWWW C6W  Q   /   9  sB5O  A    <   % 9 .   J    * "5#4?T   *   ?  Z"Z"Z"    XW     XW       76RQNZ"zPPPPf zDFFFFFr;