0Fie(ddlmZddlZddlZddlZddlZddlmZddlm Z ddl m Z ddl m Z ddlmZddlmZdd lmZdd lmZdd lmZejeZGd d e ZdS)) annotationsN) CPU_COUNT) get_template) SpecCluster)nprocesses_nthreads)Nanny) Scheduler)Security)Worker)parse_memory_limitcpeZdZdZdddddddddejddddddddddddddffd ZdZd fd ZxZ S) LocalClusteruyCreate local Scheduler and Workers This creates a "cluster" of a scheduler and workers running on the local machine. Parameters ---------- n_workers: int Number of workers to start memory_limit: str, float, int, or None, default "auto" Sets the memory limit *per worker*. Notes regarding argument data type: * If None or 0, no limit is applied. * If "auto", the total system memory is split evenly between the workers. * If a float, that fraction of the system memory is used *per worker*. * If a string giving a number of bytes (like ``"1GiB"``), that amount is used *per worker*. * If an int, that number of bytes is used *per worker*. Note that the limit will only be enforced when ``processes=True``, and the limit is only enforced on a best-effort basis — it's still possible for workers to exceed this limit. processes: bool Whether to use processes (True) or threads (False). Defaults to True, unless worker_class=Worker, in which case it defaults to False. threads_per_worker: int Number of threads per each worker scheduler_port: int Port of the scheduler. Use 0 to choose a random port (default). 8786 is a common choice. silence_logs: logging level Level of logs to print out to stdout. ``logging.WARN`` by default. Use a falsey value like False or None for no change. host: string Host address on which the scheduler will listen, defaults to only localhost ip: string Deprecated. See ``host`` above. dashboard_address: str Address on which to listen for the Bokeh diagnostics server like 'localhost:8787' or '0.0.0.0:8787'. Defaults to ':8787'. Set to ``None`` to disable the dashboard. Use ':0' for a random port. worker_dashboard_address: str Address on which to listen for the Bokeh worker diagnostics server like 'localhost:8787' or '0.0.0.0:8787'. Defaults to None which disables the dashboard. Use ':0' for a random port. diagnostics_port: int Deprecated. See dashboard_address. asynchronous: bool (False by default) Set to True if using this cluster within async/await functions or within Tornado gen.coroutines. This should remain False for normal use. blocked_handlers: List[str] A list of strings specifying a blocklist of handlers to disallow on the Scheduler, like ``['feed', 'run_function']`` service_kwargs: Dict[str, Dict] Extra keywords to hand to the running services security : Security or bool, optional Configures communication security in this cluster. Can be a security object, or True. If True, temporary self-signed credentials will be created automatically. protocol: str (optional) Protocol to use like ``tcp://``, ``tls://``, ``inproc://`` This defaults to sensible choice given other keyword arguments like ``processes`` and ``security`` interface: str (optional) Network interface to use. Defaults to lo/localhost worker_class: Worker Worker class used to instantiate workers from. Defaults to Worker if processes=False and Nanny if processes=True or omitted. **worker_kwargs: Extra worker arguments. Any additional keyword arguments will be passed to the ``Worker`` class constructor. Examples -------- >>> cluster = LocalCluster() # Create a local cluster # doctest: +SKIP >>> cluster # doctest: +SKIP LocalCluster("127.0.0.1:8786", workers=8, threads=8) >>> c = Client(cluster) # connect to local cluster # doctest: +SKIP Scale the cluster to three workers >>> cluster.scale(3) # doctest: +SKIP Pass extra keyword arguments to Bokeh >>> LocalCluster(service_kwargs={'dashboard': {'prefix': '/foo'}}) # doctest: +SKIP Nrz:8787Fc 0||}| tjd| } |dkrtjdd}d|vrtjd||dupt|t}||rtnt}d|_||_|t}n<|durtj}n$t|tstd|<|r d|vr| dd}n|r |j rd }n|js| sd }nd }| ds|dz}||d s|sd }|pi}|pi}|||rt\}}n d}t }|||rt#dt |znd}|r9|7t#dt%t'jt |z }|rd|vrt+dd|t,|d<||||| | du|||| d t0t3jt7||||| ||| du| | |pid}||dfdt9|D}t;|||||| || dS)NzMdiagnostics_port has been deprecated. Please use `dashboard_address=` insteadrz_Setting `threads_per_worker` to 0 has been deprecated. Please set to None or to a specific int. dashboardzSetting `dashboard` is discouraged. Please set `dashboard_address` to affect the scheduler (more common) and `worker_dashboard_address` for the worker (less common).Tz"security must be a Security objectz://ztls://z inproc://ztcp://inprocz 127.0.0.1r memory_limitauto)logger) hostnthreadsservicesdashboard_addressr interfaceprotocolsecurity silence_logs) rrservice_kwargsrportrrrrblocked_handlers)clsoptionsci|]}|Sr$).0iworkers 8lib/python3.11/site-packages/distributed/deploy/local.py z)LocalCluster.__init__..s7771f777) name schedulerworkersr'loop asynchronousrrscheduler_sync_interval)warningswarn issubclassrr status processesr temporary isinstance TypeErrorsplitrequire_encryptionendswith startswithrrmaxintmathceilr rupdater toolzmergedictrangesuper__init__)selfr+ n_workersthreads_per_workerr5r.startripscheduler_portrrworker_dashboard_addressdiagnostics_portrworker_servicesrr/rrr r worker_classscheduler_kwargsr0 worker_kwargsr,r-r' __class__s @r(rGzLocalCluster.__init__qs8 >D  ' M:   !1   " " M;   "&  - ' ' MO     $,O <0O0OI  $-9556L "  zzHH   )++HHHh// B@AA A   $ ::e,,Q/ $h9 $#^ $N $&#  '' (%'H < 3 3H = =r)/R  !3!; /0C0E0E- -- %."  !3!?CLSAy,>>???RSI  O+3!$QDIi)6K,L,L(M(M!N!N   }<<,>9V---M. ) .+%=5TA&$$ ,   {%#1%''%/t;&7%5   !&B  &&-@@7777eI&6&6777 %%$;  r*c td)NzdThe `cluster.start_worker` function has been removed. Please see the `cluster.scale` method instead.)NotImplementedError)rHargskwargss r( start_workerzLocalCluster.start_worker s! =   r*ctd|jj|j|}t |S)Nzlocal_cluster.html.j2)r4r5cluster_status)r[)rrenderr4r+r5rF _repr_html_)rHr[rTs r(r]zLocalCluster._repr_html_sT%&=>>EE;#n)F   ww""."AAAr*)N) __name__ __module__ __qualname____doc__loggingWARNrGrYr] __classcell__)rTs@r(rrsWWv   \!!% !3V V V V V V p   BBBBBBBBBBr*r) __future__rrbr?r1rB dask.systemr dask.widgetsrdistributed.deploy.specrdistributed.deploy.utilsrdistributed.nannyrdistributed.schedulerr distributed.securityr distributed.workerr distributed.worker_memoryr getLoggerr^rrr$r*r(rps7""""""  !!!!!!%%%%%%//////888888######++++++))))))%%%%%%888888  8 $ $~B~B~B~B~B;~B~B~B~B~Br*