o Nrf0@sdZddlmZddlZddlmZmZmZddlm Z ddl m Z ddl Z ddlmZddlmZd d Ze jZe jZe jZe jZee jZee jZee jZee jZe jZe jZe jZe jZe jZe j Z e j!Z!e j"Z"e j#Z#e$e% e j&Z&Wdn1swYe j'Z'e$e% e j(Z(Wdn1swYe j)Z)e$e% e j*Z*Wdn1swYd*d d Z+d+ddZ,ddZ-ddZ.ddZ/ddZ0ddZ1d+ddZ2d,ddZ3d+dd Z4d-d"d#Z5d$d%Z6d&d'Z7d(d)Z8dS).z- A set of NumPy functions to apply per chunk ) annotationsN) ContainerIterableSequencewraps)Integral)concat)flattencstdfdd }|S)zU A wrapper for functions that don't provide keepdims to ensure that they do. Nc s|g|Rd|i|}|s|S|}|durt|j}t|tttfs'|g}t}t|jD]}||vr:|d7}q/|tdf7}q/||}|S)NaxisN)rangendim isinstancerrrtupleslice) xr keepdimsargskwargsraxesZr_sliceZ each_axis a_callableY/var/www/html/software/conda/envs/catlas/lib/python3.10/site-packages/dask/array/chunk.pykeepdims_wrapped_callables  z3keepdims_wrapper..keepdims_wrapped_callable)NNr)rrrrrkeepdims_wrappersrFc stjD] }|vrd|<q|r$tfddtjD}|ttfddtjD}||fdttdjddi|S)aCoarsen array by applying reduction to fixed size neighborhoods Parameters ---------- reduction: function Function like np.sum, np.mean, etc... x: np.ndarray Array to be coarsened axes: dict Mapping of axis to coarsening factor Examples -------- >>> x = np.array([1, 2, 3, 4, 5, 6]) >>> coarsen(np.sum, x, {0: 2}) array([ 3, 7, 11]) >>> coarsen(np.max, x, {0: 3}) array([3, 6]) Provide dictionary of scale per dimension >>> x = np.arange(24).reshape((4, 6)) >>> x array([[ 0, 1, 2, 3, 4, 5], [ 6, 7, 8, 9, 10, 11], [12, 13, 14, 15, 16, 17], [18, 19, 20, 21, 22, 23]]) >>> coarsen(np.min, x, {0: 2, 1: 3}) array([[ 0, 3], [12, 15]]) You must avoid excess elements explicitly >>> x = np.array([1, 2, 3, 4, 5, 6, 7, 8]) >>> coarsen(np.min, x, {0: 3}, trim_excess=True) array([1, 4]) c3s>|]\}}||rtd|| ntddVqdS)rNr).0idrrr s ( zcoarsen..cs&g|]}j|||fqSr)shaper r!rrrr s&zcoarsen..r )r rr enumerater%r Zreshape)Z reductionrrZ trim_excessrr!indZnewshaperr'rcoarsenTs( ",r,csNttr g|jttrfddt|jD|tddDS)aCTrim boundaries off of array >>> x = np.arange(24).reshape((4, 6)) >>> trim(x, axes={0: 0, 1: 1}) array([[ 1, 2, 3, 4], [ 7, 8, 9, 10], [13, 14, 15, 16], [19, 20, 21, 22]]) >>> trim(x, axes={0: 1, 1: 1}) array([[ 7, 8, 9, 10], [13, 14, 15, 16]]) csg|]}|dqS)r)getr&r#rrr(sztrim..css$|] }t||r | ndVqdSr r)r axrrrr$"ztrim..)rrrdictr r)rrrr#rtrims   r1cs||dusJdt||jkr|Stj|| d}|dkr(t| dnt| |tfddt|jDS)aChunk and combine function of topk Extract the k largest elements from a on the given axis. If k is negative, extract the -k smallest elements instead. Note that, unlike in the parent function, the returned elements are not sorted internally. Trr Nc3$|] }|kr ntdVqdSr rr&r Zk_slicerrr$r/ztopk..)absr%np partitionrrr rakr rrr4rtopks "r;cs\|dusJt|||}dtj|d}|dkr|S|tfddt|jDS)zmFinal aggregation function of topk Invoke topk one final time and then sort the results internally. Trr2c3,|]}|krtdddntdVqdSNrr&r2rrr$ z!topk_aggregate..)r;r6sortrr rr8rr2rtopk_aggregates  rAcCs||fS)z^Preparatory step for argtopk Put data together with its original indices in a tuple. r)r9idxrrrargtopk_preprocesssrCcs|dusJdt|tr,tt|}tdd|D}tdd|D}n|\}}t||jkr;|Stj|| d}|dkrNt| dnt| |t fdd t |j D}t ||t ||fS) a)Chunk and combine function of argtopk Extract the indices of the k largest elements from a on the given axis. If k is negative, extract the indices of the -k smallest elements instead. Note that, unlike in the parent function, the returned elements are not sorted internally. TrcSsg|]\}}|qSrr)r ai_rrrr(szargtopk..cSsg|] \}}t||jqSr)r6Z broadcast_tor%)r rDZidxirrrr(sr2Nc3r3r rr&r4rrr$r/zargtopk..) rlistr r6Z concatenater5r%Z argpartitionrrr rtake_along_axisZ a_plus_idxr:r rr9rBZidx2rr4rargtopks   "rIcs|dusJt|dkr|n|d}t|||\}}dtj|d}t||}|dkr3|S|tfddt|jDS)zFinal aggregation function of argtopk Invoke argtopk one final time, sort the results internally, drop the data and return the index only. Trrr2c3r<r=rr&r2rrr$r?z$argtopk_aggregate..)lenrIr6ZargsortrGrr rrHrr2rargtopk_aggregates  rKcCs:ddlm}||||||d}t||kr|ddS|S)Nr) arange_safeliker>)dask.array.utilsrLrJ)startstopsteplengthdtyperNrLresrrraranges rVTcCsDddlm}t||r|}t||r|}tj|||||dS)Nr)Array)endpointrT)Zdask.array.corerWrZcomputer6linspace)rPrQnumrXrTrWrrrrY s   rYcKs|j|fi|Sr )astype)rZ astype_dtyperrrrr[sr[CcCs|dkr!z tj||d}Wntyt|}Ynw||Sz tj||d}Wnty8t|}Ynw|j|jS)Nr\rM)r6Zascontiguousarray TypeErrorviewZasfortranarrayT)rrTorderrrrr^s   r^csddlm}m}|||dtjtdk||dk|jk@}||tfddt |j DS)aChunk function of `slice_with_int_dask_array_on_axis`. Slice one chunk of x by one chunk of idx. Parameters ---------- x: ndarray, any dtype, any shape i-th chunk of x idx: ndarray, ndim=1, dtype=any integer j-th chunk of idx (cartesian product with the chunks of x) offset: ndarray, shape=(1, ), dtype=int64 Index of the first element along axis of the current chunk of x x_size: int Total size of the x da.Array along axis axis: int normalized axis to take elements from (0 <= axis < x.ndim) Returns ------- x sliced along axis, using only the elements of idx that fall inside the current chunk. r) asarray_safemeta_from_arrayrMc3r3r rr&r rBrrr$Sr/z,slice_with_int_dask_array..) rOrarbr[r6int64wherer%rr r)rrBoffsetZx_sizer rarb idx_filterrrcrslice_with_int_dask_array(s "rhc s|tj}t|dk|t||}d}d}t||D]-}||k|||k@}t|}t||d|d7||7}|jdkrK||d7}q|tfddt |j DS)aSFinal aggregation function of `slice_with_int_dask_array_on_axis`. Aggregate all chunks of x by one chunk of idx, reordering the output of `slice_with_int_dask_array`. Note that there is no combine function, as a recursive aggregation (e.g. with split_every) would not give any benefit. Parameters ---------- idx: ndarray, ndim=1, dtype=any integer j-th chunk of idx chunk_outputs: ndarray concatenation along axis of the outputs of `slice_with_int_dask_array` for all chunks of x and the j-th chunk of idx x_chunks: tuple dask chunks of the x da.Array along axis, e.g. ``(3, 3, 2)`` axis: int normalized axis to take elements from (0 <= axis < x.ndim) Returns ------- Selection from all chunks of x for the j-th chunk of idx, in the correct order rrr>c3r3r rr&r Z idx_finalrrr$s z6slice_with_int_dask_array_aggregate..) r[r6rdresumZ zeros_likeZcumsumsizerr r) rBZ chunk_outputsZx_chunksr Zx_chunk_offsetZchunk_output_offsetZx_chunkrgZidx_cumrrir#slice_with_int_dask_array_aggregateVs$     rlc Csxz||}Wnty}ztd|d}~wwz|jjs,|jd|jkr/|}W|SW|SW|Sty;Y|Sw)afGetitem function This function creates a copy of the desired selection for array-like inputs when the selection is smaller than half of the original array. This avoids excess memory usage when extracting a small portion from a large array. For more information, see https://numpy.org/doc/stable/reference/arrays.indexing.html#basic-slicing-and-indexing. Parameters ---------- obj: ndarray, string, tuple, list Object to get item from. index: int, list[int], slice() Desired selection to extract from obj. Returns ------- Selection obj[index] zTArray chunk size or shape is unknown. Possible solution with x.compute_chunk_sizes()Nr)) IndexError ValueErrorflagsZowndatarkcopyAttributeError)objindexresulterrrgetitems,   rv)Fr )TN)r\)9__doc__ __future__r contextlibcollections.abcrrr functoolsrnumbersrnumpyr6Ztlzr Z dask.corer rrjprodminmaxZargminZ nanargminZargmaxZ nanargmaxanyallZnansumZnanprodZ nancumprodZ nancumsumZnanminZnanmaxmeansuppressrqZnanmeanvarZnanvarZstdZnanstdr,r1r;rArCrIrKrVrYr[r^rhrlrvrrrrsh     #         9   . 8