o Nrf*@sddlmZddlZddlZddlmZddlmZddlm Z ddl m Z ddl Z ddlmZddlmZmZdd lmZdd lmZdd lmZdd lmZdd lmZmZdZddZddZ ddZ!dddZ"dS)) annotationsN)Counter)reduce)product)mul)config)Arraynormalize_chunks)meta_from_array)tokenize)flatten)HighLevelGraph)M parse_bytesa Dask's reshape only supports operations that merge or split existing dimensions evenly. For example: >>> x = da.ones((6, 5, 4), chunks=(3, 2, 2)) >>> x.reshape((3, 2, 5, 4)) # supported, splits 6 into 3 & 2 >>> x.reshape((30, 4)) # supported, merges 6 & 5 into 30 >>> x.reshape((4, 5, 6)) # unsupported, existing dimensions split unevenly To work around this you may call reshape in multiple passes, or (if your data is small enough) call ``compute`` first and handle reshaping in ``numpy`` directly. c stddDs Jtd}t|d}ddttD}ddtt|D}|dks6|dkrƈ|||krS|||<|||<|d8}|d8}q-|}||}|dkrid||<|d8}nU|dkrwd||<|d8}nG||kr8|d} | dkrtt| |d|kr| d8} | dkrtt| |d|kstt| |d|krtttfd dt|Drt|dD]} | || <qˈ|tt t| |||<nHt| d|dD] } | f|| <qttt t| d|d} t | | || <tt| d|dt fd d|| D||<|d8}| d}n||kr|d} | dkrktt|| |d|krk| d8} | dkrktt|| |d|ksTtt|| |d|kr}tttt|| d|dt |||<t| d|dD] } || f|| <qt fd d||D|| <| d}|d8}|dks6|dks6t |t |fS) Ncss|]}t|tVqdSN) isinstancetuple.0cr[/var/www/html/software/conda/envs/catlas/lib/python3.10/site-packages/dask/array/reshape.py $sz"reshape_rechunk..cSg|]}dqSrrrirrr 'z#reshape_rechunk..cSrrrrrrrr(rrrc3s$|] }t||kVqdSr)lenr)inchunksinshaperrrDs"c3s|]}|VqdSrrr)prodrrrRs c3s|]}|VqdSrrr)csrrrfs) allr rangerrNotImplementedError_not_implemented_messagemathr#map expand_tuplercontract_tuple) r"Zoutshaper!iiZoiZresult_inchunksZresult_outchunksdindoutZileftrZchunk_reductionZoleftr)r$r!r"r#rreshape_rechunk#sv       ""       &&Ar0cCs|dkr|Sg}|D]+}|}t||d}|d|kr.|t||t|8}|d|ks|r5||q t|t|ks@Jt|S)z >>> expand_tuple((2, 4), 2) (1, 1, 2, 2) >>> expand_tuple((2, 4), 3) (1, 1, 1, 1, 2) >>> expand_tuple((3, 4), 2) (1, 2, 2, 2) >>> expand_tuple((7, 4), 3) (2, 2, 3, 1, 1, 2) r)maxappendintsumr)chunksfactoroutrxpartrrrr+ns    r+cCs\t||dks Jg}d}|D]}||7}||}||}||}|r)||qt|S)zReturn simple chunks tuple such that factor divides all elements Examples -------- >>> contract_tuple((2, 2, 8, 4), 4) (4, 8, 4) r)r5r3r)r6r7r8ZresidualchunkdivZgoodrrrr,s  r,Tcs>ddlm}ddlm}tt||}dd|D}t|t|krSt|t|dkr0tdt|dkr=|jdkr=|S||j t t |dtfdd |D}t t|jrdtd t|jt t |d|j krqtd |j|krx|St|t|}d t||}|jdkrtt|} |fd t|tj| |fi} tdd |D} tj|| |gd} t| || |dSt|j} t|}|s| |kr|ddt| |D}t|j||j\}}t t dd|D|j j!}|durt"t#$d}t#$dd}nt"|}d}||krW|durd}t%j&||ddn8|rWt'|t'|@}g}|D]}||dkrF|(|||d8<q.|(dq.t)||||j |d}||}t*t+|j,ggdd|DR}t*t+|ggdd|DR}t*t+|}ddt-|||D} tj|| |gd} t| |||dS) abReshape array to new shape Parameters ---------- shape : int or tuple of ints The new shape should be compatible with the original shape. If an integer, then the result will be a 1-D array of that length. One shape dimension can be -1. In this case, the value is inferred from the length of the array and remaining dimensions. merge_chunks : bool, default True Whether to merge chunks using the logic in :meth:`dask.array.rechunk` when communication is necessary given the input array chunking and the output shape. With ``merge_chunks==False``, the input array will be rechunked to a chunksize of 1, which can create very many tasks. limit: int (optional) The maximum block size to target in bytes. If no limit is provided, it defaults to using the ``array.chunk-size`` Dask config value. Notes ----- This is a parallelized version of the ``np.reshape`` function with the following limitations: 1. It assumes that the array is stored in `row-major order`_ 2. It only allows for reshapings that collapse or merge dimensions like ``(1, 2, 3, 4) -> (1, 6, 4)`` or ``(64,) -> (4, 4, 4)`` .. _`row-major order`: https://en.wikipedia.org/wiki/Row-_and_column-major_order When communication is necessary this algorithm depends on the logic within rechunk. It endeavors to keep chunk sizes roughly the same when possible. See :ref:`array-chunks.reshaping` for a discussion the tradeoffs of ``merge_chunks``. See Also -------- dask.array.rechunk numpy.reshape r)PerformanceWarning)sanitize_indexcSsg|]}|dkr|qS)rrsrrrrzreshape..rz&can only specify one unknown dimensionc3s |] }|dkr n|VqdS)r?Nrr@Z missing_sizerrrszreshape..z_Array chunk size or shape is unknown. shape: %s Possible solution with x.compute_chunk_sizes()z)total size of new array must be unchangedzreshape-)rcss|]}|fVqdSrr)rdrrrrs) dependencies)metacSsi|]}|dqSrrrrrr szreshape..cSsg|]}t|qSr)r2rrrrrsNzarray.chunk-sizez array.slicing.split-large-chunksTaReshaping is producing a large chunk. To accept the large chunk and silence this warning, set the option >>> with dask.config.set(**{'array.slicing.split_large_chunks': False}): ... array.reshape(shape) To avoid creating the large chunks, set the option >>> with dask.config.set(**{'array.slicing.split_large_chunks': True}): ... array.reshape(shape)Explicitly passing ``limit`` to ``reshape`` will also silence this warning >>> array.reshape(shape, limit='128 MiB')) stacklevelauto)shapelimitdtypeZprevious_chunkscSg|]}tt|qSrr&r rrrrr(rBcSrNrrOrrrrr)rBcSs i|] \}}}|tj||fqSr)rreshape)rabrKrrrrG+s ).dask.array.corer=Zdask.array.slicingr>rr*r ValueErrorndimsizerrnpisnanr5rKstrr r Z npartitionsnextr Z __dask_keys__rrPr Zfrom_collectionsrZrechunkr&r0r6rMitemsizerrgetwarningswarnrr3r listrnamezip)r9rKZ merge_chunksrLr=r>Z known_sizesrFr`keyZdskr6graphr.r/r!Z outchunksZmax_chunksize_in_bytessplitmsgZmatching_chunksZ chunk_planr8Zx2Zin_keysZout_keysZshapesrrCrrPs *           "  rP)TN)# __future__rr)r] collectionsr functoolsr itertoolsroperatorrnumpyrWZdaskrrSrr Zdask.array.utilsr Z dask.baser Z dask.corer Zdask.highlevelgraphr Z dask.utilsrrr(r0r+r,rPrrrrs(          K