o Nrf@sddlmZddlmZddlZddlZddlZzddl m Z Wn e y,e ddwddl m ZddlmZddlmZddlmZmZdd lmZmZd d Zdd d ZdddZddZdddZddZ dddZ dS))product)ceilN)COOz,The 'sparse' package is required to use dask)tokenize) CSRReader query_rect)parse_cooler_uri partitionc st|dZ}|||durt}t|d}i}|D]}tj|jd}|dur9t||jd||<q!t j fdd|D|d}|D]}t j g||dd ||<qJWdn1scwY||||fS) Nrr)enum)keycs"i|] }|tjg|jdqS))dtype)nparrayr.0r grp\/var/www/html/software/conda/envs/catlas/lib/python3.10/site-packages/cooler/sandbox/dask.py &"z#_get_group_info..)columnsT) categoriesordered) h5pyFilelistkeyslenZ check_dtypersorted __getitem__pd DataFrame Categorical) path grouppathrfnrows categoricalsr dtmetarrr_get_group_infos&  r-cCsz=|dur |t|d}||||WdW|dur(|SS1s,wYW|dur<|dSdS|durG|ww)Nr acquirerrrelease)filepathr'r slclockr(rrr_slice_dataset/s   r4csz@|dur |t|dfdd|DWdW|dur+|SS1s/wYW|dur?|dSdS|durJ|ww)Nr csi|] }||qSrrrr(r'r2rrr?sz _slice_group..r.)r1r'rr2r3rr5r _slice_group:s   r6cCs0|D]\}}tjj|||dd||<q|S)NT)r)itemsr#r%Z from_codes)dataZcategorical_columnsr Z category_dictrrr_restore_categoriesEsr9逖cCs t|\}}t|||\}}}} t||||} d| } dttd||dd} | d|dkr:g| |dR} i} tdtt||D](}t|||d|}t|||||f}| rct || f}t j |d|j f| | |f<qGt | | || }|dur|j|ddd }|S) a Create a dask dataframe around a column-oriented table in HDF5. A table is a group containing equal-length 1D datasets. Parameters ---------- group_uri : str URI to the HDF5 group storing the table. keys : list, optional list of HDF5 Dataset keys, default is to use all keys in the group chunksize : int, optional Chunk size index : str, optional Sorted column to use as index lock : multiprocessing.Lock, optional Lock to serialize HDF5 read/write access. Default is no lock. Returns ------- :class:`dask.dataframe.DataFrame` Notes ----- Learn more about the `dask `_ project. zdaskify-h5py-table-)rNrTF)r!Zdrop)r r-rtuplerangeintrslicer6r9r#r$rddZ set_index)Z group_urir chunksizeindexr3r1r'r)r,r*token task_nameZ divisionsdskir2Z data_dictdfrrr read_tableKs,  rIc Cs|j}|d;}|d|j} t||dd} |r(t| j||||dd\} } } n | ||||\} } } t| s<| | } Wdn1sFwYt| || |f| ||||fd}|sd| }|S)Nr Zpixelsie)Z max_chunkT)Zduplex)shape) Z_is_symm_upperopenrrrqueryr ZastyperZtodense)clri0i1j0j1field sparse_arrayZis_upperZh5rreaderrGjvZarrrrr _array_selects  & rWcountFcstj||||}d|} ||f} tdddd} fddtd| d|D} tdd| D| dff} tt| ggdd| DR}fd d| D}tt||}t j || | | | d S) aj Create a parallel Dask array around the matrix representation of a cooler. Parameters ---------- clr : :class:`cooler.Cooler` Cooler object i0, i1 : int Row query range j0, j1 : int Column query range field : str Value column to query sparse_array : bool, optional Create a dask array backed by :class:`sparse.COO` sparse arrays instead of dense numpy arrays (default). chunksize : int, optional Length of the rowwise chunks to partition the underlying data into. Returns ------- :class:`dask.array.Array` zcooler-array-slice-rcsg|] \}}||fqSrr)rlohi)rPrQrr sz#load_dask_array..css |] }|d|dVqdS)r<rNr)rsrrr sz"load_dask_array..r<cSsg|]}tt|qSr)r>r )rdimrrrr\scs"g|] }tg|RqSr)rW)rr2)rMrRrSrrr\r)r,rJ) rurirWr r=rrdictzipdaZArray)rMrNrOrPrQrRrSrBrDrErJr,ZsliceschunksrvaluesrFr)rMrRrPrQrSrload_dask_arrays  rf)N)Nr:NN)rXFrY)! itertoolsrmathrrnumpyrZpandasr#sparser ImportErrorZ dask.arrayrrcZdask.dataframeZ dataframerAZ dask.basercorerrutilr r r-r4r6r9rIrWrfrrrrs.         ;