tfkcddlmZddlZddlZddlmZmZddlmZddl m Z m Z m Z m Z mZddlZddlZddlZddlmZmZddlmZmZd*d Zd+d Zd,d Zd,d Zd-d Z d. d/dZej@dejBfd0dZ"d1dZ#d2dZ$ d3 d4dZ%d5dZ&d6dZ'd7dZ(e(Z)d8dZ*d9dZ+d:dZ, d;dZ-ddZ0d.d?dZ1e d@ dAdZ2Gd d!ejfZ4dBd"Z5d.d#Z6dddejnfd$Z8dCd%Z9 d. dDd'Z:Gd(d&Z; dE dFd)Z>> for lo, hi in partition(0, 9, 2): print(lo, hi) 0 2 2 4 4 6 6 8 8 9 c3@K|]}|t|zfywN)min).0istepstops U/var/www/html/software/conda/envs/higlass/lib/python3.12/site-packages/cooler/util.py zpartition.. s! GQAHd# $ Gs)range)startrrs ``r partitionrs HeE4.F GGc|jd}t|dk(r |dd}}||fSt|dk(r|\}}|jdsd|z}||fStd)zW Parse a Cooler URI string e.g. /path/to/mycoolers.cool::/path/to/cooler z::rr/zInvalid Cooler URI string)splitlen startswith ValueError)sparts file_path group_paths rparse_cooler_urir*#s GGDME 5zQ %a#:  j  Uq % :$$S)z)J j  455rc8t|jddS)N,)intreplace)r&s ratoir06s qyyb! ""rctjd}|j|jdd\}}}t |s t |St |}|jj}|dvr|dz}t |S|dvr|dz}t |S|dvr|d z}t |Std |d ) Nz ([0-9,.]+)r,r-)KKBi)MMBi@B)GGBiʚ;zUnknown unit '') recompiler"r/r#r.floatupperstripr%)r& _NUMERIC_RE_valueunits rparse_humanizedrB:s**\*K &&qyyb'9:NAud t95z %LE ::<   D {   u:    u:    u:>$q122rcd}dfd}|jd}|dj}t|s tdt|dkr|ddfS|||d \}}|||fS) ac Parse a UCSC-style genomic region string into a triple. Parameters ---------- s : str UCSC-style string, e.g. "chr5:10,100,000-30,000,000". Ensembl and FASTA style sequence names are allowed. End coordinate must be greater than or equal to start. Returns ------- (str, int or None, int or None) c 32Kgd}dj|Dcgc]}d|dd|ddc}}tjd|tj}|j |D]#}|j }||j |f%ycc}ww) N))HYPHEN-)COORDz[0-9,]+(\.[0-9]*)?(?:[a-z]+)?)OTHERz.+z|\s*z(?Pr)z\s*)joinr9r: IGNORECASEfinditer lastgroupgroup)r& token_specpairpattern tok_regexmatchtyps r _tokenizez&parse_region_string.._tokenize^s ,,T4Qy$q'! <TUJJ#gY/? ''* (E//Cu{{3'' ' ( UsBBA+Bc|)tdjdj|||vrtd|dy)NzExpected {} token missingz or zUnexpected token "")r%formatrK)rUtokenexpecteds r _check_tokenz)parse_region_string.._check_tokenjsI ;8?? H@UVW W(" #5eWA!>??#rct|d\}}||dgt|}t|d\}}||dgt|d\}}||dfS||dgt|}||kr td||fS)N)NNrGrEzEnd coordinate less than start)nextrBr%)tokensrUrZrendr\s r_expectz$parse_region_string.._expectqs&,/ US%'+&&,/ US%(,&,/ U ;$; S%'+e$ ;=> >czr:rzChromosome name cannot be emptyr!Nr)r"r=r#r%)r&rVrar'chromrr`r\s @rparse_region_stringrdMs" (@& GGCLE !HNN E u::;; 5zA~tT""58,-JE3 5# rcrt|trt|\}}}n$|\}}}| t|n|}| t|n|} |||nd}|dn|}|| t d|}||kr t d|dks|||kDrt d|d|d|||fS#t$r}t d||d}~wwxYw) aU Genomic regions are represented as half-open intervals (0-based starts, 1-based ends) along the length coordinate of a contig/scaffold/chromosome. Parameters ---------- reg : str or tuple UCSC-style genomic region string, or Triple (chrom, start, end), where ``start`` or ``end`` may be ``None``. chromsizes : mapping, optional Lookup table of scaffold lengths to check against ``chrom`` and the ``end`` coordinate. Required if ``end`` is not supplied. Returns ------- A well-formed genomic region triple (str, int, int) NzUnknown sequence label: rz Cannot determine end coordinate.zEnd cannot be less than startzGenomic region out of bounds: [z, rJ) isinstancestrrdr.KeyErrorr%)reg chromsizesrcrr`clenes r parse_regionrms,#s/4ucuc#/E U/c#hsD$.$:z% AEE { <?@ @ U{899 qyT%#*:5'C5JKK %  D3E7;.s5!{1~5s)npasarrayr#arrayr.roziplexsort)rcolss r argnatsortrsT JJu E u:xx#&& 5u56 7D ::d4R4j !!rc t|tr#|jdr|jddt j |fdddgddgdtid |}|smg}|D]O}||djj |}|jt|d}|j|Qt j|d }|dj|_ |dS) at Parse a ``.chrom.sizes`` or ``.chromInfo.txt`` file from the UCSC database, where ``db`` is a genome assembly name. Parameters ---------- filepath_or : str or file-like Path or url to text file, or buffer. name_patterns : sequence, optional Sequence of regular expressions to capture desired sequence names. Each corresponding set of records will be sorted in natural order. all_names : bool, optional Whether to return all contigs listed in the file. Default is ``False``. Returns ------- :py:class:`pandas.Series` Series of integer bp lengths indexed by sequence name. References ---------- * `UCSC assembly terminology `_ * `GRC assembly terminology `_ z.gz compressiongzip rrnamelength)sepusecolsnamesr{axis) rfrgendswith setdefaultpdread_csvcontainsilocrappendconcatvaluesindex) filepath_or name_patterns all_nameskwargs chromtabler'rRparts rread_chromsizesrs@+s# (<( sequence record rNzNeed at least one filerT)as_rawc3,K|] }||f ywr)rrcfas rrzload_fasta...s@5"U),@s)pyfaidxr#r%Fastaupdaterecordsr)r filepathsrfilepathrrs @r load_fastars" 9~122 9~ ]]9Q<] 5! DH IIgmmHTm:BB C D@%@@G Nrcfd}tjt|jdd}tj|dt j d|d<|S)af Divide a genome into evenly sized bins. Parameters ---------- chromsizes : Series pandas Series indexed by chromosome name with chromosome lengths in bp. binsize : int size of bins in bp Returns ------- bins : :py:class:`pandas.DataFrame` Dataframe with columns: ``chrom``, ``start``, ``end``. c|}ttj|z }tjd|dzz}||d<t j |g|z|dd|dddgdSNrrr})rcrr`columns)r.r~ceilaranger DataFrame)rcrkn_binsbinedgesbinsizerjs r_eachzbinnify.._eachDsy% RWWTG^,-99Q!-7 ||g&#2xPQPR| T-  rrTr ignore_indexrc categoriesordered)rrmapkeys Categoricallistr)rjrrbintables`` rbinnifyr2s[$ yyUJOO$56QTRHd:+;+;&._eachzsjj]51!456uuQC1A5s3x?@GGQd)a- g'$s)DH M-  rTr) Bio.Restriction RestrictionBio.Seqr ImportErrorrgetattrsearchAttributeErrorr%rrr)renzymebiorstchromsrlrrrs` @@rdigestrZs"(    !FBVV,33   99S'ad CC/  B   B09:ABs" A!A:!A7: BBBct}|jddD]J\}}|j|d|dz jddj t |dkDsJyt |dk(rt t|Sy) z Infer bin size from a bin DataFrame. Assumes that the last bin of each contig is allowed to differ in size from the rest. Returns ------- int or None if bins are non-uniform rcT)observedr`rNr}r)setgroupbyrruniquer#r^iter)binssizes_chromrOs r get_binsizers EEg=  eElU7^399#2>EEGH u:> 5zQDK  rc|jdgdddgjdjddd  }t|dt|d}}t j || S) z Infer chromsizes Series from a bin DataFrame. Assumes that the last bin of each contig is allowed to differ in size from the rest. Returns ------- int or None if bins are non-uniform rclast)keepr`T)droprr)rcr`rrdata)drop_duplicates reset_indexrenamerrSeries)rrrlengthss rget_chromsizesrsu gYV4gu5EF $  &: ; :f-.Z5I0JGF 996 00rc t||\}}}|j|}|dkDs|||krU|djj|d}||dj|dj|dz}|j||}|S)zN Range query on a BED-like dataframe with non-overlapping intervals. rr`rightsiderNleft)rm get_groupr searchsortedr) groupedrjregionrcrr`resultlohis rbedslicers%VZ8E5#   u %F qyC*U++ E] ! ! . .u7 . C &/((-::3V:L LR# Mrcdt|tjr|Stj|Sr)rfh5pyDatasetr~r)rrs rasarray_or_datasetrs"1dll+1>A>rctj}t|}t|}|dk(rVtjgt tjgt tjg|j fS||}gg}}tj}td||D]h}||||z}||dd|ddk7dz} |d|k7rtjd| f} |j|| z|j|| |d}jtj|}tjtj||f} tj|}|| |fS)aS Run length encoding. Based on http://stackoverflow.com/a/32681075, which is based on the rle function from R. Parameters ---------- x : 1D array_like Input array to encode dropna: bool, optional Drop all runs of NaNs. Returns ------- start positions, run lengths, run values rrzNrr}) r~ flatnonzerorr#rr.r{nanrrr concatenatediff) r chunksizewherenstartsrlast_valrrrlocsrs rrlencodersQ* NNE u %E E AAv HHRs # HHRs # HHRu{{ +   FFvvH 1a # !a)m $QqrUaf_%) Q48 55D>D a$h agR5^^F #FggbeeFAI&'G ^^F #F 7F ""rctfdtjdjtjDS)Nc3K|]F}tjtjj|tjHywr)osaccesspathrKX_OK)rrcmds rrzcmd_exists..s7  "'',,tS)2773sA APATH)anyrenvironr"pathsep)rs`r cmd_existsrs5 JJv&,,RZZ8 rc tjtj|tj||z |Sr)r~medianabs)rrs rmadr!s, 99RVVD299T4#8894 @@rc/Kt|trd}tj||g|i|}njd}|dk(r|jj dk(rnG|dvr$|jj dk(r t d|dk(r t d|d vr t d |} ||r|jy y #|r|jwwxYww) a Context manager like ``h5py.File`` but accepts already open HDF5 file handles which do not get closed on teardown. Parameters ---------- fp : str or ``h5py.File`` object If an open file object is provided, it passes through unchanged, provided that the requested mode is compatible. If a filepath is passed, the context manager will close the file on tear down. mode : str * r Readonly, file must exist * r+ Read/write, file must exist * a Read/write if exists, create otherwise * w Truncate if exists, create otherwise * w- or x Fail if exists, create otherwise TFrr+)r$az%File object provided is not writeablewzCannot truncate open file)zw-rrz File existsN)rfrgrFilefilemoder%close)fpr)argsrown_fhfhs r open_hdf5r/s6"c YYr4 1$ 1& 1 3;277<<4/  [ RWW\\S%8DE E S[89 9 [ ]+ +   HHJ 6 HHJ sBC B3C 3CC c8eZdZdfd ZddZddZddZxZS) closing_hdf5c8t||jyr)super__init__id)selfgrp __class__s rr4zclosing_hdf5.__init__6s  rc|Srrr6s r __enter__zclosing_hdf5.__enter__9s rc6|jjSrr(r*)r6exc_infos r__exit__zclosing_hdf5.__exit__<syy  rc8|jjyrr=r:s rr*zclosing_hdf5.close?s r)r7 h5py.Group)returnrA)rBNone)__name__ __module__ __qualname__r4r;r?r* __classcell__)r8s@rr1r15s!!rr1ct|}|jD]\}} |j||<|S#t$r|j ||<Y>> make_meta([('a', 'i8'), ('b', 'O')]) Empty DataFrame Columns: [a, b] Index: [] >>> make_meta(('a', 'f8')) Series([], Name: a, dtype: float64) >>> make_meta('i8') 1 T z 1970-01-01rfoo)bVr4mSr%UO__UNKNOWN_CATEGORIES__c.|jdvr|jdS|jdk(r|jtddS|jvr0|j}|jdvr|j|S|St d|)N)rfurcr)rWr4zCan't handle dtype: )kindtypecomplexr TypeError)r{o_simple_fake_mappings r_scalar_from_dtypez&infer_meta.._scalar_from_dtype|s :: (::a= ZZ3 ::gam, , ZZ/ /$UZZ0A&+jjJ&>188E? EA E25':; ;rcdt|tjtjtjfr|St j |r>t|dr |jnt jt|}|Stdt|jd)Nr{zCan't handle meta of type 'r8) rfr Timestamp TimedeltaPeriodr~isscalarhasattrr{rarcrD)rrr{rfs r_nonempty_scalarz$infer_meta.._nonempty_scalars| a",, bii@ AH [[^&q'2AGGa8IE%e, ,|dk(r9tjtjg||j ddStjg|||S)Ncategory)rrr)r{rr)rfrgrrrr)rr{rUNKNOWN_CATEGORIESs r _empty_seriesz!infer_meta.._empty_seriess^ eS !ez&999 2344ud2A yy5t5AAr_metar)rr!c3\K|]$}t|txrt|dk(&yw)r!N)rfror#)rrs rrzinfer_meta..s'CA:a'7CFaK7Cs*,z2Expected iterable of tuples of (name, dtype), got rrr{z'Don't know how to create metadata from r)r~bool_void datetime64 timedelta64str_unicode_rlrrrfrrrrIndexrIrJror#rallr%r{r rc) rrrrmrqr_dr{rprfres @@@r infer_metar~Os9BXXd^ WWT] ]]< ( ^^A  WWU^ WWU^ [[   2 <RBq'ww!bii./vva{ Arxx 1v ]Ea E!T||?@wwy IVaQ a%0 0 IQV  !UA! QqT1Q4u55 Ae} %CCCGsK ||?@ AVaQ a%0 0 A#$%41aQ%  Q Q] HHQKE%e, , |"" =aSA BB9 J B%  sI+ *I1  I7 -I==Jc t|}ttstfdnMj }tfd|j D]\}}t |r||n|}||<||durtjg} n|D cgc]} tjg| } } t| dk(rtj| d|d} n!tjj| |} |jt|D]\} } |j| | z |Dcic]}|tjg|}}tj ||| Scc} wcc}w) ze Extracted and modified from pandas/io/parsers.py : _get_empty_meta (BSD licensed). cxsSrr) default_dtyper{srzget_meta..s E$:]rcSrr)rsrrzget_meta..sMrFrzrr)r)rrt)rrfrIrcopyrJr rr{rr# MultiIndex from_arrayssort enumeratepopr)rr{ index_columns index_namesr_dtyperOrPcolrrrrr col_namecol_dicts ` ` rget_metarsj7mG eT ":;12LLN DAq *1 '!*1CE#J   6 =HIT "E$K0II t9>HHT!W;q>:EMM--d+-FEm, DAq KKA  PWW8"))BeHo>>WHW <<' ??JXs !F"FcXt|djtj}|j }|s:tj |j t|jd|d<|S|djj|jk(jsJ|S)NrcTr) rfr{rCategoricalDtyperrrcrrcatrr|)rrjis_cats r check_binsrs W ++R-@-@ AF 99;D  JJ4 (8(8#94 W KW !!,, 0@0@@EEGGG KrGenomeSegmentationc  |j}|j}||}|j}||j|z }|j||z }g}|D]\ } |vr |j } t t j||j z } | jjdd| } | d| k7rt j| | f} |j fdt| dd| ddD|S)Nr}c3.K|] \}}||fywrr)rrr`rcs rrz%balanced_partition..s! $.E3UE3  sr) _bins_groupedsizeidxmaxlocrjr.r~rrrrextendr)gs n_chunk_max file_contigsloadingsr chrom_nbinschrmaxconstgrangesrOrkranchorsrcs @rbalanced_partitionrsG,,.K __ F(,,v..H OOF #k 1EG  u  $ }}U#27758<<#6678++$$VtV, 2;$ eeGTM*G 25gcrlGABK2P     NrceZdZddZddZy)rct||}|jddd|_|jjj}||_t ||_t|j|_ ||_ tj|jtt||_t"j$dt#j&|f|_t"j$dt#j&|jf|_|j*|dj,j.|djz|_y)NrcTF)rrrrr)rrrrrrjrrrrcontigsrrrrr#idmapr~rcumsumchrom_binoffset chrom_absposrcodes start_abspos)r6rjrnbins_per_chroms rr4zGenomeSegmentation.__init__s$ +!\\'Du\M,,113::$"4( JOO-.  YYZ__%6U3z?=ST !uuQ /(B%BCEE!RYYz/@/@%A"AB   d7m//55 6g9M9M M rcHt||j\}}}|jj|}|dkDs||j|krU|djj |d}||dj|dj |dz}|j ||}|S)Nrr`rrrr)rmrjrrrrr)r6rrcrr`rrrs rfetchzGenomeSegmentation.fetch(s(Auc##--e4 19dooe44%%225w2GBfWo,,RS1>>s>PPB[[B'F rN)rj pd.Seriesr pd.DataFrame)rrrBr)rDrErFr4rrrrrrs  rc#Kg}d}|D]D}|t|z }|j|||kDs(tj|dg}d}Ft|rtj|dyyw)a Take an incoming iterator of small data frame chunks and buffer them into an outgoing iterator of larger chunks. Parameters ---------- chunks : iterator of :py:class:`pandas.DataFrame` Each chunk should have the same column names. size : int Minimum length of output chunks. Yields ------ Larger outgoing :py:class:`pandas.DataFrame` chunks made from concatenating the incoming ones. rrN)r#rrr)chunksrbufr chunks rbufferedr2s{* C A SZ 5 t8))Ca( (CA  3xii!$$s .A6AA6)rr.rr.rr.rBzIterator[tuple[int, int]])r&rgrBztuple[str, str])r&rgrBr.)r&rgrBz"tuple[str, int | None, int | None]r)rirrjzdict | pd.Series | NonerBr)r&rgrBro)rw Iterable[str]rB list[str])rrrB np.ndarray))z ^chr[0-9]+$z ^chr[XY]$z^chrM$F)rz str | IO[str]rztuple[str, ...]rboolrBr)rrgrBr)rrrrgrBOrderedDict[str, Any])rjrrr.rBr)rrrrgrBr)rrrB int | None)rrrBr)rjzpd.Series | dictrrrBr)rrrrBznp.ndarray | h5py.Dataset)rrr rrBz)tuple[np.ndarray, np.ndarray, np.ndarray])rrgrBr)rrrrrBr)r#)r+zstr | h5py.Groupr)rgrBzContextManager[h5py.Group])rMzh5py.AttributeManagerrBrI)rrrjrrBr) rrrr.rrrzlist[int | float] | NonerBzlist[GenomicRangeTuple])i)rzIterable[pd.DataFrame]rr.rBzIterator[pd.DataFrame])= __future__rrr9 collectionsrr contextlibrtypingrrr r r rnumpyr~pandasrpandas.api.typesr r _typingrrrr*r0rBrdrmr:rYrsrxrrrrr make_bintablerrrrrrrr!r/Groupr1rQr~object_rrrrrrrrrs" 0%>> 2=H$!&#&>F+/. .'..b#-"**Xrtt"<R-"&O2 2 "2 2  2 j D"J +D\*1&  " $? !0# 0#0#/0#fA-- -  --` 4::  nCdtRZZ(@V "*. '   >:% "% %%r