DUfb LddlZddlZejejddlZddlmZddlZddl Z ddl Z ddl m Z mZddlmZddlmZmZddlmZmZdd lmZmZdd Z ddZeddddddd ddef dZdZ d dZd!dZ d"dZ! d#dZ"dS)$N)level)partial) threshold_lithreshold_otsu) CSRSelector)peaksnumutils)is_compatible_viewframeis_cooler_balanced)make_cooler_viewpool_decorator c t|}tj|}tj|t}t d|D]}t d|D]}||z|kr d|dd<|dkr||z}n||dxx|d| zcc<|dkr||z}n|d| xx||dzcc<|||r| ndxxd|||r| ndz z cc<|S)zJ Calculate the number of "good" pixels in a diamond at each bin. dtyperFN)lennpzerosboolrange) bad_bin_maskwindow ignore_diagsNn_pixelsloc_bad_bin_maski_shiftj_shifts U/var/www/html/software/conda/lib/python3.11/site-packages/cooltools/api/insulation.py get_n_pixelsr"sd LAx{{Hx...F##Q''  G <//"' QQQ !|| L0   ***l9WH9.EE***!|| L0   7(+++|GHH/EE+++ WG => ? ? ?$WG0M%NOO  ? ? ? ? " OTweightc ||j}|jdz}||z }tj|} tj|} |3t tjdt|||} n?t ||j ||} |dz|dzfd} | D]d} tj | gd }||j |jz |dz d zk}|rJtj|||g}| ||d <|d j }|jj |z }|j j |z }t#d |D]}t#d |D]}||z|kr ||z||z k||z|kz||z d kz}| tj|||z|d j ||z } |r:| tj|||z|z|d j ||z|z } ft'j5t'jd|r| | z }n| | z }|r|tj|z}dddn #1swxYwY|| | | fS)a1 Calculates the insulation score of a Hi-C interaction matrix. Parameters ---------- pixel_query : RangeQuery object A table of Hi-C interactions. Must follow the Cooler columnar format: bin1_id, bin2_id, count, balanced (optional)). bins : pandas.DataFrame A table of bins, is used to determine the span of the matrix and the locations of bad bins. window : int The width (in bins) of the diamond window to calculate the insulation score. ignore_diags : int If > 0, the interactions at separations < `ignore_diags` are ignored when calculating the insulation score. Typically, a few first diagonals of the Hi-C map should be ignored due to contamination with Hi-C artifacts. norm_by_median : bool If True, normalize the insulation score by its NaN-median. clr_weight_name : str or None Name of balancing weight column from the cooler to use. Using raw unbalanced data is not supported for insulation. rNF)rr12c8|d|z|zS)Ncount)pweight1weight2s r!zinsul_diamond..hsaj1W:5' Br#)bin1_idbin2_idr))columnsrbalancedrr)) minlengthignore)indexminmaxrrr"repeatrisnullvalues read_chunkedpd DataFramer0r/coolerannotaterbincountwarningscatch_warnings simplefilter nanmedian) pixel_querybinsrrnorm_by_medianclr_weight_name lo_bin_id hi_bin_idr sum_counts sum_balancedr transform chunk_dictchunk diag_pixelsvalid_pixel_maskijrr maskscorer,r-s @@r! insul_diamondrV2sB   I   1$IIA!J8A;;L IeSYY ' '\      ! ( ( * * 1%    "C'!C'BBBBB !..00    Z1P1P1PQQQEMEM9fqjA=MMN  H /+t_ > @ @ GG    & 2   & 2Q''  G F++  W$|33[AK/7{Q(7{a') bkdGg%{7';'B4'HTU # BK$!112W<#J/6t>N7NO"#%%%L  ,  " " ) )h'''  * 8+EE)E  ) R\%(( (E ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) (L* 44s&;J--J14J1 is_bad_binF-1rc |t|}n7 t||dd} n"#t$r} td| d} ~ wwxYw|r: t ||d} n&#t$r} td|d| d} ~ wwxYw|jd}|R ||jd d |zd }n?#td |xYwt|trntd |tj |r|g}tj |t}||zdk}tj|rtd||d|tt ||||||||| }| ||gdj}t%j|}|S)aCalculate the diamond insulation scores for all bins in a cooler. Parameters ---------- clr : cooler.Cooler A cooler with balanced Hi-C data. window_bp : int or list of integers The size of the sliding diamond window used to calculate the insulation score. If a list is provided, then a insulation score if calculated for each value of window_bp. view_df : bioframe.viewframe or None Viewframe for independent calculation of insulation scores for regions ignore_diags : int | None The number of diagonals to ignore. If None, equals the number of diagonals ignored during IC balancing. min_dist_bad_bin : int The minimal allowed distance to a bad bin to report insulation score. Fills bins that have a bad bin closer than this distance by nans. is_bad_bin_key : str Name of the output column to store bad bins append_raw_scores : bool If True, append columns with raw scores (sum_counts, sum_balanced, n_pixels) to the output table. clr_weight_name : str or None Name of the column in the bin table with weight. Using unbalanced data with `None` will avoid masking "bad" pixels. verbose : bool If True, report real-time progress. nproc : int, optional How many processes to use for calculation. Ignored if map_functor is passed. map_functor : callable, optional Map function to dispatch the matrix chunks to workers. If left unspecified, pool_decorator applies the following defaults: if nproc>1 this defaults to multiprocess.Pool; If nproc=1 this defaults the builtin map. Returns ------- ins_table : pandas.DataFrame A table containing the insulation scores of the genomic bins NT check_sorting raise_errors0view_df is not a valid viewframe or incompatibler\#provided cooler is not balanced or is missingbin-size//bins/rzEignore_diags not provided, and not found in cooler balancing weights z&ignore_diags must be int or None, got rrzThe window sizes z& has to be a multiple of the bin size chromstartendname)r r Exception ValueErrorr info _load_attrsrootrstrip isinstanceintrisscalararrayanyr_get_region_insulationr:r<concat)clr window_bpview_dfrmin_dist_bad_binis_bad_bin_keyappend_raw_scores chunksizerHverbosenproc map_functor_ebin_size bad_win_sizesjobins_region_tables ins_tables r!calculate_insulation_scorerskp"3'' X'"! AA  X X XOPPVW W X "3dKKKAA   RoRRR   x #H ??$$'A'A'AALL iXgii  L# & &R P,PPQQQ {9 K #...I(A-M vm  j - 8 j j`h j j       C$ C1R1R1R)S)Z[[ +,,I s8( AAA A B*A>>B9CC$c t|} t|d| | fd|} |jd} || z} | \}}}}|||g} || }|gd}||jdddf<|r||nd||<|rtjd ||r.| tj || }| | \}}| ||||f}t| D]\}}tj5tjd t$t'||||| \}}}}t(j||d k<t)j|}dddn #1swxYwYt(j|t)j|<||d||<||d||<|r/|jj|k}t(j|j|d||f<|r||d||<||d||<|S)zH Auxilary function to make calculate_insulation_score parallel. rr))shapefieldr|rarerfrgNregionFzProcessing region  dist_bad_binr4)rrrHrlog2_insulation_score_n_valid_pixels_ sum_counts_ sum_balanced_)rrFropenrkfetchcopylocr9loggingassignr dist_to_maskextent enumeraterArBrCRuntimeWarningrVrnanlog2isfiniterr:)rvrzrHr|rwryrr{r}rnbinsselectorr window_binsrerfrgrh region_bins ins_regionc0c1 region_queryrSwin_bin ins_trackrrLrKmask_bads r!rtrts"  OOE  eU^7iH x #Hx'K%E5#tUC F((**""6**K6667<<>>J"&JN111h;1@K O$++---e~2 0$00111 &&!.z./IJJ'  ZZ  FBBrE2b5L)L ,,FF 7  $ & & + +  !(N ; ; ;G :IaL::;7? 3Yq\334  W!.58HHHPRPVJN8%Lil%L%LL M  F7AJ3Yq\33 49EJ5y|55 6 s1AGG G Q?log2_insulation_score_{WINDOW}n_valid_pixels_{WINDOW}c|r3tjfddDd|vr~t}jD]g}t j|d|}|r:|t| dhntdg}fd|D} g} j j } j j } d j _ d d dD]\} }|d g}|D]}||j | |k}|r||jj |kz}|||j |}t#j| \}}t'j|t&jz}|||<|d|}nd}t&j||<||j||f<| |tj| }|d }| |j _ |j| ddf}|S)aKCall insulating boundaries. Find all local minima of the log2(insulation score) and calculate their chromosome-wide topographic prominence. Parameters ---------- ins_table : pandas.DataFrame A bin table with columns containing log2(insulation score), annotation of regions (required), the number of valid pixels per diamond and (optionally) the mask of bad bins. Normally, this should be an output of calculate_insulation_score. view_df : bioframe.viewframe or None Viewframe for independent boundary calls for regions min_frac_valid_pixels : float The minimal fraction of valid pixels in a diamond to be used in boundary picking and prominence calculation. min_dist_bad_bin : int The minimal allowed distance to a bad bin to be used in boundary picking. Ignore bins that have a bad bin closer than this distance. log2_ins_key, n_valid_pixels_key : str The names of the columns containing log2_insulation_score and the number of valid pixels per diamond. When a template containing `{WINDOW}` is provided, the calculation is repeated for all pairs of columns matching the template. Returns ------- ins_table : pandas.DataFrame A bin table with appended columns with boundary prominences. crg|]3\}}|tj|4S)r)rr r).0rdfrzs r! z#find_boundaries..sK   FB x'z#find_boundaries..sX  Y)000<< = A A C C  r# sorting_indexFT)dropinplacerfboundary_strength_boundary_strength)r<rugroupbysetr1rematchraddrpgroupsr5rhr: reset_index sort_valuesrr find_peak_prominencer zeros_likerrappend set_index)rrry log2_ins_keyrrzwindowscolmmin_valid_pixelsdfs index_name sorting_orderrrrrTrposspromsins_prom_trackbs_keys`` `` r!find_boundariesresP I    "+"3"3H"="=     \!!%%$ 0 0C,,H,==sCCA 0 C 1 ../// 0 tf++ C%JO*M*IO ud333''11  ^^WI & & 2 2C%,,C,889@#C()    C.2BBB<..c.::;B4HI4iZ@@KD%]955>N#(N4 3c33,BvJ#1BF4<  2 3B o & &BBHM  qqq !B Ir#cp|rK|}t| dz|D]"}tj|tj|#t j5t jd|j d}tjt j |z}td|D]Q}td|dz|z }t||z|}t j |||dz||f||<R|r|t j|z}dddn #1swxYwY|S)a> Calculates the insulation score of a Hi-C interaction matrix. Parameters ---------- mat : numpy.array A dense square matrix of Hi-C interaction frequencies. May contain nans, e.g. in rows/columns excluded from the analysis. window : int The width of the window to calculate the insulation score. ignore_diags : int If > 0, the interactions at separations < `ignore_diags` are ignored when calculating the insulation score. Typically, a few first diagonals of the Hi-C map should be ignored due to contamination with Hi-C artifacts. norm_by_median : bool If True, normalize the insulation score by its NaN-median. Returns ------- score : ndarray an array with normalized insulation scores for provided matrix rr4rN)rrr set_diagrrrArBrCronesr7r6nanmeanrD) matrrrGrRrrUlohis r!_insul_diamond_densersv6.hhjj }q(,77 . .A  c261 - - - -  " " ) )h''' IaL#q! 9 9AQA''BQZ##Bz#b1q5j!B$&6"788E!HH  ) R\%(( (E ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) Ls!B>D++D/2D/順c |t|}n7 t||dd}n"#t$r}td|d}~wwxYw|jd}|r: t ||d}n&#t$r}td|d|d}~wwxYw|:||jd d |zd }n)t|trntd |d ||z} ||zdkrtd|d|g} |gdj D]A\} } } }| | | g}| |gd}tj| ||j }|| |}t#j5t#jdt(t+|| |}tj||dk<tj|}dddn #1swxYwYtj|}t3d|D]U}|dkr||z}|tjdg|z|d| fz}|tj||ddg|zfz}Vtj||<||d<tj|tj|<||d|<t9j| \}}tj|tjz}|||<||d|<||d|<| |Ct?j | }|S)a.Calculate the diamond insulation scores and call insulating boundaries. Parameters ---------- clr : cooler.Cooler A cooler with balanced Hi-C data. Balancing weights are required for the detection of bad_bins. window_bp : int The size of the sliding diamond window used to calculate the insulation score. view_df : bioframe.viewframe or None Viewframe for independent calculation of insulation scores for regions clr_weight_name : str Name of the column in bin table that stores the balancing weights. min_dist_bad_bin : int The minimal allowed distance to a bad bin. Do not calculate insulation scores for bins having a bad bin closer than this distance. ignore_diags : int The number of diagonals to ignore. If None, equals the number of diagonals ignored during IC balancing. Returns ------- ins_table : pandas.DataFrame A table containing the insulation scores of the genomic bins and the insulating boundary strengths. NTrZr]rar^r_r`rbrcrzprovided ignore_diags z is not int or NonerzThe window size (z') has to be a multiple of the bin size rdr)balancer4bad_bin_maskedrr)!r r rirjrkr rlrmrnrorpr:rFrrisnanmatrixrArBrCrrrrrrr_rr rrr<ru)rvrwrxrHryrrrrrrrerfrgrhrrrWrrbad_bin_neighborrRrrrrs r!!_find_insulating_boundaries_densers`H"3'' X'"! AA  X X XOPPVW W Xx #H "3dKKKAA   RoRRR    HOOC #=O#=#= =    L# & &U S,SSSTTTx'K8q   \ \ \RZ \ \   #*+L+L+L#M#T"-"-uc4$XXZZ%%f--.G.G.GH Xchhjj..v66GNOO JJJ / / 5 5f = =  $ & & + +  !(N ; ; ;,Q \JJI(*Ii1n % **I  + + + + + + + + + + + + + + + =44q*++ X XAAvv#3j#@  #3beTFQJ SVUVTVSV.sa<??&:r#Otsuc(|t|kSr)rrs r!r.zinsulation..sa>!+<+<&<r#c|kSrr*)rthrs r!r.zinsulation..s !s(r#zAInsulating boundary strength threshold can be Li, Otsu or a float) rxrwrryr{rHr|r}r~)rryr is_boundary_)r r rirjfloatrrr:)rvrwrxrrHrry thresholdr{r|r}r~rrthresholding_funcrrstrong_boundariesrs @r! insulationrpsv"3'' X'#! AA X X XOPPVW W XD:: f  <<  ""C 2 2 2 2     S   + !)+'   I 3)I <<-- 0300 1 8  +< &&&'' s ) AAA A55B)rr)rrTr$)rrrrrW)rrT)rNr$rN) NNr$rrrFrXFr)#rr basicConfigINFOrA functoolsrnumpyrpandasr<r>skimage.filtersrr lib._queryrlibr r lib.checksr r lib.commonr rr"rVmaprrtrrrrr*r#r!rs ','''' 88888888$$$$$$!!!!!!!!DDDDDDDD99999999@  e5e5e5e5N   zzzzzKKK`10 bbbbJ,,,,b  ttttt   rrrrrrr#