o Nrf@sddlZddlZddlZddlZddlZddlmZddl m Z ddl m Z ddl mZmZddl mZmZeejd ed d ejd d dddejdddddejddedddejddedejddedddejddedddejdd edddejd!d"ejdd#dejd$d%d&ed'ddejd(d)d*eed+ddejd,d-ed.ddejd/d0ed1ddejd2d3ed4ddejd5d6d7dddejd8d9dddejd:d;dddejdd?ddd@dAZdS)BN)Pool)ice)Cooler)bedsliceparse_cooler_uri)cli get_loggercool_uriZ COOL_PATH)typemetavarz --cis-onlyzMCalculate weights against intra-chromosomal data only instead of genome-wide.TF)helpZis_flagdefaultz --trans-onlyzMCalculate weights against inter-chromosomal data only instead of genome-wide.z--ignore-diagszNumber of diagonals of the contact matrix to ignore, including the main diagonal. Examples: 0 ignores nothing, 1 ignores the main diagonal, 2 ignores diagonals (-1, 0, 1), etc.)rr rZ show_defaultz --ignore-distzDistance from the diagonal in bp to ignore. The maximum of the corresponding number of diagonals and `--ignore-diags` will be used.)rr z --mad-maxzIgnore bins from the contact matrix using the 'MAD-max' filter: bins whose log marginal sum is less than ``mad-max`` median absolute deviations below the median log marginal sum of all the bins in the same chromosome.z --min-nnzz_Ignore bins from the contact matrix whose marginal number of nonzeros is less than this number. z --min-countzRIgnore bins from the contact matrix whose marginal count is less than this number.z --blacklistzPath to a 3-column BED file containing genomic regions to mask out during the balancing procedure, e.g. sequence gaps or regions of poor mappability.)existsz--nprocz-pz.Number of processes to split the work between.z --chunksizez-czFControl the number of pixels handled by each worker process at a time.gcAz--tolzKThreshold value of variance of the marginals for the algorithm to converge.gh㈵>z --max-iterszGMaximum number of iterations to perform if convergence is not achieved.z--namezName of column to write to.weightz--forcez-fz=Overwrite the target dataset, 'weight', if it already exists.z--checkz4Check whether a data column 'weight' already exists.z--stdoutz8Print weight column to stdout instead of saving to file.z--convergence-policyahWhat to do with weights when balancing doesn't converge in max_iters. 'store_final': Store the final result, regardless of whether the iterations converge to the specified tolerance; 'store_nan': Store a vector of NaN values to indicate that the matrix failed to converge; 'discard': Store nothing and exit gracefully; 'error': Abort with non-zero exit status.) store_final store_nandiscarderrorrc&Cstt}t|\}}|rMt|d3}||}| |dvr.t|d| dtdnt|d|dtdWd n1sHwY| rV| rVt d t|d .}||}| |dvr|s| s~t d | d dtj dtdn|d| =Wd n1swY| d|dt |}|d ur dd l}t|%}tj|d||drdnd gdgddtid}Wd n1swY|d d jddd}|j}g}|D]\}}t|||j|j|jf} || jj qt!"|}nd }|d ur!t#|t$t!%||j&}z/|dkr/t'|}!|!j(}"nt)}"t*j+||| | |||||| |dd|"d\}#}$W|dkrP|!,n |dkr[|!,wwt!-|$ds|.d|dkrt|.dn1|d kr|.d!t!j/|#d d <n|d"kr|.d#tdn|d$kr|.d#td|rt0|#j1tj2ddd%d&d'd St|d )}||}d(d)d*}%|dj3| fd+|#i|%|d| j45|$Wd d S1swYd S),z Out-of-core matrix balancing. Matrix must be symmetric. See the help for various filtering options to mask out poorly mapped bins. COOL_PATH : Path to a COOL file. rbinsz: No 'z' column found.rz::z is balanced.rNz8Provide at most one of --cis-only and --trans-only flagszr+'z' column already exists. z Use --force option to overwrite.)filez Balancing "" i)rrr)chromstartendr )sepheaderZusecolsnamesZdtypeT)ZobservedF) chunksizecis_only trans_onlytolmin_nnz min_count blacklistmad_max max_iters ignore_diagsZrescale_marginalsZuse_lockmapZ convergedz+Iteration limit reached without convergencerz6Storing final result. Check log to assess convergence.rzSaving weights as NaN.rzDiscarding result and aborting.rz%g)r$indexZna_repZ float_formatgzip) compressionZcompression_optsdata)6r __name__rh5pyFileclickZechosysexitZ UsageErrorprintstderrinforcsvopenpdZread_csvSniffer has_headerreadstrrgroupby chromsizesZiterrowsrr r!r"appendr2valuesnpZ concatenatemaxintceilZbinsizerZimap_unorderedr0rZiterative_correctioncloseallrnanZSeriesZ to_stringstdoutZcreate_datasetattrsupdate)&r Znprocr&r-r*r+r,r/r)r'r(r.nameforcecheckrRZconvergence_policyZ ignore_distloggerZ cool_pathZ group_pathZh5grpZclrr@fZ bad_regionsZ bins_groupedrHZbad_bins_regresultpoolZmap_ZbiasstatsZh5optsr`[/var/www/html/software/conda/envs/catlas/lib/python3.10/site-packages/cooler/cli/balance.pybalances#                                $rb)r;r:r8numpyrKZpandasrBZ multiprocessrr1rapirutilrrr r commandargumentrFoptionrMPathfloatZChoicerbr`r`r`ras       2