DUf{ZddlZddlZddlmZddlZ d dZ d dZdS) N)helpersFTc Z|s|} n<|jdd|i|} |r)| } tj| n| } t | ||i|} |rtj| |r;|| } t | t jrtj| | S| S)z Given a BedTool object `orig_bedtool`, call its `method` with `args` and `kwargs` and then call `reduce_func` on the results. See parallel_apply docstring for details gN)shufflesortrclose_or_deletegetattr isinstance pybedtoolsBedTool) orig_bedtoolshuffle_kwargs genome_fnmethod method_args method_kwargsr r reduce_functo_useshuffledresultreduceds P/var/www/html/software/conda/lib/python3.11/site-packages/pybedtools/parallel.py_parallel_wraprs( '<'FF)F~FF  ]]__F  #H - - - -F $WVV $ $k C] C CF('''+f%% fj0 1 1 ,  #F + + + c # K|pi}|pd}t|ts4t|tstdt |z|pi}|r|rtdr'|s%|stdt j|}t||||||||   fd| dkr,t| D]}tdi|VdS| r| ntj | fd t| D}t|D]]\}}| V|r@tjd |ztj^dS) aA Call an arbitrary BedTool method many times in parallel. An example use-case is to generate a null distribution of intersections, and then compare this to the actual intersections. **Important:** due to a known file handle leak in BedTool.__len__, it's best to simply check the number of lines in the file, as in the below function. This works because BEDTools programs strip any non-interval lines in the results. >>> # set up example BedTools >>> a = pybedtools.example_bedtool('a.bed') >>> b = pybedtools.example_bedtool('b.bed') >>> # Method of `a` to call: >>> method = 'intersect' >>> # Kwargs provided to `a.intersect` each iteration >>> method_kwargs = dict(b=b, u=True) >>> # Function that will be called on the results of >>> # `a.intersect(**method_kwargs)`. >>> def reduce_func(x): ... return sum(1 for _ in open(x.fn)) >>> # Create a small artificial genome for this test (generally you'd >>> # use an assembly name, like "hg19"): >>> genome = dict(chr1=(0, 1000)) >>> # Do 10 iterations using 1 process for this test (generally you'd >>> # use 1000+ iterations, and as many processes as you have CPUs) >>> results = pybedtools.parallel.parallel_apply(a, method, genome=genome, ... method_kwargs=method_kwargs, iterations=10, processes=1, ... reduce_func=reduce_func, debug=True, report_iterations=True) >>> # get results >>> print(list(results)) [1, 0, 1, 2, 4, 2, 2, 1, 2, 4] >>> # We can compare this to the actual intersection: >>> reduce_func(a.intersect(**method_kwargs)) 3 Alternatively, we could use the `a.jaccard` method, which already does the reduction to a dictionary. However, the Jaccard method requires the input to be sorted. Here, we specify `sort=True` to sort each shuffled BedTool before calling its `jaccard` method. >>> from pybedtools.parallel import parallel_apply >>> a = pybedtools.example_bedtool('a.bed') >>> results = parallel_apply(a, method='jaccard', method_args=(b,), ... genome=genome, iterations=3, processes=1, sort=True, debug=True) >>> for i in results: ... print(sorted(i.items())) [('intersection', 12), ('jaccard', 0.0171184), ('n_intersections', 1), ('union', 701)] [('intersection', 0), ('jaccard', 0.0), ('n_intersections', 0), ('union', 527)] [('intersection', 73), ('jaccard', 0.137996), ('n_intersections', 1), ('union', 529)] Parameters ---------- orig_bedtool : BedTool method : str The method of `orig_bedtool` to run method_args : tuple Passed directly to getattr(orig_bedtool, method)() method_kwargs : dict Passed directly to getattr(orig_bedtool, method)() shuffle : bool If True, then `orig_bedtool` will be shuffled at each iteration and that shuffled version's `method` will be called with `method_args` and `method_kwargs`. shuffle_kwargs : dict If `shuffle` is True, these are passed to `orig_bedtool.shuffle()`. You do not need to pass the genome here; that's handled separately by the `genome` and `genome_fn` kwargs. iterations : int Number of iterations to perform genome : string or dict If string, then assume it is the assembly name (e.g., hg19) and get a dictionary of chromsizes for that assembly, then converts to a filename. genome_fn : str Mutually exclusive with `genome`; `genome_fn` must be an existing filename with the chromsizes. Use the `genome` kwarg instead if you'd rather supply an assembly or dict. reduce_func : callable Function or other callable object that accepts, as its only argument, the results from `orig_bedtool.method()`. For example, if you care about the number of results, then you can use `reduce_func=len`. processes : int Number of processes to run. If `processes=1`, then multiprocessing is not used (making it much easier to debug). This argument is ignored if `_orig_pool` is provided. sort : bool If both `shuffle` and `sort` are True, then the shuffled BedTool will then be sorted. Use this if `method` requires sorted input. _orig_pool : multiprocessing.Pool instance If provided, uses `_orig_pool` instead of creating one. In this case, `processes` will be ignored. debug : bool If True, then use the current iteration index as the seed to shuffle. report_iterations : bool If True, then report the number of iterations to stderr. rz+method_args must be a list or tuple, got %sz1only of of genome_fn or genome should be providedz.add_seeds'  1W 1/0F# $V , rrNc \g|](}td|)S)r) apply_asyncr).0it_parallel_wrap_kwargsr$ps r z"parallel_apply..sF  nb((27L*M*MNNrz%s )r listtuple ValueErrortyper chromsizes_to_filedictrangermultiprocessingPool enumerategetsysstderrwriteflush)rrgenomerrrrrr processesr _orig_pool iterationsr#report_iterationsr(resultsr!rr)r$r*s ` ` @@@rparallel_applyrB3s;R$)rN#K k4 ( ( K1O1O  9D >  U#5f==I !%#     A~~ ## H HB GG88B0E#F#FGG G G G G,    + + ##G'""1eegg   J  VaZ ( ( ( J      r)FTN) NNNNNTNrFNrFF)r7r3rr rrBrrrrDs   ))))^    r