DUfw&ddlZ ddZd dZdS) NsnappyT@c  ddl}ddl}n#t$rtdwxYwt|tjr|f} t |D][\} } |j| d} | dkr|j j || j f|||d|} | | |\ | dS#| wxYw)a0 Save an iterable of dataframe chunks to a single Apache Parquet file. For more info about Parquet, see https://arrow.apache.org/docs/python/parquet.html. Parameters ---------- pieces : DataFrame or iterable of DataFrame Chunks to write outpath : str Path to output file row_group_size : int Number of rows per row group compression : {'snappy', 'gzip', 'brotli', 'none'}, optional Compression algorithm. Can be set on a per-column basis with a dictionary of column names to compression lib. use_dictionary : bool, optional Use dictionary encoding. Can be set on a per-column basis with a list of column names. See also -------- pyarrow.parquet.write_table pyarrow.parquet.ParquetFile fastparquet rNz0Saving to parquet requires the `pyarrow` packageF)preserve_index) compressionuse_dictionaryversion)row_group_size)pyarrowpyarrow.parquet ImportError isinstancepd DataFrame enumerateTable from_pandasparquet ParquetWriterschema write_tableclose) piecesoutpathr rrr kwargspar ipiecetablewriters X/var/www/html/software/conda/lib/python3.11/site-packages/bioframe/sandbox/parquet_io.py to_parquetr"s:FN NNNLMMMN&",''!&)) E EHAuH((u(EEEAvv11L!,#1#      u^  D D D D E   s %A+CCFc |dd|stj|fd|S ddlm}n#t $rt dwxYwGfdd|}||fi|S) a Load DataFrames from Parquet files, optionally in pieces. Parameters ---------- filepath : str, pathlib.Path, pyarrow.NativeFile, or file-like object Readable source. For passing bytes or buffer-like file containing a Parquet file, use pyarorw.BufferReader columns: list If not None, only these columns will be read from the row groups. A column name may be a prefix of a nested field, e.g. 'a' will select 'a.b', 'a.c', and 'a.d.e' iterator : boolean, default False Return an iterator object that yields row group DataFrames and provides the ParquetFile interface. use_threads : boolean, default True Perform multi-threaded column reads memory_map : boolean, default True If the source is a file path, use a memory map to read file, which can improve performance in some environments Returns ------- DataFrame or ParquetFileIterator use_threadsTcolumnsr$r) ParquetFilez;Iterating over Parquet data requires the `pyarrow` package.c"eZdZdZfdZdS))read_parquet..ParquetFileIteratorc|S)N)selfs r!__iter__z2read_parquet..ParquetFileIterator.__iter__ks ct|dsd|_|j|jkr/||jd}|xjdz c_nt|S)N_rgidrT)r&r$use_pandas_metadata)hasattrr0num_row_groupsread_row_group StopIteration to_pandas)r,rgr&r$s r!__next__z2read_parquet..ParquetFileIterator.__next__ns{tW--#!"DJ: 333,, '$/,0 -B JJ!OJJJ''||~~%r.N)__name__ __module__ __qualname__r-r9r%sr!ParquetFileIteratorr)jsB    & & & & & & & &r.r=)popr read_parquetr r'r )filepathr&iteratorrr'r=r$s ` @r!r?r?As6**]D11K 7  %;  BH     3 3 3 3 3 3 3   M    & & & & & & & &+ & & &&#"866v666s 8A)NrTr)NF)pandasrr"r?r+r.r!rCsR   ::::z<7<7<7<7<7<7r.