3ddZddlmZddlmZddlmZddlmZddl m Z m Z m Z m Z ddlmZGdd ZGd d ZGd d ZGddeZdS)zB Extension for processing data before they are exported to feeds. )BZ2File)GzipFile)IOBase)LZMAFile)AnyBinaryIODictList load_objectcNeZdZdZdedeeefddfdZde de fdZ d d Z dS) GzipPlugina  Compresses received data using `gzip `_. Accepted ``feed_options`` parameters: - `gzip_compresslevel` - `gzip_mtime` - `gzip_filename` See :py:class:`gzip.GzipFile` for more info about parameters. file feed_optionsreturnNc||_||_|jdd}|jd}|jd}t|jd||||_dS)Ngzip_compresslevel gzip_mtime gzip_filenamewb)fileobjmode compresslevelmtimefilename)rrgetrgzipfile)selfrrcompress_levelrrs @lib/python3.11/site-packages/scrapy/extensions/postprocessing.py__init__zGzipPlugin.__init__s (*../CQGG!%%l33$((99 I(     datac6|j|SN)rwriterr$s r!r'zGzipPlugin.write(}""4(((r#cj|j|jdSr&)rcloserrs r!r+zGzipPlugin.close+.  r#rN __name__ __module__ __qualname____doc__rr strrr"bytesintr'r+r#r!rr s    X  T#s(^       )%)C))))r#rcNeZdZdZdedeeefddfdZde de fdZ d d Z dS) Bz2Pluginz Compresses received data using `bz2 `_. Accepted ``feed_options`` parameters: - `bz2_compresslevel` See :py:class:`bz2.BZ2File` for more info about parameters. rrrNc||_||_|jdd}t|jd||_dS)Nbz2_compresslevelrr)rrr)rrrrbz2file)rrrr s r!r"zBz2Plugin.__init__;sM (*../BAFFYT    r#r$c6|j|Sr&)r<r'r(s r!r'zBz2Plugin.writeCs|!!$'''r#cj|j|jdSr&)r<r+rr,s r!r+zBz2Plugin.closeFs.  r#r.r/r7r#r!r9r90s X T#s(^     (%(C((((r#r9cNeZdZdZdedeeefddfdZde de fdZ d d Z dS) LZMAPluginu Compresses received data using `lzma `_. Accepted ``feed_options`` parameters: - `lzma_format` - `lzma_check` - `lzma_preset` - `lzma_filters` .. note:: ``lzma_filters`` cannot be used in pypy version 7.3.1 and older. See :py:class:`lzma.LZMAFile` for more info about parameters. rrrNc2||_||_|jd}|jdd}|jd}|jd}t|jd|||||_dS)N lzma_format lzma_check lzma_preset lzma_filtersr)rrformatcheckpresetfilters)rrrrlzmafile)rrrrGrHrIrJs r!r"zLZMAPlugin.__init__\s ("&&}55!%%lB77"&&}55#''77 Y     r#r$c6|j|Sr&)rKr'r(s r!r'zLZMAPlugin.writemr)r#cj|j|jdSr&)rKr+rr,s r!r+zLZMAPlugin.closepr-r#r.r/r7r#r!r@r@Ks  X T#s(^     ")%)C))))r#r@ceZdZdZdeededeeefddfdZ de de fd Z de fd Z dd Zdefd Zdeedeefd ZdefdZdS)PostProcessingManagera This will manage and use declared plugins to process data in a pipeline-ish way. :param plugins: all the declared plugins for the feed :type plugins: list :param file: final target file where the processed data will be written :type file: file like object pluginsrrrNc|||_||_||_||_dSr&) _load_pluginsrPrr_get_head_plugin head_plugin)rrPrrs r!r"zPostProcessingManager.__init__sB))'22  (0022r#r$c6|j|S)a Uses all the declared plugins to process data first, then writes the processed data to target file. :param data: data passed to be written to target file :type data: bytes :return: returns number of bytes written :rtype: int )rTr'r(s r!r'zPostProcessingManager.writes%%d+++r#c4|jSr&)rtellr,s r!rWzPostProcessingManager.tellsy~~r#c8|jdS)zC Close the target file along with all the plugins. N)rTr+r,s r!r+zPostProcessingManager.closes      r#cdS)NTr7r,s r!writablezPostProcessingManager.writablestr#cd|D}|S)Nc,g|]}t|Sr7r ).0plugins r! z7PostProcessingManager._load_plugins..s ===6;v&&===r#r7)rrPs r!rRz#PostProcessingManager._load_pluginss==W===r#c\|j}|jdddD]}|||j}|S)NrD)rrPr)rprevr^s r!rSz&PostProcessingManager._get_head_plugins@yl44R4( 3 3F6$ 122DD r#r.)r0r1r2r3r rrr r4r"r5r6r'rWr+boolrZrRrSr7r#r!rOrOxs3Cy3(03@DS#X3 3333 ,% ,C , , , , c    !!!! $T#Y49#r#rON)r3bz2rgzipriorlzmartypingrrr r scrapy.utils.miscr rr9r@rOr7r#r!ris=,,,,,,,,,,,,))))))        F6''''''''Z11111F11111r#