3dddlZddlZddlmZddlmZdejzdzZdejzdzZdZ dZ d Z d Z d Z Gd d ZdS)N)Failure) referer_strzScraped from %(src)sz%(item)szDropped: %(exception)szZCrawled (%(status)s) %(request)s%(request_flags)s (referer: %(referer)s)%(response_flags)szError processing %(item)sz:Spider error processing %(request)s (referer: %(referer)s)zError downloading %(request)sz)Error downloading %(request)s: %(errmsg)scNeZdZdZdZdZdZdZdZd dZ e d Z dS) LogFormattera%Class for generating log messages for different actions. All methods must return a dictionary listing the parameters ``level``, ``msg`` and ``args`` which are going to be used for constructing the log message when calling ``logging.log``. Dictionary keys for the method outputs: * ``level`` is the log level for that action, you can use those from the `python logging library `_ : ``logging.DEBUG``, ``logging.INFO``, ``logging.WARNING``, ``logging.ERROR`` and ``logging.CRITICAL``. * ``msg`` should be a string that can contain different formatting placeholders. This string, formatted with the provided ``args``, is going to be the long message for that action. * ``args`` should be a tuple or dict with the formatting placeholders for ``msg``. The final log message is computed as ``msg % args``. Users can define their own ``LogFormatter`` class if they want to customize how each action is logged or if they want to omit it entirely. In order to omit logging an action the method must return ``None``. Here is an example on how to create a custom log formatter to lower the severity level of the log message when an item is dropped from the pipeline:: class PoliteLogFormatter(logformatter.LogFormatter): def dropped(self, item, exception, response, spider): return { 'level': logging.INFO, # lowering the level from logging.WARNING 'msg': "Dropped: %(exception)s" + os.linesep + "%(item)s", 'args': { 'exception': exception, 'item': item, } } c |jrdt|jnd}|jrdt|jnd}tjt|j||t |||ddS)z0Logs a message when the crawler finds a webpage. )statusrequest request_flagsrefererresponse_flagsflagslevelmsgargs)rstrloggingDEBUG CRAWLEDMSGr r)selfr responsespiderr rs 3lib/python3.11/site-packages/scrapy/logformatter.pycrawledzLogFormatter.crawled7s4;MI0C ..000r 6>nL2S00222"]"/"!.&w//"0'   ct|tr|}n|}tjt ||ddS)z3Logs a message when an item is scraped by a spider.)srcitemr) isinstancergetErrorMessagerr SCRAPEDMSG)rr rrrs rscrapedzLogFormatter.scrapedIsS h ( ( **,,CCC]   rc2tjt||ddS)zULogs a message when an item is dropped while it is passing through the item pipeline.) exceptionr r)rWARNING DROPPEDMSGrr r&rrs rdroppedzLogFormatter.droppedXs+_&   rc0tjtd|idS)zLogs a message when an item causes an error while it is passing through the item pipeline. .. versionadded:: 2.0 r r)rERROR ITEMERRORMSGr)s r item_errorzLogFormatter.item_errorcs%]   rcLtjt|t|ddS)zLLogs an error message from a spider. .. versionadded:: 2.0 )r r r)rr,SPIDERERRORMSGr)rfailurer rrs r spider_errorzLogFormatter.spider_errorqs3 ]!"&w//   rNcVd|i}|r t}||d<nt}tj||dS)zLogs a download error message from a spider (typically coming from the engine). .. versionadded:: 2.0 r errmsgr)DOWNLOADERRORMSG_LONGDOWNLOADERRORMSG_SHORTrr,)rr1r rr4rrs rdownload_errorzLogFormatter.download_errorsE 7#  )'C#DNN(C]   rc|SN)clscrawlers r from_crawlerzLogFormatter.from_crawlers suu rr9) __name__ __module__ __qualname____doc__rr$r*r.r2r7 classmethodr=r:rrrrs##J   $                    $[rr)rostwisted.python.failurerscrapy.utils.requestrlinesepr#r(rr-r0r6r5rr:rrrGs ******,,,,,, #bj 0: = % 2Z ? i * M8CBBBBBBBBBBr