&th]|dZddlZddlZddlmZddlZddlZddlZddlm Z ddl m Z m Z m Z mZmZmZddlmZddlmZddlmZdd lmZdd lmZmZdd lmZdd lmZerdd l m Z ej!e"Z#GddeZ$Gdde%Z&ddddddddddddddd Z'Gd!d"e%Z(d#e)d$e*fd%Z+Gd&d'e%Z,id(d)d*d+d,d-d.d/d0d1d2d3d4d5d6d7d8d9d:d;dd?d@dAdBdCdDdEdFdGdHdIidJdKdLdMdNdOdPdQdRdSdTdUdVdWdXdYdZd[d\d]d^d_d`dadbdcdddedfdgdhdidjdkidldmdndodpdqdrdsdtdudvdwdxdydzd{d|d}d~ddddddddddddddddddddddddd Z-dS)zNetwork Manager module.N) OrderedDict)SimpleNamespace) AwaitableDictListOptionalUnion TYPE_CHECKING)unquote) EventEmitter) CDPSession) NetworkError) FrameManagerFrame) debugError)Multimap)SetceZdZdZeddddZdeded d ffd Zd e e e fd d fd Z de e e fd d fdZ d e e e ffdZ ded d fdZde d d fdZded d fdZd2dZde d d fdZde ded d fdZded d fdZde dee d d fdZde d d fdZ d3dd d!ed"e d#ed$ed%e d d fd&Zd'e dee d(e d)ed*e d+e d,ee d-ed d d fd.Zded d fd/Zded d fd0Zded d fd1Z xZ!S)4NetworkManagerzNetworkManager class.requestresponse requestfailedrequestfinished)RequestResponse RequestFailedRequestFinishedclient frameManagerreturnNct|_|_t _t _t_d_ d_ t_ d_ d_t_t_jdfdjdjjdjjdjjdjjd jdS) zMake new NetworkManager.FNzNetwork.requestWillBeSentchjj|SN)_client_loop create_task_onRequestWillBeSent)eventselfs m/var/www/html/mycamper/aliexpress-site/backend/venv/lib/python3.11/site-packages/pyppeteer/network_manager.pyz)NetworkManager.__init__..:s-$,,88))%00zNetwork.requestInterceptedzNetwork.requestServedFromCachezNetwork.responseReceivedzNetwork.loadingFinishedzNetwork.loadingFailed)super__init__r$ _frameManagerdict_requestIdToRequest_requestIdToResponseWillBeSentr_extraHTTPHeaders_offline _credentialsset_attemptedAuthentications_userRequestInterceptionEnabled#_protocolRequestInterceptionEnabledr_requestHashToRequestIds_requestHashToInterceptionIdson_onRequestIntercepted_onRequestServedFromCache_onResponseReceived_onLoadingFinished_onLoadingFailed)r)rr __class__s` r*r.zNetworkManager.__init__(sI  )AE IM+8C # 6:=@UU&/4,380(0 %-5ZZ*  '       4d6PQQQ 8$:XYYY 2D4LMMM 143JKKK /1FGGGGGr, credentialscLK||_|d{VdS)z"Provide credentials for http auth.N)r5"_updateProtocolRequestInterception)r)rCs r* authenticatezNetworkManager.authenticateDs8'5577777777777r,extraHTTPHeadersc \Kt|_|D]Y\}}t|ts#t d|dt |d||j|<Z|j dd|jid{VdS)zSet extra http headers.zExpected value of header "z" to be string, but z is found.zNetwork.setExtraHTTPHeadersheadersN) rr3items isinstancestr TypeErrortypelowerr$send)r)rGkvs r*setExtraHTTPHeadersz"NetworkManager.setExtraHTTPHeadersIs"-$**,, 2 2DAqa%% 0///77///00012D "17799 - -l =!*D,B CEE E E E E E E E E Er,c$tdi|jS)zGet extra http headers.)r0r3r)s r*rGzNetworkManager.extraHTTPHeadersVs--d,---r,valuecK|j|krdS||_|jd|jddddd{VdS)z#Change offline mode enable/disable.Nz Network.emulateNetworkConditionsr)offlinelatencydownloadThroughputuploadThroughput)r4r$rPr)rWs r*setOfflineModezNetworkManager.setOfflineModeZs~ =E ! ! F l B}"$ " E E            r, userAgentcPK|jdd|id{VdS)zSet user agent.zNetwork.setUserAgentOverrider`N)r$rP)r)r`s r* setUserAgentzNetworkManager.setUserAgentfsSl >!,i 8:: : : : : : : : : :r,cLK||_|d{VdS)zEnable request interception.N)r8rEr^s r*setRequestInterceptionz%NetworkManager.setRequestInterceptionks8/4,5577777777777r,cK|jpt|j}||jkrdS||_|rddigng}t j|jdd|i|jdd|id{VdS)N urlPattern*zNetwork.setCacheDisabled cacheDisabledzNetwork.setRequestInterceptionpatterns)r8boolr5r9asynciogatherr$rP)r)enabledris r*rEz1NetworkManager._updateProtocolRequestInterceptionps7+)**  d> > > F3:0,3;\3'((n L  * '*   L  0X&     r,r(cK|jrt|di}|j|}|r2||||j||nK|j||d||j |d<dS||ddS)Nr requestId) r9generateRequestHashgetr; firstValue _onRequestdeleter:r6r2)r)r( requestHashinterceptionIds r*r'z#NetworkManager._onRequestWillBeSents  3 -eii 2.F.FGGK!?JJ;WWN T~666299+~VVVV-11+uyy?U?UVVVNS3EIIk4J4JK F t$$$$$r,methodmsgcK |j||d{VdS#t$r }tt|Yd}~dSd}~wwxYwr#)r$rP Exceptionrlogger)r)rwrxes r*_sendzNetworkManager._sends| ",##FC00 0 0 0 0 0 0 0 0 0 " " " vq ! ! ! ! ! ! ! ! ! "s!' AA  Ac |drd}|d|jvrd}n)|jr"d}|j|dt |did}t |did}|jj|d |d|||d d dS|j sB|j r;|jj|d d|dit|d }|j |}|rn|j|}|||d|j |||j|ddS|j||ddS) N authChallengeDefaultrv CancelAuthProvideCredentialsr5usernamepassword"Network.continueInterceptedRequest)rrr)rvauthChallengeResponser)rqr7r5addgetattrr$r%r&r}r8r9rpr:rrr2rsrtpopr;r6)r)r(rrrrurorequestWillBeSentEvents r*r=z$NetworkManager._onRequestIntercepteds 99_ % %  H%&$*HHH'" L/.2259I3JKKKt^R88<N4O4O P P P  ) 0 0i H H H  / 3 3It D D D D D  . 2 2;FV@W X X X X Xr,rvc Lt}|dr|j|d}|r|d}|||d|d|d|d|d|j}t |d|dko|d d k}||d||d id ||d d |d i|d|dS)NredirectResponserostatusrI fromDiskCachefromServiceWorkerSecurityDetailsloaderIdrNDocumentrurlframeId)listrqr1_handleRequestRedirect_redirectChainrj_handleRequestStart)r)r(rv redirectChainrrisNavigationRequests r*rszNetworkManager._onRequests'+vv 99' ( ( 7.2253EFFG 7#();#< ++$((22$((33$((99$(()<==$(():;; !( 6 " IIk " "eii &;&; ; , IIf   +     +   IIi $ $ ( ( / /  IIfb ! ! IIi $ $ IIi  r,cv|j|d}|r d|_dSdS)NroT)r1rq_fromMemoryCache)r)r(rs r*r>z(NetworkManager._onRequestServedFromCachesD*..uyy/E/EFF  ,'+G $ $ $ , ,r,rredirectStatusredirectHeadersrrsecurityDetailsc t|j||||||}||_|j||t d|j|j d|j |j | tjj|| tjj|dS)Nz2Response body is unavailable for redirect response)rr$ _responserappend_bodyLoadedPromiseFulfillrr1r _requestIdr7discard_interceptionIdemitrEventsr)r)rrrrrrrs r*rz%NetworkManager._handleRequestRedirectsDL'>+]>> &..w/FGGG .'0(;;; .'7AAAAAr,rorr resourceTyperequestPayloadrrc d} |r!|j|j|} t|j||||j|||| | } | |j|<|tjj| dSr#) r/framerr$r8r1rrr) r)rorvrrrrrrrrs r*rz"NetworkManager._handleRequestStarts   6t)5&,,W55E$, >->&}NN/6 + .'/99999r,c |j|d}|sdS|di}t|j||dd|di|d|d|d}||_|t jj|dS) NrorrrrIrrr)r1rqrr$rrrr)r)r(r_resprs r*r?z"NetworkManager._onResponseReceiveds*..u[/ABB  F *b))DL'!IIh22!IIi44!IIo66!II&9::!II&788 :: % .'0(;;;;;r,cR|j|d}|sdS|j}|r|d|j|jd|j|j| tj j |dS)Nro) r1rqrrrrr7rrrrrrr)r(rrs r*r@z!NetworkManager._onLoadingFinisheds*..u[/ABB  F#  5  . .t 4 4 4  $$W%7>>> &..w/FGGG .'7AAAAAr,c|j|d}|sdS|d|_|j}|r|d|j|jd|j|j | tj j |dS)Nro errorText)r1rq _failureTextrrrrr7rrrrrrrs r*rAzNetworkManager._onLoadingFailed s*..u[/ABB  F$yy55#  5  . .t 4 4 4  $$W%7>>> &..w/FGGG .'5w?????r,)r Nr#)"__name__ __module__ __qualname____doc__rrr rr.rrLrFrSrGrjr_rbrdrEr'r0r}r=rrsr>intrrrr?r@rA __classcell__)rBs@r*rrs _%) FHzHH$HHHHHH88d38n88888 E$sCx. E'+ E E E E.$sCx.....  $  4     :C:D:::: 8$848888     $ % % % % % %"#"D"T"""" 'Y4'YD'Y'Y'Y'YR  hsm     >,t,,,,,8<BBiBB04BEIB26B15BAEBBBB":S:,4SM:@C:15:EH:-1:i|]\}}||SrUrO.0rQrRs r* z$Request.__init__..W&BBB$!QABBBr,)r$r_isNavigationRequestr_allowInterception_interceptionHandledrr_urlrO _resourceTyperq_method _postDatarJ_headers_framerr) r)rrorvrrrrrrrrIs r*r.zRequest.__init__Cs  #$7!-"3$)!-1+/ )//11{{8,,  Z00++i,,BB'--//BBB  + %r,c|jS)zURL of this request.rrVs r*rz Request.url] yr,c|jS)a;Resource type of this request perceived by the rendering engine. ResourceType will be one of the following: ``document``, ``stylesheet``, ``image``, ``media``, ``font``, ``script``, ``texttrack``, ``xhr``, ``fetch``, ``eventsource``, ``websocket``, ``manifest``, ``other``. )rrVs r*rzRequest.resourceTypebs !!r,c|jS)z/Return this request's method (GET, POST, etc.).)rrVs r*rwzRequest.methodm |r,c|jS)z!Return post body of this request.)rrVs r*rzRequest.postDatar ~r,c|jS)zgReturn a dictionary of HTTP headers of this request. All header names are lower-case. rrVs r*rIzRequest.headersw }r,rc|jS)zReturn matching :class:`Response` object, or ``None``. If the response has not been received, return ``None``. )rrVs r*rzRequest.responses ~r,c|jS)zReturn a matching :class:`~pyppeteer.frame_manager.frame` object. Return ``None`` if navigating to error page. )rrVs r*rz Request.frames {r,c|jS)z3Whether this request is driving frame's navigation.)rrVs r*rzRequest.isNavigationRequests ((r,c4tj|jS)aReturn chain of requests initiated to fetch a resource. * If there are no redirects and request was successful, the chain will be empty. * If a server responds with at least a single redirect, then the chain will contain all the requests that were redirected. ``redirectChain`` is shared between all the requests of the same chain. )copyrrVs r*rzRequest.redirectChainsy,---r,c&|jsdSd|jiS)a:Return error text. Return ``None`` unless this request was failed, as reported by ``requestfailed`` event. When request failed, this method return dictionary which has a ``errorText`` field, which contains human-readable error message, e.g. ``'net::ERR_RAILED'``. Nr)rrVs r*failurezRequest.failures"  4T.//r, overridescRK|i}|jstd|jrtdd|_d|ji}|| |jd|d{VdS#t$r }tt|Yd}~dSd}~wwxYw)aYContinue request with optional request overrides. To use this method, request interception should be enabled by :meth:`pyppeteer.page.Page.setRequestInterception`. If request interception is not enabled, raise ``NetworkError``. ``overrides`` can have the following fields: * ``url`` (str): If set, the request url will be changed. * ``method`` (str): If set, change the request method (e.g. ``GET``). * ``postData`` (str): If set, change the post data or request. * ``headers`` (dict): If set, change the request HTTP header. N$Request interception is not enabled.Request is already handled.Trvr) rrrrupdater$rPrzrr{)r)roptr|s r* continue_zRequest.continue_s  I& GEFF F  $ ><== =$(!!56 9 ",##$H#NN N N N N N N N N N " " " vq ! ! ! ! ! ! ! ! ! "s!A<< B&B!!B&rcNK|jdrdS|jstd|jrtdd|_|dr7t |dtr|dd}n|d}i}|dr.|dD]%}|d||| <&|d r |d |d <|rd |vrt||d <|d d }t|d}d|d|}d}||z} |D]}| |d|||} | |z} | d} |r| |z} tj | d} |jd|j| dd{VdS#t$$r } t't(| Yd} ~ dSd} ~ wwxYw)aVFulfills request with given response. To use this, request interception should by enabled by :meth:`pyppeteer.page.Page.setRequestInterception`. Request interception is not enabled, raise ``NetworkError``. ``response`` is a dictionary which can have the following fields: * ``status`` (int): Response status code, defaults to 200. * ``headers`` (dict): Optional response headers. * ``contentType`` (str): If set, equals to setting ``Content-Type`` response header. * ``body`` (str|bytes): Optional response body. data:NrrTbodyutf-8rI contentTypez content-typezcontent-lengthrrz HTTP/1.1  z z: asciir)rv rawResponse)r startswithrrrrqrKrLencoderOlen statusTextsbase64 b64encodedecoder$rPrrzrr{) r)r responseBodyresponseHeadersheader statusCode statusText statusLineCRLFtextresponseBufferrr|s r*respondzRequest.responds 9   ( (  F& GEFF F  $ ><== =$(! <<   0Jx/?$E$E 0,4V,<,C,CG,L,LLL#<<//L << " " N"9- N N2:92Ef2M // << & & F.6}.EON +  B,OCC03L0A0AO, -\\(C00  __Z44 :::j:: D % F FFEFEEof&=EtEEDDd{W--  ;+l:N&~66==gFF  ",##$H"&"6*KK          " " " vq ! ! ! ! ! ! ! ! ! "s)G:: H$HH$failed errorCodecKt|}|s"td||jstd|jrtdd|_ |jdt|j|d{VdS#t$r }tt|Yd}~dSd}~wwxYw)arAbort request. To use this, request interception should be enabled by :meth:`pyppeteer.page.Page.setRequestInterception`. If request interception is not enabled, raise ``NetworkError``. ``errorCode`` is an optional error code string. Defaults to ``failed``, could be one of the following: - ``aborted``: An operation was aborted (due to user action). - ``accessdenied``: Permission to access a resource, other than the network, was denied. - ``addressunreachable``: The IP address is unreachable. This usually means that there is no route to the specified host or network. - ``blockedbyclient``: The client chose to block the request. - ``blockedbyresponse``: The request failed because the request was delivered along with requirements which are not met ('X-Frame-Options' and 'Content-Security-Policy' ancestor check, for instance). - ``connectionaborted``: A connection timeout as a result of not receiving an ACK for data sent. - ``connectionclosed``: A connection was closed (corresponding to a TCP FIN). - ``connectionfailed``: A connection attempt failed. - ``connectionrefused``: A connection attempt was refused. - ``connectionreset``: A connection was reset (corresponding to a TCP RST). - ``internetdisconnected``: The Internet connection has been lost. - ``namenotresolved``: The host name could not be resolved. - ``timedout``: An operation timed out. - ``failed``: A generic failure occurred. zUnknown error code: {}rrTr)rv errorReasonN) errorReasonsrformatrrr$rPr0rrzrr{)r)rrr|s r*abortz Request.aborts B#9-  K7>>yIIJJ J& GEFF F  $ ><== =$(! ",##$H$#3'KKK          " " " vq ! ! ! ! ! ! ! ! ! "s(5B C )CC r#)r)rrrrr rrLrjr0rrr.propertyrrrwrrrIrrrrrrrrrUr,r*rr/s&&z&hsm&!)#&EI&$(&/2&BE&&(0&!%Y &  &&&&4SX"c"""X" X(3-XX(:.XxX)T)))) .tI . . .X . 0$ 0 0 0 0"""""""<9"d9"t9"9"9"9"v/"/"S/"/"/"/"/"/"/"r,rAborted AccessDeniedAddressUnreachableBlockedByClientBlockedByResponseConnectionAbortedConnectionClosedConnectionFailedConnectionRefusedConnectionResetInternetDisconnectedNameNotResolvedTimedOutFailed)aborted accessdeniedaddressunreachableblockedbyclientblockedbyresponseconnectionabortedconnectionclosedconnectionfailedconnectionrefusedconnectionresetinternetdisconnectednamenotresolvedtimedoutrceZdZdZ ddedededeeefde de d ed dfd Z d e e d dfd Z ed efdZed e fdZed efdZed efdZed eedffdZd efdZd eefdZd efdZd efdZed efdZed e fdZed e fdZdS)rzCResponse class represents responses which are received by ``Page``.NrrrrIrrrr c||_||_||_|jj|_|jj|_|j|_||_ ||_ d| D|_ i|_ |r8t|d|d|d|d|d|_ dSdS)Nc>i|]\}}||SrUrrs r*rz%Response.__init__..[rr, subjectNameissuer validFromvalidToprotocol)r$_request_statusr% create_future_contentPromise_bodyLoadedPromiserr_fromDiskCache_fromServiceWorkerrJr_securityDetailsr)r)rrrrIrrrs r*r.zResponse.__init__Ns   #|1??AA"&,"4"B"B"D"DK +"3BB'--//BBB >@  $3 .) , * + %%D ! ! !  r,rWc:|j|dSr#)r1 set_resultr^s r*rz"Response._bodyLoadedPromiseFulfillfs **511111r,c|jS)zURL of the response.rrVs r*rz Response.urlirr,cB|jdkpd|jcxkodkncS)z@Return bool whether this request is successful (200-299) or not.rri+r.rVs r*okz Response.okns4|q >C4<$>$>$>$>3$>$>$>$>>r,c|jS)zStatus code of the response.r9rVs r*rzResponse.statussrr,c|jS)zfReturn dictionary of HTTP headers of this response. All header names are lower-case. rrVs r*rIzResponse.headersxrr,rc|jS)zReturn security details associated with this response. Security details if the response was received over the secure connection, or `None` otherwise. )r4rVs r*rzResponse.securityDetailss $$r,c*K|jd{V}t|tr||jdd|jjid{V}|dd}|drtj |S|S)NzNetwork.getResponseBodyrorr, base64Encoded) r1rKrzr$rPr-rrqr b64decode)r)resultrrs r*_bufreadzResponse._bufreads....... fi ( ( L**+D 1G         ||FC(( << ( ( *#D)) ) r,c|js1|jj|S|jS)z>'** *r,cbK|d{V}tj|S)z)Get JSON representation of response body.N)rjsonloadsrGs r*rJz Response.jsons5 ######z'"""r,c|jS)z%Get matching :class:`Request` object.)r-rVs r*rzResponse.request }r,c(|jp |jjS)zReturn ``True`` if the response was served from cache. Here `cache` is either the browser's disk cache or memory cache. )r2r-rrVs r* fromCachezResponse.fromCaches "Ddm&DDr,c|jS)z?Return ``True`` if the response was served by a service worker.)r3rVs r*rzResponse.fromServiceWorkers &&r,r#)rrrrr rrrrLrjr.rrzrr rr:rrIr rbytesrBrrErr0rJrrOrrUr,r*rrKsaMMCGzGSsCx.9=$(;?02x /B2t2222SX?D???X?XX%t->'>!?%%%X%     $ %($$$$ +C++++#D#### XE4EEEXE'4'''X'''r,rrr c|dd} t|}n#t$rYnwxYw||d|did}|dsrt |d}||D]4}|d|}|}|dvr)||d|<5tj |S) zGenerate request hash.rrrwr)rrwrrIrrI)acceptrefererz/x-devtools-emulate-network-conditions-client-idcookie) rqr rzrrkeyssortrOrJdumps)r normalizedURL_hashrIr headerValues r*rprps"KKr**M  ..      ++h''KK ++   E  # #G , , 3wy)..0011  3 3F!),V4K\\^^F '2E) V $ $ :e  s ( 55c eZdZdZdedededededdf d Zedefd Zedefd Z edefd Z edefd Z edefdZ dS)rz6Class represents responses which are received by page.r(r)r*r+r,r NcL||_||_||_||_||_dSr#) _subjectName_issuer _validFrom_validTo _protocol)r)r(r)r*r+r,s r*r.zSecurityDetails.__init__s*' # !r,c|jS)z:Return the subject to which the certificate was issued to.)r^rVs r*r(zSecurityDetails.subjectNames   r,c|jS)z;Return a string with the name of issuer of the certificate.)r_rVs r*r)zSecurityDetails.issuerrr,c|jS)zrsW ###### !!!!!!HHHHHHHHHHHHHHHH ++++++))))))77777777''''''''''''  8 $ $N@N@N@N@N@\N@N@N@bG"G"G"G"G"fG"G"G"V".(,,**,(2( $m'm'm'm'm'vm'm'm'`#@"""""f"""J= := = <= 4 =  9 =  : = *= <= = >= = 9= = = 7=  ;!=" >#==$ ;%=& >'=( )=* +=, =-=. >/=0 1=2 ;3=4 ;5=6 7=8 9=: *;=< ==> :?=@ 6A=B C=D E===F G=H >I=J #K=L "M=N O=P ?Q=R S=T !U=V 8W=X Y=Z [=\ "]=^ _=` ,a=b *c=d "e=f g==h   ' $ !   ,y=== r,