o E'hZ@sddlmZddlZddlZddlZddlmZmZddlmZddl m Z ddl m Z ddl mZm Z ddlmZdd lmZmZddlZdd lmZdd lmZdd lmZd dlmZmZmZd dlm Z m!Z!m"Z"m#Z#m$Z$e%dZ&GdddZ'dS)) annotationsN)datetimetimezone)cached_property)cycle)shuffle)sleeptime) TracebackType)AnyLiteral)_Element) HTMLParser)document_fromstring)DuckDuckGoSearchExceptionRatelimitExceptionTimeoutException)_expand_proxy_tb_alias _extract_vqd _normalize_normalize_url json_loadszduckduckgo_search.DDGSc@seZdZdZ     dSdTddZdUddZ   dVdWddZedXddZdYdZd#d$Z       d[d\d1d2Z d]d4d5Z 6 7  8 d^d_d?d@Z 6  d`dadAdBZ 6  d`dadCdDZ 6 7       dbdcdJdKZ 6 7     dddedOdPZ 6 7  dfdgdQdRZdS)hDDGSzBDuckDuckgo_search class to get search results from duckduckgo.com.N Theadersdict[str, str] | Noneproxy str | Noneproxiesdict[str, str] | str | Nonetimeout int | NoneverifyboolreturnNonec Cstjd}|r |nt||_|jdust|jtsJd|s;|r;tjdddt|t r8|dp7|dn||_|r?|ni|_ d |j d <||_ t j |j|j d d d d d |d|_d|_dS)aInitialize the DDGS object. Args: headers (dict, optional): Dictionary of headers for the HTTP client. Defaults to None. proxy (str, optional): proxy for the HTTP client, supports http/https/socks5 protocols. example: "http://user:pass@example.com:3128". Defaults to None. timeout (int, optional): Timeout value for the HTTP client. Defaults to 10. verify (bool): SSL verification when making the request. Defaults to True. Z DDGS_PROXYNzproxy must be a strz-'proxies' is deprecated, use 'proxy' instead.r stacklevelhttphttpshttps://duckduckgo.com/RefererTrandomF)rr!Z cookie_storeZrefererZ impersonateZimpersonate_osZfollow_redirectsr#)osenvirongetrr isinstancestrwarningswarndictrr!primpZClientclientsleep_timestamp)selfrrrr!r#Z ddgs_proxyr;YC:\Users\shubh\Videos\agent\nova\Lib\site-packages\duckduckgo_search/duckduckgo_search.py__init__"s,   z DDGS.__init__cCs|SNr;r:r;r;r< __enter__IszDDGS.__enter__exc_typetype[BaseException] | Noneexc_valBaseException | Noneexc_tbTracebackType | NonecCsdSr>r;)r:rArCrEr;r;r<__exit__Lsz DDGS.__exit__ LHTMLParsercCstdddddS)zGet HTML parser.TF)Zremove_blank_textZremove_commentsZ remove_pisZ collect_ids)rHr?r;r;r<parserTsz DDGS.parser? sleeptimefloatcCs6|jsdn t|jdkrdn|}t|_t|dS)zSleep between API requests.r.N)r9r r)r:rKdelayr;r;r<_sleepYs" z DDGS._sleepmethodCLiteral['GET', 'HEAD', 'OPTIONS', 'DELETE', 'POST', 'PUT', 'PATCH']urlr3paramscontent bytes | Nonedatacookiesjsonr float | Nonec Cs|z|jj||||||||| p|jd } Wn4tyL} z(dt| vr8t|dt| j d| | t |dt| j d| | d} ~ wwt d| j d| j| jdkra| S| jdvrrt| j d| jdt | j d |d |d |) N)rSrTrVrrWrXr!r  z: z _get_url() )i-iiiiz Ratelimitz return None. params=z content=z data=)rOr8requestr! Exceptionr3lowerrtype__name__rloggerdebugrR status_coder) r:rPrRrSrTrVrrWrXr!respexr;r;r<_get_url_s2        z DDGS._get_urlkeywordscCs |jddd|idj}t||S)z!Get vqd value for a search query.GETzhttps://duckduckgo.comqrS)rgrTr)r:rh resp_contentr;r;r<_get_vqds z DDGS._get_vqdwt-wtmoderateautoregion safesearch timelimitbackend max_resultslist[dict[str, str]]c Cs|dvrtjd|dddd}|dkrddgn|g}t|gd }} |D]?} z| dkr7|||||}n | dkrC|||||}|WStyf} ztd | d | | } WYd } ~ q'd } ~ wwt| ) ajDuckDuckGo text search. Query params: https://duckduckgo.com/params. Args: keywords: keywords for query. region: wt-wt, us-en, uk-en, ru-ru, etc. Defaults to "wt-wt". safesearch: on, moderate, off. Defaults to "moderate". timelimit: d, w, m, y. Defaults to None. backend: auto, html, lite. Defaults to auto. auto - try all backends in random order, html - collect data from https://html.duckduckgo.com, lite - collect data from https://lite.duckduckgo.com. max_results: max number of results. If None, returns results only from the first response. Defaults to None. Returns: List of dictionaries with search results, or None if there was an error. Raises: DuckDuckGoSearchException: Base exception for duckduckgo_search errors. RatelimitException: Inherits from DuckDuckGoSearchException, raised for exceeding API request rate limits. TimeoutException: Inherits from DuckDuckGoSearchException, raised for API request timeouts. )apiZecosiazbackend=z$ is deprecated, using backend='auto'r'rphtmlZliteNzError to search using z backend: ) r4r5r _text_html _text_liter^rbinfor) r:rhrqrrrsrtrubackendsresultserrbrfr;r;r<texts&  z DDGS.textcCs|sJd|d|d}|r||d<t}g}tdD]}|jdd|dj} d | vr.|St| |j} | d } t| tsB|S| D]t} t| t r| d } | r]t| tr]t | d nd}|r||vr| d s| || d}|rt|trt |d nd}| d}|rt|trd dd|Dnd}|t|t|t|d|rt||kr|SqD| d}|r|s|St|tr|dnd}t|t r|d}|d}t|trt|trddt||D}q|S)Nkeywords is mandatory)rjrkldfPOSTz https://html.duckduckgo.com/htmlrVs No results.z //div[h2]z ./a/@hrefrzhttp://www.google.com/search?q=z%https://duckduckgo.com/y.js?ad_domainz ./h2/a/text()z ./a//text()cs|]}t|VqdSr>r3.0xr;r;r< z"DDGS._text_html..titlehrefbodyz.//div[@class="nav-link"].//input[@type="hidden"]/@name.//input[@type="hidden"]/@valuecSi|] \}}t|t|qSr;rrnvr;r;r< z#DDGS._text_html..)setrangergrTrrIxpathr2listr r3 startswithaddjoinappendrrlenzip)r:rhrqrsrupayloadcacher~_rltreeelementse hrefxpathr titlexpathr bodyxpathrnpx next_pagenamesvaluesr;r;r<rzsj          &     zDDGS._text_htmlcsB|sJd||d}|r||d<t}g}tdD]}|jdd|dj} d| vr.|St| |j} | d } t| tsB|St t td d| D]\} } t| t r| d kr| d }|rmt|trmt |d nd}|dus|||vs|| d rfddtdDqN||| d}|rt|trt |d nd}qN| dkr| d}|rt|trddd|Dnd}|r|t|t|t|d|rt||kr|SqN| d}|r|s|St|tr|dnd}t|t r|d}|d}t|trt|trddt ||D}q|S)Nr)rjrrrrz!https://lite.duckduckgo.com/lite/rsNo more results.z//table[last()]//trrz .//a//@hrefrrcsg|]}tdqSr>)next)rrrr;r< (sz#DDGS._text_lite..z .//a//text()rrxz&.//td[@class='result-snippet']//text()csrr>rrr;r;r<r0rz"DDGS._text_lite..rz(//form[./input[contains(@value, 'ext')]]rrrcSrr;rrr;r;r<rGrz#DDGS._text_lite..)rrrgrTrrIrr2rrrr r3rrrstriprrrr)r:rhrqrsrurrr~rrlrrirrrrrrrrrrrr;rr<r{sz                zDDGS._text_litesizecolor type_imagelayout license_imagec Cs|sJd||} dddd} |rd|nd}|r!d|nd}|r*d|nd}|r3d |nd}|rs&