o F'hf@sTdZddlmZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl m Z mZddlmZddlmZmZmZddlmZddlmZdd lmZmZmZmZeeZej Z!Gd d d e"Z#Gd d d eZ$GdddeZ%Gddde"Z&Gddde"Z'Gddde(Z)Gddde)Z*Gddde)Z+ddZ,dS)z.. include:: ../README.md) annotationsN) urlencodeurlparse) urlretrieve)datetime timedeltatimezone)timegm)Enum)Dict GeneratorListOptionalc @sreZdZUdZded< ded< ded< ded< ded < ded < d ed < d ed < d ed< ded< ded< ded< d ed< ded< eedgdddddggdf dEddZdFddZdGdd ZdGd!d"Z dHd$d%Z dGd&d'Z dIdJd*d+Z ,  -dKdLd1d2Z ,  -dKdLd3d4ZdMd5d6ZdNd9d:ZdOd=d>ZGd?d@d@eZGdAdBdBeZGdCdDdDeZdS)PResultz An entry in an arXiv query results feed. See [the arXiv API User's Manual: Details of Atom Results Returned](https://arxiv.org/help/api/user-manual#_details_of_atom_results_returned). strentry_idrupdated publishedtitle List[Author]authorssummary Optional[str]comment journal_refdoiprimary_category List[str] categories List[Link]linkspdf_urlfeedparser.FeedParserDict_rawNcCs^||_||_||_||_||_||_||_||_| |_| |_ | |_ | |_ t | |_| |_dS)z Constructs an arXiv search result item. In most cases, prefer using `Result._from_feed_entry` to parsing and constructing `Result`s yourself. N)rrrrrrrrrrrr r _get_pdf_urlr!r#)selfrrrrrrrrrrrr r#r'DC:\Users\shubh\Videos\agent\nova\Lib\site-packages\arxiv/__init__.py__init__Js  zResult.__init__entryreturncCst|ds tdd}t|dr|j}ntd|jt|jt|jt|j t dd|dd|j D|j |d |d |d |jd d d|jDdd|jD|d S)zk Converts a feedparser entry for an arXiv search result feed into a Result object. id0rz7Result %s is missing title attribute; defaulting to '0'z\s+ cSg|]}tj|qSr')rAuthor_from_feed_author).0ar'r'r( z+Result._from_feed_entry..Z arxiv_commentZarxiv_journal_refZ arxiv_doitermcSsg|]}|dqS)r6)get)r2tagr'r'r(r4scSr/r')rLink_from_feed_linkr2linkr'r'r(r4r5) rrrrrrrrrrrr r#)hasattrrMissingFieldErrorrloggerwarningr, _to_datetimeZupdated_parsedZpublished_parsedresubrrr7Zarxiv_primary_categorytagsr )r*rr'r'r(_from_feed_entryqs*       zResult._from_feed_entrycC|jSN)rr&r'r'r(__str__zResult.__str__cCsndt|t|jt|jt|jt|jt|jt|jt|j t|j t|j t|j t|j t|j S)Nz{}(entry_id={}, updated={}, published={}, title={}, authors={}, summary={}, comment={}, journal_ref={}, doi={}, primary_category={}, categories={}, links={}))format _classnamereprrrrrrrrrrrrr rHr'r'r(__repr__s"zResult.__repr__boolcCst|tr |j|jkSdSNF) isinstancerrr&otherr'r'r(__eq__s  z Result.__eq__cCs|jddS)a; Returns the short ID for this result. + If the result URL is `"https://arxiv.org/abs/2107.05580v1"`, `result.get_short_id()` returns `2107.05580v1`. + If the result URL is `"https://arxiv.org/abs/quant-ph/0201082v1"`, `result.get_short_id()` returns `"quant-ph/0201082v1"` (the pre-March 2007 arXiv identifier format). For an explanation of the difference between arXiv's legacy and current identifiers, see [Understanding the arXiv identifier](https://arxiv.org/help/arxiv_identifier). zarxiv.org/abs/)rsplitrHr'r'r( get_short_idszResult.get_short_idpdf extensioncCs6|jr|jnd}d|ddtdd||gS)zK A default `to_filename` function for the extension given. ZUNTITLED./_z[^\w])rjoinrWreplacerBrC)r&rYZnonempty_titler'r'r(_get_default_filenames zResult._get_default_filename./export.arxiv.orgdirpathfilenamedownload_domaincCs:|s|}tj||}t|j|}t||\}}|S)z Downloads the PDF for this result to the specified directory. The filename is generated by calling `to_filename(self)`. )r_ospathr]r_substitute_domainr!r)r&rbrcrdrfr! written_pathr\r'r'r( download_pdfs  zResult.download_pdfc CsH|s|d}tj||}t|j|}|dd}t||\}}|S)z Downloads the source tarfile for this result to the specified directory. The filename is generated by calling `to_filename(self)`. ztar.gzz/pdf/z/src/) r_rerfr]rrgr!r^r) r&rbrcrdrfr!Zsrc_urlrhr\r'r'r(download_sources  zResult.download_sourcecCsBdd|D}t|dkrdSt|dkrtd|d|dS)z Finds the PDF link among a result's links and returns its URL. Should only be called once for a given `Result`, in its constructor. After construction, the URL should be available in `Result.pdf_url`. cSsg|] }|jdkr|jqSrX)rhrefr;r'r'r(r4sz'Result._get_pdf_url..rNz'Result has multiple PDF links; using %s)lenr?r@)r Zpdf_urlsr'r'r(r%s   zResult._get_pdf_urltstime.struct_timecCstjt|tjdS)z Converts a UTC time.struct_time into a time-zone-aware datetime. This will be replaced with feedparser functionality [when it becomes available](https://github.com/kurtmckee/feedparser/issues/212). )tz)r fromtimestampr rutc)ror'r'r(rAszResult._to_datetimeurldomaincCst|}|j|dS)z Replaces the domain of the given URL with the specified domain. This is useful for testing purposes. )netloc)r_replacegeturl)rtru parsed_urlr'r'r(rg szResult._substitute_domainc@sNeZdZUdZded< dddZdd d Zdd d Zdd dZdddZ dS) Result.AuthorzJ A light inner class for representing a result's authors. rnamecCs ||_dS)z Constructs an `Author` with the specified name. In most cases, prefer using `Author._from_feed_author` to parsing and constructing `Author`s yourself. Nr{)r&r{r'r'r(r)s zResult.Author.__init__ feed_authorr"r+cCs t|jS)z Constructs an `Author` with the name specified in an author object from a feed entry. See usage in `Result._from_feed_entry`. )rr0r{)r}r'r'r(r1$s zResult.Author._from_feed_authorcCrFrGr|rHr'r'r(rI-rJzResult.Author.__str__cCdt|t|jSNz{}({}))rKrLrMr{rHr'r'r(rN0zResult.Author.__repr__rOcCt|tjr |j|jkSdSrP)rQrr0r{rRr'r'r(rT3  zResult.Author.__eq__N)r{r)r}r"r+rzr+rr+rO) __name__ __module__ __qualname____doc____annotations__r)r1rIrNrTr'r'r'r(r0s   r0c@steZdZUdZded< ded< ded< ded<   ddd d ZdddZdddZdddZdddZ dS) Result.LinkzH A light inner class for representing a result's links. rrlrrrel content_typeNcCs||_||_||_||_dS)z Constructs a `Link` with the specified link metadata. In most cases, prefer using `Link._from_feed_link` to parsing and constructing `Link`s yourself. Nrlrrr)r&rlrrrr'r'r(r)Fs  zResult.Link.__init__ feed_linkr"r+cCs&tj|j|d|d|ddS)z Constructs a `Link` with link metadata specified in a link object from a feed entry. See usage in `Result._from_feed_entry`. rrrr)rr9rlr7)rr'r'r(r:Xs zResult.Link._from_feed_linkcCrFrG)rlrHr'r'r(rIfrJzResult.Link.__str__cCs.dt|t|jt|jt|jt|jS)Nz){}({}, title={}, rel={}, content_type={}))rKrLrMrlrrrrHr'r'r(rNiszResult.Link.__repr__rOcCrrP)rQrr9rlrRr'r'r(rTrrzResult.Link.__eq__)NNN)rlrrrrrrr)rr"r+rrr) rrrrrr)r:rIrNrTr'r'r'r(r98s$     r9c@s8eZdZUdZded< ded< ddZd dd Zd S) zResult.MissingFieldErrorzg An error indicating an entry is unparseable because it lacks required fields. r missing_fieldmessagecCs||_d|_dS)Nz&Entry from arXiv missing required info)rr)r&rr'r'r(r)s z!Result.MissingFieldError.__init__r+cCr~r)rKrLrMrrHr'r'r(rNrz!Result.MissingFieldError.__repr__Nr)rrrrrr)rNr'r'r'r(r>ws r>)rrrrrrrrrrrrrrrrrrrrrrr rr#r")r*r"r+rrrrk)rYrr+r)r`r$ra)rbrrcrrdrr+r)r rr+r)rorpr+r)rtrrurr+r)rrrrr _DEFAULT_TIMEr)rErIrNrTrWr_rirjr%rArgobjectr0r9 Exceptionr>r'r'r'r(rs~  '         %?rc@seZdZdZdZdZdZdS) SortCriterionz A SortCriterion identifies a property by which search results can be sorted. See [the arXiv API User's Manual: sort order for return results](https://arxiv.org/help/api/user-manual#sort). Z relevanceZlastUpdatedDateZ submittedDateN)rrrr RelevanceZLastUpdatedDateZ SubmittedDater'r'r'r(rs rc@seZdZdZdZdZdS) SortOrderz A SortOrder indicates order in which search results are sorted according to the specified arxiv.SortCriterion. See [the arXiv API User's Manual: sort order for return results](https://arxiv.org/help/api/user-manual#sort). Z ascendingZ descendingN)rrrrZ Ascending Descendingr'r'r'r(rsrc@seZdZUdZded< ded< ded< ded < d ed < d gd ejejfdddZ dddZ dddZ d ddZ d!d"ddZ d S)#Searchz A specification for a search of arXiv's database. To run a search, use `Search.run` to use a default client or `Client.run` with a specific client. rqueryrid_list int | None max_resultsrsort_byr sort_orderr$NcCs0||_||_|tjkr dn||_||_||_dS)zM Constructs an arXiv API search with the specified criteria. N)rrmathinfrrr)r&rrrrrr'r'r(r)s   zSearch.__init__r+cCt|SrGrMrHr'r'r(rIzSearch.__str__c Cs6dt|t|jt|jt|jt|jt|jS)NzC{}(query={}, id_list={}, max_results={}, sort_by={}, sort_order={}))rKrLrMrrrrrrHr'r'r(rNszSearch.__repr__Dict[str, str]cCs |jd|j|jj|jjdS)zx Returns a dict of search parameters that should be included in an API request for this search. ,)Z search_queryrZsortByZ sortOrder)rr]rrvaluerrHr'r'r( _url_argss  zSearch._url_argsroffsetintGenerator[Result, None, None]cCs tjdtddtj||dS)z Executes the specified search using a default arXiv API client. For info on default behavior, see `Client.__init__` and `Client.results`. **Deprecated** after 2.0.0; use `Client.results`. zGThe 'Search.results' method is deprecated, use 'Client.results' instead) stacklevel)r)warningswarnDeprecationWarningClientresults)r&rr'r'r(rs zSearch.results) rrrrrrrrrrr)r+rr)rrr+r)rrrrrrrrrr)rIrNrrr'r'r'r(rs,      rc@seZdZUdZdZ ded< ded< ded< ded <d ed <d/d0ddZd1ddZd1ddZd2d3ddZ d2d3ddZ d4d!d"Z d5d6d)d*Z d7d,d-Z d.S)8rz Specifies a strategy for fetching results from arXiv's API. This class obscures pagination and retry logic, and exposes `Client.results`. z%https://export.arxiv.org/api/query?{}r page_sizefloat delay_seconds num_retriesr_last_request_dtzrequests.Session_sessiond@cCs&||_||_||_d|_t|_dS)ag Constructs an arXiv API client with the specified options. Note: the default parameters should provide a robust request strategy for most use cases. Extreme page sizes, delays, or retries risk violating the arXiv [API Terms of Use](https://arxiv.org/help/api/tou), brittle behavior, and inconsistent results. N)rrrrrequestsSessionr)r&rrrr'r'r(r)(s  zClient.__init__r+rcCrrGrrHr'r'r(rI7rzClient.__str__cC&dt|t|jt|jt|jS)Nz2{}(page_size={}, delay_seconds={}, num_retries={}))rKrLrMrrrrHr'r'r(rN;s zClient.__repr__rsearchrrrcCs<|jr|j|nd}|r|dkrtdSt||||S)ab Uses this client configuration to fetch one page of the search results at a time, yielding the parsed `Result`s, until `max_results` results have been yielded or there are no more search results. If all tries fail, raises an `UnexpectedEmptyPageError` or `HTTPError`. Setting a nonzero `offset` discards leading records in the result set. When `offset` is greater than or equal to `search.max_results`, the full result set is discarded. For more on using generators, see [Generators](https://wiki.python.org/moin/Generators). Nrr')riter itertoolsislice_results)r&rrlimitr'r'r(rCs zClient.resultsc cs||||j}|j|dd}|jstddSt|jj}tdt |j||jru|jD]#}zt |VWq0t j yS}z t d|WYd}~q0d}~ww|t |j7}||kradS||||j}|j|dd}|js-dSdS)NT) first_pagez)Got empty first page; stopping generationz&Got first page: %d of %d total resultszSkipping partial result: %sF) _format_urlr _parse_feedentriesr?inforfeedZopensearch_totalresultsrnrrEr>r@)r&rrpage_urlr total_resultsr*er'r'r(rWs4   zClient._resultsstartcCs(|}|||d|jt|S)z Construct a request API for search that returns up to `page_size` results starting with the result at index `start`. )rr)rupdatequery_url_formatrKr)r&rrrZurl_argsr'r'r(rpszClient._format_urlTrtrrO _try_indexr"c Csz |j|||dWStttjjfy?}z%||jkr2td|||j |||ddWYd}~Std|||d}~ww)z Fetches the specified URL and parses it with feedparser. If a request fails or is unexpectedly empty, retries the request up to `self.num_retries` times. )r try_indexzGot error (try %d): %srm)rrNzGiving up (try %d): %s) _Client__try_parse_feed HTTPErrorUnexpectedEmptyPageErrorr exceptionsConnectionErrorrr?debugr)r&rtrrerrr'r'r(r~s   zClient._parse_feedrc Cs|jdur't|jd}t|j}||kr'||}td|t |td||||j j |ddid}t|_|j t jjkrLt|||j t|j}t|jdkra|sat||||jrqtd d |vrn|jnd|S) z Recursive helper for _parse_feed. Enforces `self.delay_seconds`: if that number of seconds has not passed since `_parse_feed` was last called, sleeps until delay_seconds seconds have passed. N)secondszSleeping: %f secondsz(Requesting page (first: %r, try: %d): %sz user-agentzarxiv.py/2.2.0)headersrz Bozo feed; consider handling: %sbozo_exception)rrrrnow total_secondsr?rtimesleeprr7 status_codercodesOKr feedparserparsecontentrnrrZbozor@r) r&rtrrrequiredZsince_last_requestZto_sleepresprr'r'r(Z__try_parse_feeds*       zClient.__try_parse_feedN)rrr)rrrrrrrr)rrrrr+r)rrrrrrr+r)Tr)rtrrrOrrr+r")rtrrrOrrr+r")rrrrrrr)rIrNrrrrrr'r'r'r(rs*        rcsLeZdZUdZded< ded< ded< d fdd Zd d d ZZS) ArxivErrorz$This package's base Exception class.rrtrretryrcs$||_||_||_t|jdS)zZ Constructs an `ArxivError` encountered while fetching the specified URL. N)rtrrsuperr))r&rtrr __class__r'r(r)szArxivError.__init__r+cCsd|j|jS)Nz{} ({}))rKrrtrHr'r'r(rIszArxivError.__str__)rtrrrrrr)rrrrrr)rI __classcell__r'r'rr(rs  rcs8eZdZUdZded< d fdd Zdd d ZZS)ra  An error raised when a page of results that should be non-empty is empty. This should never happen in theory, but happens sporadically due to brittleness in the underlying arXiv API; usually resolved by retries. See `Client.results` for usage. r"raw_feedrtrrrcs ||_||_t||ddS)z} Constructs an `UnexpectedEmptyPageError` encountered for the specified API URL after `retry` tries. z&Page of results was unexpectedly emptyN)rtrrr))r&rtrrrr'r(r)sz!UnexpectedEmptyPageError.__init__r+cCrNz{}({}, {}, {}))rKrLrMrtrrrHr'r'r(rNz!UnexpectedEmptyPageError.__repr__)rtrrrrr"rrrrrrr)rNrr'r'rr(rs   rcs8eZdZUdZded< d fdd Zd d d ZZS)rzm A non-200 status encountered while fetching a page of results. See `Client.results` for usage. rstatusrtrrcs(||_||_t||d|jdS)z Constructs an `HTTPError` for the specified status code, encountered for the specified API URL after `retry` tries. z Page request resulted in HTTP {}N)rtrrr)rK)r&rtrrrr'r(r)s zHTTPError.__init__r+cCrr)rKrLrMrtrrrHr'r'r(rN rzHTTPError.__repr__)rtrrrrrrrr'r'rr(rs  rcCsd|jjS)zAA helper function for use in __repr__ methods: arxiv.Result.Link.zarxiv.{})rKrr)or'r'r(rLsrL)-r __future__rloggingrrrrerrBrr urllib.parserrurllib.requestrrrrcalendarr enumr typingr r r r getLoggerrr?minrrrrrrrrrrrrLr'r'r'r(s>     p a7