Format: application/x-python-code
1 o
2
3 /'h�} �
4 @ s� d dl Z d dlZd dlZd dlZd dlZd dlZddlmZ ddl m
5 Z
6 m Z m Z m
7 Z
8 mZmZmZ ddlmZmZ ddlmZ ddlmZmZmZ G dd � d e
9 je je je
10 jejejej� ZdS )
11 � N� )�_cp1252)�_base�cc�dc�georss�itunes�mediarss�psc)�_sanitize_html�_HTMLSanitizer)�FeedParserDict)�_urljoin�make_safe_absolute_uri�resolve_relative_urisc s� e Zd Zi dd�dd�dd�dd�dd�dd�dd�dd�d d�d
12 d�d d�d d�d
13 d�dd�dd�dd�dd�i dd�dd�dd�dd �d d �d d �d d!�d"d#�d$d%�d&d'�d(d)�d*d+�d,d-�d.d/�d0d1�d2d3�d4d5��i d6d7�d8d9�d:d;�d<d=�d>d=�d?d@�dAdB�dCdB�dDdE�dFdG�dHdI�dJdK�dLdM�dNdO�dPdQ�dRdS�dTdU��dVdWdXdYdZd[d\d]d^d_d`dadbdcddde��Zi Zh df�Zh dg�Zh dg�ZdhdihZ� fdjdk�Z dldm� Z
14 dndo� Z dpdq� Z drds� Z
15 dtdu� Zd�dwdx�Zdydz� Zd{d|� Zd}d~� Zdd�� Zed�d�� �Zd�d�� Zd�d�� Zed�d�� �Zed�d�� �Zd�d�� Zd�d�d��Zd�d�� Z d�d�� Z ed�d�� �Z d�d�� Z d�d�� Z d�d�� Z!ed�d�� �Z"d�d�d��Z#d�d�� Z$d�d�d��Z%d�d�� Z&d�d�d��Z'd�d�� Z(d�d�� Z)d�d�� Z*� Z+S )��_FeedParserMixin� � http://backend.userland.com/rssz%http://blogs.law.harvard.edu/tech/rss�http://purl.org/rss/1.0/�&http://my.netscape.com/rdf/simple/0.9/z http://example.com/newformat#zhttp://example.com/nechozhttp://purl.org/echo/zuri/of/echo/namespace#zhttp://purl.org/pie/zhttp://purl.org/atom/ns#zhttp://www.w3.org/2005/Atomz'http://purl.org/rss/1.0/modules/rss091#zhttp://webns.net/mvcb/Zadminz,http://purl.org/rss/1.0/modules/aggregation/�agz)http://purl.org/rss/1.0/modules/annotate/Zannotatez!http://media.tangent.org/rss/1.0/Zaudioz-http://backend.userland.com/blogChannelModuleZ blogChannelz%http://creativecommons.org/ns#licenser zhttp://web.resource.org/cc/z>http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.htmlZcreativeCommonsz4http://backend.userland.com/creativeCommonsRssModulez'http://purl.org/rss/1.0/modules/company�coz(http://purl.org/rss/1.0/modules/content/�contentz&http://my.theinfo.org/changed/1.0/rss/�cpz http://purl.org/dc/elements/1.1/r zhttp://purl.org/dc/terms/Zdctermsz&http://purl.org/rss/1.0/modules/email/�emailz&http://purl.org/rss/1.0/modules/event/Zevz*http://rssnamespace.org/feedburner/ext/1.0Z
16 feedburnerz http://freshmeat.net/rss/fm/�fmzhttp://xmlns.com/foaf/0.1/Zfoafz(http://www.w3.org/2003/01/geo/wgs84_pos#Zgeoz http://www.georss.org/georssr zhttp://www.opengis.net/gmlZgmlzhttp://postneo.com/icbm/Zicbmz&http://purl.org/rss/1.0/modules/image/�imagez*http://www.itunes.com/DTDs/PodCast-1.0.dtdr z'http://example.com/DTDs/PodCast-1.0.dtdz%http://purl.org/rss/1.0/modules/link/�lz http://search.yahoo.com/mrssZmediaz http://search.yahoo.com/mrss/z4http://madskills.com/public/xml/rss/module/pingback/Zpingbackz.http://prismstandard.org/namespaces/1.2/basic/Zprismz+http://www.w3.org/1999/02/22-rdf-syntax-ns#�rdfz%http://www.w3.org/2000/01/rdf-schema#Zrdfsz*http://purl.org/rss/1.0/modules/reference/�refz*http://purl.org/rss/1.0/modules/richequiv/Zreqvz'http://purl.org/rss/1.0/modules/search/�searchz&http://purl.org/rss/1.0/modules/slash/�slashz)http://schemas.xmlsoap.org/soap/envelope/�soap�ss�str�subZsyZszfZtaxoZthr�tiZ trackbackZwfwZwiki�xhtmlZxlink�xmlr
17 )z.http://purl.org/rss/1.0/modules/servicestatus/z-http://hacks.benhammersley.com/rss/streaming/z-http://purl.org/rss/1.0/modules/subscription/z,http://purl.org/rss/1.0/modules/syndication/z/http://schemas.pocketsoap.com/rss/myDescModule/z)http://purl.org/rss/1.0/modules/taxonomy/z*http://purl.org/rss/1.0/modules/threading/z*http://purl.org/rss/1.0/modules/textinput/z5http://madskills.com/public/xml/rss/module/trackback/z$http://wellformedweb.org/commentAPI/z%http://purl.org/rss/1.0/modules/wiki/z http://www.w3.org/1999/xhtmlz http://www.w3.org/1999/xlinkz$http://www.w3.org/XML/1998/namespacez"http://podlove.org/simple-chapters>
18 �icon�linkZlogo�docs�id�hrefZwfw_commentrss�urlZ wfw_comment�comments> �summaryZrightsZtagline�infor �subtitle� description�title� copyright�application/xhtml+xml� text/htmlc s� | j s| j�� D ] \}}|| j |�� < qt� | _g | _d| _i | _d| _ d| _
19 d| _ d| _ d| _
20 d| _d| _d| _d| _t� | _t� | _d | _i | _g | _g | _g | _d| _d| _d| _d| _ | j rl| j � dd�| jd< i | _ t t!| ��"� d S )Nr r ������_�-�language)#�_matchnamespaces�
21 namespaces�items�lowerr
22 �feeddata�entries�version�namespaces_in_use�infeed�inentry� incontent� intextinput�inimageZinauthorZ
23 incontributorZ inpublisher�insource�
24 sourcedata�
25 contentparamsZ _summaryKey� namespacemap� elementstack� basestack� langstack�svgOK� title_depth�depthZ
26 hasContent�lang�replace�property_depth_map�superr �__init__)�self�k�v�� __class__� �FC:\Users\shubh\Videos\agent\nova\Lib\site-packages\feedparser/mixin.pyrW � s> z_FeedParserMixin.__init__c C s t ��N)�NotImplementedError)rX �kvr] r] r^ �_normalize_attributes� � z&_FeedParserMixin._normalize_attributesc s� � j d7 _ � fdd�|D �}t|�}|�d|�d��p � j}t|t�r,|�� jd�}� jr:t� j|�p7� j� _nt � j|�� _|�d|�d��}|d krQd }n|d u rX� j
27 }|rg|d
28 v rg|� d d �� j d
29 < |� _
30 � j
31 �� j� � j�|� |D ] \}}|�d�r�� �|dd � |� qy|dkr�� �d |� qy� jr�� j�dd��d�s�|dv r�d S d� jd< � j�r
32 � j�d�dk�r
33 |�d�dkr�|�dd�\}}� j�|d �}|dkr�|dkr�|�d|f� |dkr�|dkr�|�d|f� |dkr�� jd7 _� jd|� �|�f d d �S |�d�dk�r|�dd�\}} nd |}} � j�||�}|�r.|d }|�s9|d v�r9d � _ |�sD|d v�rDd � _ d | | }
34 z
35 t � |
36 �} | |�W S t �yz || } t |�d k�ro� �!| d� Y S � �"� }
37 ||
38 | < Y d S w )!Nr c s g | ]}� � |��qS r] )rb )�.0�attr�rX r] r^ �
39 <listcomp>� s z5_FeedParserMixin.unknown_starttag.<locals>.<listcomp>zxml:base�base�ignorezxml:langrS r )�feedZrsszrdf:RDFr9 r: r; zxmlns:� Zxmlns�typer( �z xhtml:div�divr6 �:r8 �mathz"http://www.w3.org/1998/Math/MathML�svgzhttp://www.w3.org/2000/svgz<%s%s>r ��escape)r4 r* r3 �name)r4 r* r3 r. r- �width�heightZ_start_)#rR �dict�get�baseuri�
40 isinstance�bytes�decode�encodingr r rS rT r@ rN �appendrO �
41 startswith�track_namespacerF rK �endswith�find�splitrC rP � handle_data�strattrsrL rG rH �getattr�AttributeError�len�push� _get_context)rX �tag�attrs�attrs_dry rS �prefix�uri� namespace�suffix�
42 methodname�methodZ unknown_tag�contextr] rf r^ �unknown_starttag� s|
43
44 �
45
46
47
48 �z!_FeedParserMixin.unknown_starttagc C st |� d�dkr|�dd�\}}nd|}}| j�||�}|r"|d }|dkr0| jr0| jd8 _d| | }z| jr=t� �t| |�}|� W n tyV | �|| � Y nw | jro| j �dd ��
49 d �so|d
50 v rjd S d | j d< | jr�| j �d�d kr�|�d�d }| j d | d
51 d� | j r�| j �� | j r�| j d r�| j d | _
52 | jr�| j�� | jr�| jd | _| jd8 _d S )Nro r8 r r r9 rq Z_end_rl r( rm r6 z</%s>r rr )r� r� rL rx rP r� r� �poprF rK r� r� rN ry rO rS rR )rX r� r� r� r� r� r] r] r^ �unknown_endtag/ sB
53
54
55 �
56
57
58 z _FeedParserMixin.unknown_endtagc C sr | j sd S |�� }|dv rd| }n|d dkr"t|dd � d�}nt|�}t|��d�}| j d d �|� d S )
59 N)
60 Z34�38�39Z60Z62Zx22Zx26Zx27Zx3cZx3ez&#%s;r �xr � �utf-8r8 � )rM r? �int�chr�encoder~ )rX r �text�cr] r] r^ �handle_charref[ s
61 z _FeedParserMixin.handle_charrefc C s� | j sd S |dv rd| }n;|| jv r(| j| }|�d�r'|�d�r'| �|�S n!ztjj| W n
62 ty= d| }Y n w ttjj| �� d�}| j d d �
63 |� d S )N)�lt�gt�quot�ampZaposz&%s;z&#�;r� r8 r� ) rM �entitiesr r� �handle_entityref�html�name2codepoint�KeyErrorr� r� r~ )rX r r� r] r] r^ r� j s
64
65
66
67 � �z!_FeedParserMixin.handle_entityrefr c C sD | j sd S |r| j�d�dkrtjj�|�}| j d d �|� d S )Nrl r6 r8 r� )rM rK rx r( �sax�saxutilsrs r~ )rX r� rs r] r] r^ r� } s
68 z _FeedParserMixin.handle_datac C � d S r_ r] �rX r� r] r] r^ �handle_comment� � z _FeedParserMixin.handle_commentc C r� r_ r] r� r] r] r^ � handle_pi� r� z_FeedParserMixin.handle_pic C r� r_ r] r� r] r] r^ � handle_decl� rc z _FeedParserMixin.handle_declc C s� | j ||d � dkr3| j �d|�}|dkr t| j �}|S | �tjj�| j |d |� �d� |d S | j �d|�}|dkrB|d S |S ) N� z <![CDATA[z]]>r8 r � �>r )�rawdatar� r� r� r( r� r� rs )rX �irY r] r] r^ �parse_declaration� s
69 $z"_FeedParserMixin.parse_declarationc C s@ | � � } | dks | dkrd} | S | dkrd} | S | dkr d} | S )Nr� �plain�
70 text/plainr� r7 r' r6 )r? )� content_typer] r] r^ �map_content_type� s ��z!_FeedParserMixin.map_content_typec C s� |� � }| js ||fdkrd| _n|dkrd| _n|dkr d| _|�d�dkr+d }|}|| jv rB| j| | j|< || j| j| < d S || j|pHd
71 < d S ) N)Nr Zrss090r Zrss10zhttp://www.w3.org/2005/atom�atom10zbackend.userland.com/rssr8 r r )r? rB r� r< rL rC )rX r� r� Zlowerurir] r] r^ r� � s
72 z _FeedParserMixin.track_namespacec C s t | jpd|�S )Nr )r ry )rX r� r] r] r^ � resolve_uri� � z _FeedParserMixin.resolve_uric C s |S r_ r] )�element�datar] r] r^ �decode_entities� r� z _FeedParserMixin.decode_entitiesc C s d� dd� | D ��S )Nr c s s4 � | ]}d |d t jj�|d ddi�f V qdS )z %s="%s"r r �"z"N)r( r� r� rs �rd �tr] r] r^ � <genexpr>� s
73 � "�
74 �z,_FeedParserMixin.strattrs.<locals>.<genexpr>)�join)r� r] r] r^ r� � s
75 �z_FeedParserMixin.strattrsc C s | j �||g g� d S r_ )rM r~ )rX r� �expecting_textr] r] r^ r� � s z_FeedParserMixin.pushc
76 C s� | j sd S | j d d |krd S | j �� \}}}t|�D ]\}}t|t�r,|�d�||< q | jdkr�| j�dd�dkr�|rZt |�dkrZ|d �
77 � sZ|d= |rZt |�dkrZ|d �
78 � rI|ryt |�dkry|d �
79 � sy|d= |ryt |�dkry|d �
80 � rh|r�|d d ks�|d � d
81 �r�|d d kr�d}|d d� D ] }|� d �r�|d8 }|dkr� nq�|� d
82 �r�|� d�s�|d7 }q�|dd� }d�
83 |�} |r�| �
84 � } |s�| S tr�| j�dd�r�z
85 t�| �d���d�} W n tjtjtfy� Y nw || jv �r| �r|dk�r| j�r| �| �} | j�dd��s| �|| �} | j� d��s1| j�d�dk�r1| �| ��r1d| jd< z| jd= W n
86 t�yA Y nw z| jd= W n
87 t�yR Y nw | �| j�dd��| j v }
88 |
89 �r{| j �r{|| j v �r{t | | j | j | j�dd��} |
90 �r�| j!�r�|| j"v �r�t#| | j | j�dd��} | j �r�t| t��r�| �| j d�} | j dv �r�t| t��s�z
91 | �d��d�} W n t$tf�y� Y nw t| t��s�| �%t&�} |dv �r�| S |dk�r�d| j' k �r�| j(k�r�| S | j)�r�| j*�s�|d k�r| j+d �,|g � t-�.| j�} | | d < | j+d | �/| � | S |d k�rH| j0�sF| �1d d �} t2�3d!d"| �} | | j+d |< | �rF| | j+d d# d d$< | S |d%k�rOd&}| j4�,| j+d i ��|�} | d u �sg| j(| k�ry| j(| j4| j+d |< | | j+d |< | j5�r�t-�.| j�} | | d < | | j+d |d' < | S | j6�s�| j*�r�| �7� }
92 |d%k�r�d(}| |
93 |< |d k�r�t2�3d!d"| �} | |
94 |< | |
95 d# d d$< | S | j5�r�t-�.| j�} | | d < | |
96 |d' < | S ))Nr8 r r� r� rl r� r6 r z<div>z<div z</div>z</�<z/>r �base64�utf8r, Zatomr� r7 �moderi )r� zutf-8_INVALID_PYTHON_3z
97 iso-8859-1)�category�tagsZitunes_keywordsr4 r �valuer* z&�&z&([A-Za-z0-9_]+);z&\g<1>�linksr- r3 r0 �_detailr2 )8rM r� � enumeraterz r{ r| rB rK rx r� �stripr r� r� r� � decodebytesr� �binascii�Error�
98 Incomplete�UnicodeDecodeError�can_be_relative_uriZ
99 guidislinkr� r� �looks_like_htmlr� r� �
100 html_typesr �can_contain_relative_urisry r} Z
101 sanitize_html� can_contain_dangerous_markupr �UnicodeEncodeError� translater rQ rR rE rI rA �
102 setdefault�copy�deepcopyr~ rH rT �rer% rU rF rD r� )rX r� Zstrip_whitespacer� �piecesr� rZ rR Zpiece�outputZ
103 is_htmlishrK Zold_value_depthr� r] r] r^ r� � s�
104 � � �*
105 ��
106 �
107
108 � � �
109
110 $�
111 $
112 �
113 � �
114
115 � z_FeedParserMixin.popc C sl | j d7 _ | jr| j�dd�| _t| �|�d|��| j| jd��| _| �|| j�| jd< | � ||� d S )Nr r9 r: rl )rl r; rh r� )
116 rF rS rT r
117 r� rx ry rK �
118 _is_base64r� )rX r� r� Zdefault_content_typer� r] r] r^ � push_contenti s
119 �z _FeedParserMixin.push_contentc C s& | � |�}| jd8 _| j�� |S )Nr )r� rF rK �clear)rX r� r� r] r] r^ � pop_contentt s
120
121 z _FeedParserMixin.pop_contentc C s\ t �d| �st �d| �sdS tdd� t �d| �D ��r dS tdd� t �d| �D ��r,dS d S )
122 z3
123 :type s: str
124 :rtype: bool
125 z</(\w+)>z&#?\w+;Fc s s" � | ] }|� � tjvr|V qd S r_ )r? r Zacceptable_elementsr� r] r] r^ r� � s � z3_FeedParserMixin.looks_like_html.<locals>.<genexpr>z</?(\w+)c s s � | ] }|t jjvr|V qd S r_ )r� r� �
126 entitydefs)rd �er] r] r^ r� � s � z&(\w+);T)r� r �any�findall)�sr] r] r^ r� ~ s z _FeedParserMixin.looks_like_htmlc C sL |� d�}|dkr$|d |� }||d d � }| j�||�}|d | }|S )Nro r8 r )r� rL rx )rX rt Zcolonposr� r� r] r] r^ �_map_to_standard_prefix� s
127 z(_FeedParserMixin._map_to_standard_prefixc C s |� | �|��S r_ )rx r� )rX r� rt r] r] r^ �_get_attribute� r� z _FeedParserMixin._get_attributec C sT |� dd�dkr
128 dS | jd �d�rdS | jd �d�r dS | jd �d �r(dS dS )
129 Nr� r r� r rl ztext/r z+xmlz/xml)rx rK r r� )rX r� rK r] r] r^ r� � s z_FeedParserMixin._is_base64c
130 C sh | � d| � d| � dd ���}|r2z| d= W n ty Y nw z| d= W n ty- Y nw || d< | S )Nr. r� r- )rx r� )r� r- r] r] r^ �
131 _enforce_href� s
132 �
133 �z _FeedParserMixin._enforce_hrefFc C s( | � � }|r |||< d S |�||� d S r_ )r� r� )rX �keyr� � overwriter� r] r] r^ �_save� s z_FeedParserMixin._savec C s` | j r| j}|S | jrd| jv r| jd }|S | jr!| jd }|S | jr+| jd }|S | j}|S )Nr Z textinputr8 )rI rJ rH r@ rG rE rA )rX r� r] r] r^ r� � s �
134 �
135 �
136 �z _FeedParserMixin._get_context�authorc C sV | � � }|�|d t� � |||d |< | �� |�dt� g� ||d d |< d S )Nr� Zauthorsr8 )r� r� r
137 �_sync_author_detail)rX r� r� r� r� r] r] r^ � _save_author� s z _FeedParserMixin._save_authorc C s, | � � }|�dt� g� ||d d |< d S )NZ contributorsr8 )r� r� r
138 )rX r� r� r� r] r] r^ �_save_contributor� s z"_FeedParserMixin._save_contributorc C sZ | � � }|�d| t� g�d }|r<|�d�}|�d�}|r*|r*d||f ||< d S |r2|||< d S |r:|||< d S d S |�|�d }}|sHd S t�d|�}|r�|�d�}|�|d�}|�d d�}|�d
139 d�}|�d d�}|�� }|r|d d kr|d
140 d � }|r�|d dkr�|d d� }|�� }|s�|r�|�d| |� |r�||d< |r�||d< d S d S )Nz%ssr8 rt r z%s (%s)z�(([a-zA-Z0-9\_\-\.\+]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([a-zA-Z0-9\-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?))(\?subject=\S+)?r r z()z<>z<>�(r �)z %s_detail) r� rx r
141 r� r �grouprT r� r� )rX r� r� �detailrt r r� Z
142 emailmatchr] r] r^ r� � sF
143
144 �
145 �z$_FeedParserMixin._sync_author_detailc C sL | � � }|�dg �}|s|s|sd S t|||d�}||vr$|�|� d S d S )Nr� )�term�scheme�label)r� r� r
146 r~ )rX r r r r� r� r� r] r] r^ �_add_tag s �z_FeedParserMixin._add_tagc C s | � dd� d S )Nr� r )r� )rX r� r] r] r^ � _start_tags s z _FeedParserMixin._start_tagsc C s, | � d��d�D ] }| �|�� d d � qd S )Nr� �,)r� r� r r� )rX r r] r] r^ � _end_tags s �z_FeedParserMixin._end_tags)r )F)r� ),�__name__�
147 __module__� __qualname__r= r< r� r� r� r� rW rb r� r� r� r� r� r� r� r� r� � staticmethodr� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r r r �
148 __classcell__r] r] r[ r^ r * sL �������� �
149 � � �
150 �������������� � � � � �!�"�#�$�%�&�'�(�)�*�,�-�.�/�0�1�2�3�4�5�6�F
151
152 � -Z,
153
154
155
156
157
158
159
160
161
162
163
164
165 & r ) r� r� r� �
166 html.entitiesr� r� Zxml.sax.saxutilsr( r r= r r r r r r r
167 Z sanitizerr r �utilr
168 �urlsr r r � Namespacer r] r] r] r^ �<module> s( $
169 �