Format: application/x-python-code
1 o
2
3 2'h�/ � @ s� d dl Z d dlZd dlZd dlZd dlZd dlZd dlZd dlZd dlZ d dl
4 m Z d dl m
5 Z
6 d dlmZ d dlmZ d dlmZ d dlmZ dd ed
7 e jfd d �ZG d
8 d� de�ZdS )� N)�contextmanager)�Path)�sleep)�time)� BaseCache)�FileSystemSerializer� �string�returnc C s
9 t �| �S )z�Don't access ``hashlib.md5`` until runtime. FIPS builds may not include
10 md5, in which case the import and use as a default would fail before the
11 developer can configure something else.
12 )�hashlib�md5)r � r
13 �CC:\Users\shubh\Videos\agent\nova\Lib\site-packages\cachelib/file.py� _lazy_md5 s
14 r c @ s e Zd ZdZdZdZee�Ze � Z
15 dAde de d e d
16 e
17 je d e
18 jf
19 d d
20 �Zde fdd�Zede fdd��Z dBde
21 je de
22 je ddfdd�Zde
23 je de fdd�Zde defdd �Zde
24 je ddf fd d �Zdefd d �Zd!eddfd"d#�Z defd$d%�Z dCd&d'�Z defd(d)�Z d*e de fd+d,�Z d*e de
25 jfd-d.�Z!dDd*e de
26 jde
27 je defd/d0�Z" 1dEd*e de
28 jde
29 je d2edef
30 d3d4�Z#dFd*e d2edefd5d6�Z$d*e defd7d8�Z%d9e
31 j&d:e
32 jd;e
33 jde
34 jfd<d=�Z'e(d>e d
35 e de
36 jfd?d@��Z)dS )G�FileSystemCachea1 A cache that stores the items on the file system. This cache depends
37 on being the only user of the `cache_dir`. Make absolutely sure that
38 nobody but this cache stores files there or otherwise the cache will
39 randomly delete files therein.
40
41 :param cache_dir: the directory where cache files are stored.
42 :param threshold: the maximum number of items the cache stores before
43 it starts deleting some. A threshold value of 0
44 indicates no threshold.
45 :param default_timeout: the default timeout that is used if no timeout is
46 specified on :meth:`~BaseCache.set`. A timeout of
47 0 indicates that the cache never expires.
48 :param mode: the file mode wanted for the cache files, default 0600
49 :param hash_method: Default hashlib.md5. The hash method used to
50 generate the filename for cached results.
51 Default is lazy loaded and can be overriden by
52 seeting `_default_hash_method`
53 z .__wz_cacheZ__wz_cache_count�� �, N� cache_dir� threshold�default_timeout�mode� hash_methodc
54 C s� t �| |� || _|| _| j| _|d ur|| _|| _| jd u r$| �� | _zt� | j� W n t
55 yE } z
56 |j t j kr;� W Y d }~nd }~ww | jdkrY| j
57 tt| �� ��d� d S d S )Nr ��value)r �__init__�_path�
58 _threshold�_default_hash_method� _hash_method�_mode� _get_compatible_platform_mode�os�makedirs�OSError�errno�EEXIST�
59 _update_count�len�list� _list_dir)�selfr r r r r �exr
60 r
61 r r 8 s(
62
63 ���
64 �zFileSystemCache.__init__r
65 c C s d}t �� dkr tj}|S )Ni� �Windows)�platform�system�stat�S_IWRITE)r* r r
66 r
67 r r Z s z-FileSystemCache._get_compatible_platform_modec C s | � | j�pdS �Nr )�get�_fs_count_file�r* r
68 r
69 r � _file_count` s zFileSystemCache._file_count�deltar c C s< | j dkrd S |r| j| }n|pd}| j| j|dd� d S )Nr T)� mgmt_element)r r5 �setr3 )r* r6 r Z new_countr
70 r
71 r r& d s
72 z FileSystemCache._update_count�timeoutc C s* t �| |�}|dkrtt� �| }t|�S r1 )r �_normalize_timeout�intr )r* r9 r
73 r
74 r r: p s z"FileSystemCache._normalize_timeout�namec C s, | � | j��tj�d }||kp|�| j�S )N�����)�
75 _get_filenamer3 �splitr! �sep�endswith�_fs_transaction_suffix)r* r< Zfshashr
76 r
77 r �_is_mgmtv s zFileSystemCache._is_mgmtc s � fdd�t �� j�D �S )z2return a list of (fully qualified) cache filenamesc 3 s* � | ]}� � |�stj�� j|�V qd S �N)rC r! �path�joinr )�.0�fnr4 r
78 r � <genexpr>| s � �
79 �z,FileSystemCache._list_dir.<locals>.<genexpr>)r! �listdirr r4 r
80 r4 r r) z s
81
82 �zFileSystemCache._list_dirc C s | j dko
83 | j| j kS r1 )r r5 r4 r
84 r
85 r �_over_threshold� s z FileSystemCache._over_threshold�nowc C s� | � � D ]U}z6| �|d��}t�d|�d��d }W d � n1 s#w Y |dkr;||k r;t�|� | jdd� W q tyE Y q t t
86 tj fyY t j
87 d|dd � Y qw d S )
88 N�rb�I� r r= �r6 �/Exception raised while handling cache file '%s'T��exc_info)r) �_safe_stream_open�struct�unpack�readr! �remover& �FileNotFoundErrorr# �EOFError�error�logging�warning)r* rL �fname�f�expiresr
89 r
90 r �_remove_expired� s& �
91 �
92 ���z FileSystemCache._remove_expiredc C s( g }| � � D ]I}z*| �|d��}t�d|�d��d }|�||f� W d � n1 s,w Y W q ty; Y q tttj fyO t
93 j d|dd� Y qw dd � t |d
94 d � d �D �}|D ]2}z
95 t
96 �|� | jd
97 d� W n tyw Y n ty� t
98 j d|dd� Y dS w | �� s� dS q_dS )NrM rN rO r rQ TrR c s s � | ]\}}|V qd S rD r
99 )rG �_r^ r
100 r
101 r rI � s �
102 �z0FileSystemCache._remove_older.<locals>.<genexpr>c S s | d S r1 r
103 )�itemr
104 r
105 r �<lambda>� s z/FileSystemCache._remove_older.<locals>.<lambda>)�keyr= rP F)r) rT rU rV rW �appendrY r# rZ r[ r\ r] �sortedr! rX r& rK )r* Zexp_fname_tuplesr^ r_ � timestampZ fname_sortedr
106 r
107 r �
108 _remove_older� sN ��
109 ���
110 ���z FileSystemCache._remove_olderc C s0 | � � r t� }| �|� | � � r| �� d S d S rD )rK r ra ri )r* rL r
111 r
112 r �_prune� s
113 �zFileSystemCache._prunec C sx t | �� �D ]-\}}zt�|� W q ty Y q ty3 tjd|dd� | j| d� Y dS w | jdd� dS )NrQ TrR rP Fr r ) � enumerater) r! rX rY r# r\ r] r& )r* �ir^ r
114 r
115 r �clear� s �� zFileSystemCache.clearre c C sF t |t�r|�d�}| �|��� }n tdt|�� ���tj� | j
116 |�S )Nzutf-8z$Key must be a string, received type ) �
117 isinstance�str�encoder � hexdigest� TypeError�typer! rE rF r )r* re ZbkeyZ bkey_hashr
118 r
119 r r> � s
120
121
122 z FileSystemCache._get_filenamec
123 C s� | � |�}z=| �|d��,}t�d|�d��d }|dks!|t� kr/| j�|�W d � W S W d � W d S 1 s;w Y W d S tyL Y d S t t
124 tj fya t j
125 d|dd� Y d S w )NrM rN rO r rQ TrR )r> rT rU rV rW r �
126 serializer�loadrY r# rZ r[ r\ r] �r* re �filenamer_ Z pickle_timer
127 r
128 r r2 � s.
129
130 � � � ����zFileSystemCache.getc C s( | � |�}tj�|�s| �|||�S dS �NF)r> r! rE �existsr8 )r* re r r9 rw r
131 r
132 r �add� s
133 zFileSystemCache.addFr7 c C s
134 |rd}n| � � | �|�}| �|�}tj�|�}zKtj| j| j d�\}}t�
135 |d��} | � t �
136 d|�� | j�|| � W d � n1 sFw Y | �tj||� | �tj|| j� t|��� j}
137 W n tyv tjd|dd� Y dS w |s�|s�| jd d
138 � |
139 dkS ) Nr )�suffix�dir�wbrN rQ TrR F� rP )rj r: r> r! rE �isfile�tempfile�mkstemprB r �fdopen�writerU �packrt �dump� _run_safely�replace�chmodr r r/ �st_sizer# r\ r] r& ) r* re r r9 r7 rw � overwrite�fd�tmpr_ �fsizer
140 r
141 r r8 � s8
142
143
144 � � �� zFileSystemCache.setc C s^ z
145 t �| �|�� W n ty Y dS ty$ tjddd� Y dS w |s-| jdd� dS )NTz*Exception raised while handling cache filerR Fr= rP )r! rX r> rY r# r\ r] r& )r* re r7 r
146 r
147 r �delete s � zFileSystemCache.deletec
148 C s� | � |�}z:| �|d��)}t�d|�d��d }|dks!|t� kr+ W d � W dS W d � W dS 1 s8w Y W d S tyI Y dS tttj fy^ t
149 j d|dd� Y dS w ) NrM rN rO r TFrQ rR ) r> rT rU rV rW r rY r# rZ r[ r\ r] rv r
150 r
151 r �has* s&
152 �(� ��zFileSystemCache.hasrH �args�kwargsc O s� t �� dkr9d}d}d}d}||k r7z ||i |��}W |S ty2 t|� ||7 }|d9 }Y nw ||k s|S ||i |��}|S )ztOn Windows os.replace, os.chmod and open can yield
153 permission errors if executed by two different processes.r, Ng����MbP?g $@g � )r- r. �PermissionErrorr )r* rH r� r� �outputZ wait_stepZmax_sleep_timeZtotal_sleep_timer
154 r
155 r r� = s& � �� �zFileSystemCache._run_safelyrE c c s< � | � t||�}|d u rt�z
156 |V W |�� d S |�� w rD )r� �openr# �close)r* rE r �fsr
157 r
158 r rT T s �z!FileSystemCache._safe_stream_open)r r NN)NN)r
159 NrD rx )F)*�__name__�
160 __module__� __qualname__�__doc__rB r3 � staticmethodr r r rt ro r; �_t�Optional�Anyr r �propertyr5 r& r: �boolrC � Generatorr) rK �floatra ri rj rm r> r2 rz r8 r� r� �Callabler� r rT r
161 r
162 r
163 r r sz �����
164 �"���
165 �
166 #$
167 �����
168 �+" r )r )r$ r r\ r! r- r/ rU r� �typingr� �
169 contextlibr �pathlibr r r Z
170 cachelib.baser Zcachelib.serializersr �bytesr� r r r
171 r
172 r
173 r �<module> s"