Format: application/x-python-code
1 o
2
3 �h�i � @ s d Z ddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl
4 m Z ddl m
5 Z
6 mZ ddlmZmZ ddlmZ ddlmZmZmZmZmZ ddlZddlmZ dd lm Z dd
7 l m Z dZ d Z d Z!G d
8 d� d�Z"G dd� d�Z#G dd� de�Z$dS )z.Wrappers for forwarding stdout/stderr over zmq� N)�b2a_hex)� defaultdict�deque)�StringIO�
9 TextIOBase)�local)�Any�Callable�Deque�Dict�Optional)�extract_header)�IOLoop)� ZMQStream� i� c @ s� e Zd ZdZd*dd�Zdd� Zdd� Zd d
10 � Zd d � Ze d
11 d� �Z
12 dd� Z dd� Z dd� Z
13 dd� Zdd� Zdd� Zdd � Zd d � Zd d � Ze d!d"� �Zd#d$� Zd%d&� Zd'd(� Zd)S )+� IOPubThreada An object for sending IOPub messages in a background thread
14
15 Prevents a blocking main thread from delaying output from threads.
16
17 IOPubThread(pub_socket).background_socket is a Socket-API-providing object
18 whose IO is always run in a thread.
19 Fc C s� || _ d| _t| �| _t�� | _|| _tdd�| _ |r | �
20 � t � � | _
21 t� | _i | _t �� | _d| _d| _| �� t j| jdd�| _d| j_d| j_d| j_d| j_ dS )a Create IOPub thread
22
23 Parameters
24 ----------
25 socket : zmq.PUB Socket
26 the socket on which messages will be sent.
27 pipe : bool
28 Whether this process should listen for IOPub messages
29 piped from subprocesses.
30 F)Z make_current�
31 N�IOPub)�target�nameT) �socket�_stopped�BackgroundSocketZbackground_socket�os�getpid� _master_pid�
32 _pipe_flagr �io_loop�_setup_pipe_in� threadingr �_localr �_events� _event_pipes�Lock�_event_pipe_gc_lock�_event_pipe_gc_seconds�_event_pipe_gc_task�_setup_event_pipe�Thread� _thread_main�thread�daemonZpydev_do_not_traceZis_pydev_daemon_threadr )�selfr �pipe� r. �Hc:\Users\shubh\Videos\agent\nova\Lib\site-packages\ipykernel/iostream.py�__init__1 s(
33
34
35
36 zIOPubThread.__init__c sn � fdd�}� j �|� � js� j �� � jdur.� fdd�}� js)� j �|� n� j�� � j jdd� dS )z.The inner loop that's actually run in a threadc s t �� �� �� _d S �N)�asyncio�
37 ensure_future�_run_event_pipe_gcr& r. �r, r. r/ �_start_event_gcT s z1IOPubThread._thread_main.<locals>._start_event_gcNc � s �� j �� d S r1 )r& �cancelr. r5 r. r/ �_cancel` s �z)IOPubThread._thread_main.<locals>._cancelT)Zall_fds)r Zrun_syncr �startr& r7 �close)r, r6 r8 r. r5 r/ r) Q s
38
39
40 zIOPubThread._thread_mainc C sf | j j}|� tj�}d|_tt�d���d�}d| }| _ |�
41 |� t || j �| _
42 | j
43 �| j� dS )zLCreate the PULL socket listening for events that should fire in this thread.r � �asciiz inproc://%sN)r �context�zmq�PULL�lingerr r �urandom�decode�_event_interface�bindr r Z
44 _event_puller�on_recv�
45 _handle_event)r, �ctx�pipe_in�_uuidZifacer. r. r/ r' j s
46 z IOPubThread._setup_event_pipec
47 � sd � t �| j�I dH z | �� I dH W n ty0 } ztd|� �tjd� W Y d}~nd}~ww q)z&Task to run event pipe gc continuouslyTNz)Exception in IOPubThread._event_pipe_gc: ��file)r2 �sleepr% �_event_pipe_gc� Exception�print�sys�
48 __stderr__)r, �er. r. r/ r4 v s � ���z IOPubThread._run_event_pipe_gcc � sh �| j sdS | j�! t| j �� �D ]\}}|�� s!|�� | j |= qW d � dS 1 s-w Y dS )z.run a single garbage collection on event pipesN)r" r$ �list�items�is_aliver: )r, r* r r. r. r/ rM s ���"�zIOPubThread._event_pipe_gcc C s� z| j j}W |S tyD | jj}|�tj�}d|_|�| j � || j _| j
49 � || j t �
50 � < W d � Y |S 1 s<w Y Y |S w )zSthread-local event pipe for signaling events that should be processed in the threadr N)r �
51 event_pipe�AttributeErrorr r= r> �PUSHr@ �connectrC r$ r" r �current_thread)r, rV rG r. r. r/ � _event_pipe� s
52
53 � ���zIOPubThread._event_pipec C s, t | j�}t|�D ]
54 }| j�� }|� q dS )z�Handle an event on the event pipe
55
56 Content of the message is ignored.
57
58 Whenever *an* event arrives on the event stream,
59 *all* waiting events are processed in order.
60 N)�lenr! �range�popleft)r, �msgZn_events�_Zevent_fr. r. r/ rF � s
61
62
63
64 �zIOPubThread._handle_eventc
65 C s� | j j}t�d�| _|� tj�}d|_z|�d�| _ W n& tj
66 yA } zt j d| d dd� d| _
67 |�� W Y d }~d S d }~ww t|| j�| _| j�| j� d S )
68 z7setup listening pipe for IOPub from forked subprocessesr; r ztcp://127.0.0.1z)Couldn't bind IOPub Pipe to 127.0.0.1: %sz'
69 subprocess output will be unavailable.� ��
70 stacklevelFN)r r= r rA �
71 _pipe_uuidr> r? r@ Zbind_to_random_port�
72 _pipe_portZZMQError�warnings�warnr r: r r Z_pipe_inrE �_handle_pipe_msg)r, rG rH rR r. r. r/ r � s( ���� zIOPubThread._setup_pipe_inc C sJ | j r| �� s dS |d | jkrtd|tjd� dS | �|dd� � dS )z'handle a pipe message from a subprocessNr zBad pipe message: %srJ r )r �_is_master_processrd rO rP rQ �send_multipart�r, r_ r. r. r/ rh � s z IOPubThread._handle_pipe_msgc C s2 t �� }|�t j�}d|_|�d| j � ||fS )Ni� ztcp://127.0.0.1:%i)r> �Contextr rX r@ rY re )r, rG �pipe_outr. r. r/ �_setup_pipe_out� s
73 zIOPubThread._setup_pipe_outc C � t �� | jkS r1 �r r r r5 r. r. r/ ri � � z IOPubThread._is_master_processc C s | j r| �� r tS tS )z8check for forks, and switch to zmq pipeline if necessary)r ri �MASTER�CHILDr5 r. r. r/ �_check_mp_mode� s zIOPubThread._check_mp_modec C s" d| j _| j �� t�| j� dS )zStart the IOPub threadr N)r* r r9 �atexit�register�stopr5 r. r. r/ r9 � s
74 zIOPubThread.startc C sh d| _ | j�� s
75 dS | j�| jj� | jjdd� | j�� r$d}t|��| j� � D ]\}}|�
76 � q)dS )zStop the IOPub threadTN� )�timeoutz,IOPub thread did not terminate in 30 seconds) r r* rU r Z add_callbackrw �join� TimeoutErrorr" rT r: )r, r_ �_threadrV r. r. r/ rw � s
77
78
79 �zIOPubThread.stopc C s | j rdS | j�� d| _dS )zClose the IOPub thread.N)�closedr r: r5 r. r. r/ r: � s
80
81 zIOPubThread.closec C �
82 | j d u S r1 )r r5 r. r. r/ r} � �
83 zIOPubThread.closedc C s0 | j �� r| j�|� | j�d� dS |� dS )ztSchedule a function to be called in our IO thread.
84
85 If the thread is not running, call immediately.
86 � N)r* rU r! �appendr[ �send)r, �fr. r. r/ �schedule s
87
88 zIOPubThread.schedulec s �� � ��fdd�� dS )z�send_multipart schedules actual zmq send in my thread.
89
90 If my thread isn't running (e.g. forked process), send immediately.
91 c s �j � i ���S r1 )� _really_sendr. ��args�kwargsr, r. r/ �<lambda> s z,IOPubThread.send_multipart.<locals>.<lambda>N)r� �r, r� r� r. r� r/ rj s zIOPubThread.send_multipartc O sx | j rdS | �� }|tkr | jj|g|�R i |�� dS | �� \}}|j| jg|�g|�R i |�� |�� |�� dS )z)The callback that actually sends messagesN) r} rt rs r rj rn rd r: �term)r, r_ r� r� Zmp_moderG rm r. r. r/ r� s zIOPubThread._really_sendN)F)�__name__�
92 __module__� __qualname__�__doc__r0 r) r' r4 rM �propertyr[ rF r rh rn ri rt r9 rw r: r} r� rj r� r. r. r. r/ r ( s.
93
94
95 r c sH e Zd ZdZdZdd� Z� fdd�Z� fdd�Zd d
96 � Zd d � Z � Z
97 S )
98 r z>Wrapper around IOPub thread that provides zmq send[_multipart]Nc C s
99 || _ dS )zInitialize the socket.N)� io_thread)r, r� r. r. r/ r0 / r zBackgroundSocket.__init__c sr |� d�r|�d�rt� �|� | jdusJ �t| jj|�r3tjd|� d|� �t dd� t
100 | jj|�S t� �|�S )z2Wrap socket attr access for backward-compatibility�__Nz Accessing zmq Socket attribute �O on BackgroundSocket is deprecated since ipykernel 4.3.0 use .io_thread.socket.ra rb ) �
101 startswith�endswith�super� __getattr__r� �hasattrr rf rg �DeprecationWarning�getattr)r, �attr�� __class__r. r/ r� 3 s �� z BackgroundSocket.__getattr__c sl |dks|� d�r|�d�rt� �||� dS tjd|� d|� �tdd� | jdus,J �t| jj ||� dS )z Set an attribute on the socket.r� r� z Setting zmq Socket attribute r� ra rb N)
102 r� r� r� � __setattr__rf rg r� r� �setattrr )r, r� �valuer� r. r/ r� D s ��z BackgroundSocket.__setattr__c O s | j |gg|�R i |��S )z Send a message to the socket.)rj )r, r_ r� r� r. r. r/ r� S s zBackgroundSocket.sendc O s | j dusJ �| j j|i |��S )zSchedule send in IO threadN)r� rj r� r. r. r/ rj W s z BackgroundSocket.send_multipart) r� r� r� r� r� r0 r� r� r� rj �
103 __classcell__r. r. r� r/ r * s r c @ s e Zd ZU dZdZdZdZdZdZe e
104 e d< dd� Z d d
105 � Z
106 d7d d d
107 �dd�Zedd� �Zejdd� �Zdd� Zdd� Zdd� Zdd� Zdd � Zed d � �Zd d � Zd!d"� Zd#d$� Zd%ed&e e fd'd(�Z d)d*� Z d+d,� Z d-d.� Z d/d0� Z!ed1d2� �Z"d3d4� Z#d5d6� Z$dS )8� OutStreamzpA file like object that publishes the stream to a 0MQ PUB socket.
108
109 Output is handed off to an IO Thread
110 r g�������?NzUTF-8�_excc C s$ t | dd�dur | jS d}t�|��)z^
111 Things like subprocess will peak and write to the fileno() of stderr/stdout.
112 �_original_stdstream_copyN�fileno)r� r� �io�UnsupportedOperationrk r. r. r/ r� l s
113 zOutStream.filenoc C s� z4t �| jt�}|r,| jr2| �|jdd�� t �| j|� t �| jt�}|r/| js
114 W dS W dS W dS W dS tyC t �
115 � | _ Y dS w )aj
116 We've redirected standards streams 0 and 1 into a pipe.
117
118 We need to watch in a thread and redirect them to the right places.
119
120 1) the ZMQ channels to show in notebook interfaces,
121 2) the original stdout/err, to capture errors in terminals.
122
123 We cannot schedule this on the ioloop thread, as this might be blocking.
124
125 �replace)�errorsN) r �read�_fid�PIPE_BUFFER_SIZE�
126 _should_watch�writerB r� rN rP �exc_infor� )r, Zbtsr. r. r/ �_watch_pipe_fdu s
127
128 "� �zOutStream._watch_pipe_fdTF)�watchfd�isattyc
129 C s� |dur t jdtdd� || _t|t�s&t jd| tdd� t|�}|�� || _|| _d|� � | _
130 t � d�| _
131 | j
132 �i � i | _i | _i | _t�� | _d| _d| _|j| _t�� | _t t �| _ d| _ t |�| _!d| _"t#� | _$|r�t%j&�'d �s~t%j&�'d
133 �r�d tj(vs�|d kr�d
134 | _"| �)|� |r�t*|d�r�t*|d�r�| j"r�z|�+� }W n t,y� d}Y nw |dur�|| j-kr�t.�/t.�0| j1d��}|| _ dS d} t2| ��dS )a�
135 Parameters
136 ----------
137 session : object
138 the session object
139 pub_thread : threading.Thread
140 the publication thread
141 name : str {'stderr', 'stdout'}
142 the name of the standard stream to replace
143 pipe : object
144 the pipe object
145 echo : bool
146 whether to echo output
147 watchfd : bool (default, True)
148 Watch the file descriptor corresponding to the replaced stream.
149 This is useful if you know some underlying code will write directly
150 the file descriptor by its number. It will spawn a watching thread,
151 that will swap the give file descriptor for a pipe, read from the
152 pipe, and insert this into the current Stream.
153 isatty : bool (default, False)
154 Indication of whether this stream has terminal capabilities (e.g. can handle colors)
155
156 NzKpipe argument to OutStream is deprecated and ignored since ipykernel 4.2.3.ra rb zISince IPykernel 4.3, OutStream should be created with IOPubThread, not %rs stream.�
157 parent_headerF�linux�darwinZPYTEST_CURRENT_TEST�forceTr� r� �wz(echo argument must be a file-like object)3rf rg r� �session�
158 isinstancer r9 �
159 pub_threadr �encode�topic� contextvars�
160 ContextVar�_parent_header�set�_thread_to_parent�_thread_to_parent_header�_parent_header_globalr r r �_flush_pending�_subprocess_flush_pendingr �_io_loopr �RLock� _buffer_lockr r �_buffers�echo�bool�_isattyr� r r rP �platformr� �environ�_setup_stream_redirectsr� r� rN �_original_stdstream_fdr� �
161 TextIOWrapper�FileIOr� �
162 ValueError)
163 r, r� r� r r- r� r� r� Zecho_fdr_ r. r. r/ r0 � s� "�
164 ���
165
166
167
168 �
169 �
170 �
171
172 ���
173 �zOutStream.__init__c C sr z| j �� W S ty8 z t�� j}|| jv r"| j| }|| jv s| j| W Y S ty7 | j Y Y S w w r1 )
174 r� �get� LookupErrorr rZ �identr� r� �KeyErrorr� )r, �identityr. r. r/ r� � s
175
176
177
178 � ��zOutStream.parent_headerc C s || _ | j�|�S r1 )r� r� r� )r, r� r. r. r/ r� s c C s | j S )zpReturn a bool indicating whether this is an 'interactive' stream.
179
180 Returns:
181 Boolean
182 )r� r5 r. r. r/ r� s zOutStream.isattyc C sj t �� \}}tt|��� }| _t �|�| _t �||� || _ d | _
183 t j | j
184 d�| _d| j_| j�� d S )N)r T)r r- r� rP r� r� �dupr� �dup2r� r� r r( r� �watch_fd_threadr+ r9 )r, r �pr�pwZfnor. r. r/ r� s z!OutStream._setup_stream_redirectsc C ro r1 rp r5 r. r. r/ ri & rq z OutStream._is_master_processc C s t |�| _dS )zSet the parent header.N)r
185 r� )r, �parentr. r. r/ �
186 set_parent) s zOutStream.set_parentc C sj | j r d| _ t�| jd� | j�� t�| j| j� t�| j� | j r0| j \}}}t
187 � |||� d| _ dS )zClose the stream.F� N)
188 r� r r� r� r� rz r� r� r: r� � traceback�print_exceptionr� )r, �etyper� �tbr. r. r/ r: - s
189
190 zOutStream.closec C r~ r1 )r� r5 r. r. r/ r} = r zOutStream.closedc s, � j rdS d� _ � fdd�}� j�|� dS )zuschedule a flush in the IO thread
191
192 call this on write, to indicate that flush should be called soon.
193 NTc s � j �� j� j� d S r1 )r� �
194 call_later�flush_interval�_flushr. r5 r. r/ �_schedule_in_threadK s z6OutStream._schedule_flush.<locals>._schedule_in_thread)r� r� r� )r, r� r. r5 r/ �_schedule_flushA s
195 zOutStream._schedule_flushc C s� | j r<| j jdur<| j j�� r<| j jjt�� jkr<| j �| j� t�� }| j �|j � |�
196 | j �s:t dt
197 jd� dS dS | �� dS )zStrigger actual zmq send
198
199 send will happen in the background thread
200 NzIOStream.flush timed outrJ )r� r* rU r� r rZ r� r� �Eventr� �wait�
201 flush_timeoutrO rP rQ )r, Zevtr. r. r/ �flushP s �
202 � � zOutStream.flushc
203 C s� d| _ d| _| jdur5z| j�� W n" ty4 } z| jtjur*td|� �tjd� W Y d}~nd}~ww | �� D ]8\}}|rqt �
204 � | j _ | j
205 |d�}| j jd||d�}| jD ]}||�}|du re dS qW| j j| j|| jd� q9dS ) z�This is where the actual send happens.
206
207 _flush should generally be called in the IO thread,
208 unless the thread has been destroyed (e.g. forked subprocess).
209 FNzFlush failed: rJ )r �text�stream)r� )r� )r� r� r� r� �OSErrorrP rQ rO �_flush_buffersr r r� �pidr r_ �_hooksr� r� r� )r, rR r� �data�contentr_ �hookr. r. r/ r� h s6
210 ��
211 ����zOutStream._flush�string�returnc
212 C s | j }t|t�sdt|�� �}t|��| jdurCz| j�|� W n" tyB } z| jtj ur8t
213 d|� �tj d� W Y d}~nd}~ww | j du rNd}t |��| �
214 � }| j� | jt|�� � �|� W d � n1 smw Y |r�| jrydS d| _| j �| j� t|�S | �� t|�S )z�Write to current stream after encoding if necessary
215
216 Returns
217 -------
218 len : int
219 number of items from input parameter written to stream.
220
221 z"write() argument must be str, not NzWrite failed: rJ � I/O operation on closed fileT)r� r� �str�type� TypeErrorr� r� r� rP rQ rO r� r� ri r� r� � frozensetrT r� r� r� r� r\ )r, r� r� r_ rR Zis_childr. r. r/ r� � s6
222
223 ��
224
225 ��zOutStream.writec C s. | j du r d}t|��|D ]}| �|� q
226 dS )zWrite lines to the stream.Nr� )r� r� r� )r, �sequencer_ r� r. r. r/ �
227 writelines� s
228 �zOutStream.writelinesc C s dS )z$Test whether the stream is writable.Tr. r5 r. r. r/ �writable� s zOutStream.writablec c s> � | � � }|�� D ]\}}|�� }|�� t|�|fV q dS )z<clear the current buffer and return the current buffer data.N)�_rotate_buffersrT �getvaluer: �dict)r, �buffersZ
229 frozen_parent�bufferr� r. r. r/ r� � s ��zOutStream._flush_buffersc C s< | j � | j}tt�| _W d � |S 1 sw Y |S )z@Returns the current buffer and replaces it with an empty buffer.N)r� r� r r )r, Z old_buffersr. r. r/ r � s
230 ��zOutStream._rotate_buffersc C s t | jd�s
231 g | j_| jjS )N�hooks)r� r r r5 r. r. r/ r� � s zOutStream._hooksc C s | j �|� dS )a�
232 Registers a hook with the thread-local storage.
233
234 Parameters
235 ----------
236 hook : Any callable object
237
238 Returns
239 -------
240 Either a publishable message, or `None`.
241 The hook callable must return a message from
242 the __call__ method if they still require the
243 `session.send` method to be called after transformation.
244 Returning `None` will halt that execution path, and
245 session.send will not be called.
246 N)r� r� �r, r� r. r. r/ �
247 register_hook� s zOutStream.register_hookc C s( z | j �|� W dS ty Y dS w )aB
248 Un-registers a hook with the thread-local storage.
249
250 Parameters
251 ----------
252 hook : Any callable object which has previously been
253 registered as a hook.
254
255 Returns
256 -------
257 bool - `True` if the hook was removed, `False` if it wasn't
258 found.
259 TF)r� �remover� r r. r. r/ �unregister_hook� s �zOutStream.unregister_hook)NN)%r� r� r� r� r� r� r� �encodingr� r r �__annotations__r� r� r0 r� r� �setterr� r� ri r� r: r} r� r� r� r� �intr� r� r r� r r� r r
260 r. r. r. r/ r� ] sL
261 � �n
262
263
264
265 '+
266 r� )%r� r2 ru r� r� r rP r r� rf �binasciir � collectionsr r r r r �typingr r r
267 r r r> Zjupyter_client.sessionr
268 Ztornado.ioloopr Zzmq.eventloop.zmqstreamr rr rs r� r r r� r. r. r. r/ �<module> s6 3