inputtransformer2.cpython-310.pyc // MEDIA VIEWER

Format: application/x-python-code
⬇ Download File (24.2 KB)
   1  o
   2  
   3  ��h5s�@sfdZddlZddlmZmZddlZddlZddlZddlm	Z	m
   4  Z
   5  mZmZddl
   6  Z
   7  ddlmZe�d�Zdd�Zd	d
   8  �ZGdd�d�Zee�d
   9  �e�d�d�Zee�dej��Zdd�Zdeefdd�Zdefdd�Zde
  10  eefdefdd�ZGdd�d�ZGdd �d e�ZGd!d"�d"e�Zd#Z d$Z!d%Z"d&Z#d'Z$d(Z%d)Z&d*Z'd+Z(hd,�Z)d$d&hZ*d-d.�Z+d/d0�Z,d1d2�Z-d3d4�Z.d5d6�Z/d7d8�Z0d9d:�Z1e d;j2e!d<j2e"e,e#e-e$e.e&e/e'e0e(e1iZ3Gd=d>�d>e�Z4e�d?ej�Z5Gd@dA�dAe�Z6dBe	e7fdCdD�Z8dEe	ej9fdFdG�Z:dHe7fdIdJ�Z;dKZ<GdLdM�dM�Z=dNdO�Z>GdPdQ�dQe�Z?GdRdS�dSe�Z@ejAZBe@eBdT�ZCdS)Ua5Input transformer machinery to support IPython special syntax.
  11  
  12  This includes the machinery to recognise and transform ``%magic`` commands,
  13  ``!system`` commands, ``help?`` querying, prompt stripping, and so forth.
  14  
  15  Added: IPython 7.0. Replaces inputsplitter and inputtransformer which were
  16  deprecated in 7.0.
  17  �N)�CommandCompiler�Compile)�List�Tuple�Optional�Any)�	tokenutilz^[ \t]+cCs:|s|St|�D]\}}|r|��s||d�Sq|S)zyRemove leading empty lines
  18  
  19      If the leading lines are empty or contain only whitespace, they will be
  20      removed.
  21      N)�	enumerate�isspace)�lines�i�line�r�Tc:\Users\shubh\Videos\agent\nova\Lib\site-packages\IPython/core/inputtransformer2.py�leading_empty_liness�rcsD|s|St�|d�}|s|S|�d��t�����fdd�|D�S)z�Remove leading indentation.
  22  
  23      If the first line starts with a spaces or tabs, the same whitespace will be
  24      removed from each following line in the cell.
  25      rcs&g|]}|���r|�d�n|�qS�N)�
  26  startswith��.0�l��n�spacerr�
  27  <listcomp>3s�z"leading_indent.<locals>.<listcomp>)�
  28  _indent_re�match�group�len�r�mrrr�leading_indent&s
  29  �r c@s*eZdZdZd	dd�Zdd�Zdd�ZdS)
  30  �PromptStripperaARemove matching input prompts from a block of input.
  31  
  32      Parameters
  33      ----------
  34      prompt_re : regular expression
  35          A regular expression matching any input prompt (including continuation,
  36          e.g. ``...``)
  37      initial_re : regular expression, optional
  38          A regular expression matching only the initial prompt, but not continuation.
  39          If no initial expression is given, prompt_re will be used everywhere.
  40          Used mainly for plain Python prompts (``>>>``), where the continuation prompt
  41          ``...`` is a valid Python expression in Python 3, so shouldn't be stripped.
  42  
  43      Notes
  44      -----
  45  
  46      If initial_re and prompt_re differ,
  47      only initial_re will be tested against the first line.
  48      If any prompt is found on the first two lines,
  49      prompts will be stripped from the rest of the block.
  50      NcCs||_|p||_dSr��	prompt_re�
  51  initial_re)�selfr#r$rrr�__init__LszPromptStripper.__init__cs�fdd�|D�S)Ncsg|]}�jjd|dd��qS)��)�count)r#�subr�r%rrrQsz)PromptStripper._strip.<locals>.<listcomp>r�r%rrr+r�_stripPszPromptStripper._stripcCsB|s|S|j�|d�st|�dkr|j�|d�r|�|�S|S�Nrr()r$rrr#r-r,rrr�__call__Ss
  52  zPromptStripper.__call__r)�__name__�
  53  __module__�__qualname__�__doc__r&r-r/rrrrr!6s
  54  
  55  r!z^(>>>|\.\.\.)( |$)z	^>>>( |$)r"a
  56          ^(                         # Match from the beginning of a line, either:
  57  
  58                                     # 1. First-line prompt:
  59          ((\[nav\]|\[ins\])?\ )?    # Vi editing mode prompt, if it's there
  60          In\                        # The 'In' of the prompt, with a space
  61          \[\d+\]:                   # Command index, as displayed in the prompt
  62          \                          # With a mandatory trailing space
  63  
  64          |                          # ... or ...
  65  
  66                                     # 2. The three dots of the multiline prompt
  67          \s*                        # All leading whitespace characters
  68          \.{3,}:                    # The three (or more) dots
  69          \ ?                        # With an optional trailing space
  70  
  71          )
  72          cCsl|r	|d�d�s|St�d|d�r|S|ddd����d�\}}}d�|dd��}d|||fgS)	Nr�%%z%%\w+\?�� r'r(z)get_ipython().run_cell_magic(%r, %r, %r)
  73  )r�rer�rstrip�	partition�join)r�
  74  magic_name�_�
  75  first_line�bodyrrr�
  76  cell_magicys �r?�returncCsbd}t|�D](\}}|j}|dkr|dkr|S|dvr"|d7}q|dvr.|dkr.|d8}qdS)z�Get the index of the first assignment in the line ('=' not inside brackets)
  77  
  78      Note: We don't try to support multiple special assignment (a = b = %foo)
  79      r�=��[�(�{r(��]�)�}N)r	�string)Z
  80  token_lineZparen_levelr�ti�srrr�_find_assign_op�s
  81  �rM�
  82  start_linecCs>|}||�d�r|d7}|t|�kr	|S||�d�s	|S)zjFind the last line of a line explicitly extended using backslashes.
  83  
  84      Uses 0-indexed line numbers.
  85      z\
  86  r()�endswithr)rrN�end_linerrr�find_end_of_continued_line�s�rQ�startrPcCs\||d|dd�g||dd|d�}d�dd�|dd�D�|d��g�S)aAssemble a single line from multiple continued line pieces
  87  
  88      Continued lines are lines ending in ``\``, and the line following the last
  89      ``\`` in the block.
  90  
  91      For example, this code continues over multiple lines::
  92  
  93          if (assign_ix is not None) \
  94               and (len(line) >= assign_ix + 2) \
  95               and (line[assign_ix+1].string == '%') \
  96               and (line[assign_ix+2].type == tokenize.NAME):
  97  
  98      This statement contains four continued line pieces.
  99      Assembling these pieces into a single line would give::
 100  
 101          if (assign_ix is not None) and (len(line) >= assign_ix + 2) and (line[...
 102  
 103      This uses 0-indexed line numbers. *start* is (lineno, colno).
 104  
 105      Used to allow ``%magic`` and ``!system`` commands to be continued over
 106      multiple lines.
 107      rr(Nr6cSsg|]
 108  }|��dd��qS)N�����)r8)r�prrrr�sz+assemble_continued_line.<locals>.<listcomp>rS)r:r8)rrRrP�partsrrr�assemble_continued_line�s2�rVc@sBeZdZdZdZdd�Zdd�Zedd��Zd	e	e
 109  fd
 110  d�ZdS)
 111  �TokenTransformBasea�Base class for transformations which examine tokens.
 112  
 113      Special syntax should not be transformed when it occurs inside strings or
 114      comments. This is hard to reliably avoid with regexes. The solution is to
 115      tokenise the code as Python, and recognise the special syntax in the tokens.
 116  
 117      IPython's special syntax is not valid Python syntax, so tokenising may go
 118      wrong after the special syntax starts. These classes therefore find and
 119      transform *one* instance of special syntax at a time into regular Python
 120      syntax. After each transformation, tokens are regenerated to find the next
 121      piece of special syntax.
 122  
 123      Subclasses need to implement one class method (find)
 124      and one regular method (transform).
 125  
 126      The priority attribute can select which transformation to apply if multiple
 127      transformers match in the same place. Lower numbers have higher priority.
 128      This allows "%magic?" to be turned into a help call rather than a magic call.
 129      �
 130  cCs|j|j|jfSr)rN�	start_col�priorityr+rrr�sortby��zTokenTransformBase.sortbycCs|dd|_|d|_dSr.)rNrY)r%rRrrrr&�szTokenTransformBase.__init__cC�t�)a�Find one instance of special syntax in the provided tokens.
 131  
 132          Tokens are grouped into logical lines for convenience,
 133          so it is easy to e.g. look at the first token of each line.
 134          *tokens_by_line* is a list of lists of tokenize.TokenInfo objects.
 135  
 136          This should return an instance of its class, pointing to the start
 137          position it has found, or None if it found no match.
 138          ��NotImplementedError��cls�tokens_by_linerrr�find�szTokenTransformBase.findrcCr])z�Transform one instance of special syntax found by ``find()``
 139  
 140          Takes a list of strings representing physical lines,
 141          returns a similar list of transformed lines.
 142          r^r,rrr�	transform�szTokenTransformBase.transformN)r0r1r2r3rZr[r&�classmethodrcr�strrdrrrrrW�s
 143  rWc@s.eZdZdZedd��Zdeefdd�ZdS)�MagicAssignz2Transformer for assignments from magics (a = %foo)cCsj|D]0}t|�}|dur2t|�|dkr2||djdkr2||djtjkr2|||dj�SqdS)z@Find the first magic assignment (a = %foo) in the cell.
 144          Nr5r(�%)rMrrJ�type�tokenize�NAMErR�rarbr
 145  �	assign_ixrrrrc�s��zMagicAssign.findrcCs�|j|j}}||d|�}t||�}t|||f|�}|�d�s%J|��|dd��d�\}}}	|d|�}
 146  d�||	�}||d}||dd�}
 147  |
 148  |g|
 149  S)zJTransform a magic assignment found by the ``find()`` classmethod.
 150          Nrhr(r6z(get_ipython().run_line_magic({!r}, {!r})�
 151  )rNrYrQrVrr9�format)r%rrNrY�lhsrP�rhsr;r<�args�lines_before�call�new_line�lines_afterrrrrd�s
 152  zMagicAssign.transformN)	r0r1r2r3rercrrfrdrrrrrg�s
 153  
 154  rgc@sFeZdZdZedd��Zedd��Zedd��Zdee	fd	d
 155  �Z
 156  dS)�SystemAssignz;Transformer for assignments from system commands (a = !foo)cCs�|D]b}t|�}|durd||j���d�sdt|�|dkrd||djtjkrd|d}|t|�krd||jtjkrd||jdkrJ|||j	�S||j�
 157  �sRn|d7}|t|�krd||jtjks:qdS�NrAr5r(�!)rMr
 158  �striprrrirj�
 159  ERRORTOKENrJrRr
 160  )rarbr
 161  rm�ixrrr�find_pre_312s"����zSystemAssign.find_pre_312cCs~|D]:}t|�}|dur<||j���d�s<t|�|dkr<||djtjkr<||djdkr<|||dj	�SqdSrx)
 162  rMr
 163  rzrrrirj�OPrJrRrlrrr�
 164  find_post_312#s���zSystemAssign.find_post_312cCstjdkr
 165  |�|�S|�|�S)z8Find the first system assignment (a = !foo) in the cell.)��)�sys�version_infor}rr`rrrrc0s
 166  
 167  
 168  zSystemAssign.findrcCs�|j|j}}||d|�}t||�}t|||f|�}|�d�s%J|��|dd�}|d|�}d�|�}	||	d}
 169  ||dd�}||
 170  g|S)zKTransform a system assignment found by the ``find()`` classmethod.
 171          Nryr(�get_ipython().getoutput({!r})rn)rNrYrQrVrro)r%rrNrYrprPrq�cmdrsrtrurvrrrrd7s
 172  
 173  zSystemAssign.transformN)r0r1r2r3rer}rrcrrfrdrrrrrws
 174  
 175  
 176  rwryz!!�?�??rhr4�,�;�/>r�ryr�r�rhr�cCsP|dkrdnd|vrdnd}d�||g�}|�d�\}}}|�t�}d||fS)zUPrepares a pinfo(2)/psearch call from a target name and the escape
 177      (i.e. ? or ??)r�Zpinfo2�*ZpsearchZpinfor6�$get_ipython().run_line_magic(%r, %r))r:r9�lstrip�	ESC_MAGIC)�target�esc�method�argZt_magic_namer<Z
 178  t_magic_arg_srrr�_make_help_call\s�
 179  r�cC�|sdSt|d�S)zrTranslate lines escaped with: ?
 180  
 181      A naked help line should fire the intro help screen (shell.show_usage())
 182      �get_ipython().show_usage()r��r���contentrrr�_tr_helpi�
 183  r�cCr�)zsTranslate lines escaped with: ??
 184  
 185      A naked help line should fire the intro help screen (shell.show_usage())
 186      r�r�r�r�rrr�	_tr_help2sr�r�cC�|�d�\}}}d||fS)z.Translate lines escaped with a percent sign: %r6r��r9�r��namer<rrrrr�	_tr_magic}�r�cCs&|�d�\}}}d|d�|���fS)z'Translate lines escaped with a comma: ,r6�%s("%s")z", ")r9r:�splitr�rrr�	_tr_quote�sr�cCr�)z+Translate lines escaped with a semicolon: ;r6r�r�r�rrr�
 187  _tr_quote2�r�r�cCs>|�d�\}}}|dkrtdt�d���d|d�|���fS)z'Translate lines escaped with a slash: /r6r'�"z%" must be followed by a callable namez%s(%s)z, )r9�SyntaxError�	ESC_SHELLr:r�r�rrr�	_tr_paren�sr�zget_ipython().system({!r})r�c@s$eZdZdZedd��Zdd�ZdS)�EscapedCommandz9Transformer for escaped commands like %foo, !foo, or /foocCs�|D]B}|sqd}t|�}||kr/||jtjtjhvr/|d7}||kr/||jtjtjhvs||kr4q||jtvrD|||j�SqdS)zGFind the first escaped command (%foo, !foo, etc.) in the cell.
 188          rr(N)rrirj�INDENT�DEDENTrJ�ESCAPE_SINGLESrR)rarbr
 189  r|�llrrrrc�s���zEscapedCommand.findc
 190  Cs�|j|j}}||d|�}t||�}t|||f|�}t|�dkr8|dd�tvr8|dd�|dd�}}n
 191  |dd�|dd�}}|tvrPt||�}	nd}	|d|�}
 192  ||	d}||dd�}|
 193  |g|S)zGTransform an escaped line found by the ``find()`` classmethod.
 194          Nr(r5r'rn)rNrYrQrVr�ESCAPE_DOUBLES�tr)
 195  r%rrNrY�indentrPr
 196  �escaper�rtrsrurvrrrrd�s
 197  zEscapedCommand.transformN)r0r1r2r3rercrdrrrrr��s
 198  
 199  r�z�(%{0,2}
 200      (?!\d)[\w*]+            # Variable name
 201      (\.(?!\d)[\w*]+|\[-?[0-9]+\])*       # .etc.etc or [0], we only support literal integers.
 202      )
 203      (\?\??)$                # ? or ??
 204      cs8eZdZdZdZ�fdd�Zedd��Zdd�Z�Z	S)	�HelpEndz+Transformer for help syntax: obj? and obj??�cs(t��|�|dd|_|d|_dSr.)�superr&�q_lineZq_col)r%rRZq_locn��	__class__rrr&�szHelpEnd.__init__cCsz|D]8}t|�dkr:|djdkr:d}||jtjtjhvr-|d7}||jtjtjhvs|||j|dj�SqdS)z8Find the first help command (foo?) in the cell.
 205          r5�����r�rr(N)rrJrirjr�r�rR)rarbr
 206  r|rrrrc�s���zHelpEnd.findcCs�d�||j|jd��}|d|j�||jd�}}|d|j�}||jdd�}t�|�}|s7t|��|dus?J|��|�d�}|�d�}	t||	�}
 207  ||
 208  d}||g|S)zFTransform a help command found by the ``find()`` classmethod.
 209          r'r(Nr�rn)	r:rNr�rY�_help_end_re�searchr�rr�)r%rZpiecer�r�rsrvrr�r�rtrurrrrd�s
 210  
 211  
 212  
 213  zHelpEnd.transform)
 214  r0r1r2r3rZr&rercrd�
 215  __classcell__rrr�rr��s
 216  r�rcCs�tjtj}}gg}t|�dkr|d�d�stjddd�d}zBtjt	|�j
 217  dgd�D]4}|d	�|�|j|ksC|j|krI|dkrI|�g�q,|j
 218  d
 219  vrS|d7}q,|j
 220  dvr`|dkr`|d8}q,Wn
 221  tjylYnw|d	su|��|S)acTokenize a series of lines and group tokens by line.
 222  
 223      The tokens for a multiline Python string or expression are grouped as one
 224      line. All lines except the last lines should keep their line ending ('\n',
 225      '\r\n') for this to properly work. Use `.splitlines(keeplineending=True)`
 226      for example when passing block of text to this function.
 227  
 228      r(r)rn�
 229  z
 230  ��z�`make_tokens_by_line` received a list of lines which do not have lineending markers ('\n', '\r', '\r\n', '\x0b', '\x0c'), behavior will be unspecifiedr5��
 231  stacklevelzexpected EOF)Zextra_errors_to_catchrSrBrF)rj�NEWLINE�NLrrO�warnings�warnrZgenerate_tokens_catch_errors�iter�__next__�appendrirJ�
 232  TokenError�pop)rr�r�rb�parenlev�tokenrrr�make_tokens_by_lines:�
 233  �
 234  
 235  
 236  
 237  ���r��tokenscCsFd}|D]}|jdvr|d7}q|jdvr |d8}|dkr dSqdS)zBCheck if the depth of brackets in the list of tokens drops below 0rrBr(rFTF)rJ)r�r�r�rrr�has_sunken_brackets2s
 238  
 239  
 240  �r�rLcCs\tjdtdd�|�d�s|d7}|jdd�}t|�D]}td�|D]}td|�q#qd	S)
 241  zFor investigation and debuggingz4show_linewise_tokens is deprecated since IPython 8.6r5r�rnT��keependszLine -------r6N)r�r��DeprecationWarningrO�
 242  splitlinesr��print)rLrr
 243  Ztokinforrr�show_linewise_tokens?s�
 244  ��r�i�c@sHeZdZdZdd�Zdd�Zdd�Zded	efd
 245  d�Zdefdd
 246  �Z	dS)�TransformerManagerz�Applies various transformations to a cell or code block.
 247  
 248      The key methods for external use are ``transform_cell()``
 249      and ``check_complete()``.
 250      cCs(ttttg|_tg|_ttt	t
 251  g|_dSr)rr �classic_prompt�ipython_prompt�cleanup_transformsr?�line_transformsrgrwr�r��token_transformersr+rrrr&Ws��
 252  �zTransformerManager.__init__c	Cs�t|�}g}|jD]}|�|�}|r|�|�q	|sd|fSt|tjd�}|D]}z
 253  d|�|�fWSty<Yq'wd|fS)a2Find and run the transform earliest in the code.
 254  
 255          Returns (changed, lines).
 256  
 257          This method is called repeatedly until changed is False, indicating
 258          that all available transformations are complete.
 259  
 260          The tokens following IPython special syntax might not be valid, so
 261          the transformed code is retokenised every time to identify the next
 262          piece of special syntax. Hopefully long code cells are mostly valid
 263          Python, not using lots of IPython special syntax, so this shouldn't be
 264          a performance issue.
 265          F)�keyT)	r�r�rcr��sortedrWr[rdr�)r%rrb�
 266  candidatesZtransformer_clsZtransformerZordered_transformersrrr�do_one_token_transformhs"
 267  
 268  
 269  ��z)TransformerManager.do_one_token_transformcCs4tt�D]}|�|�\}}|s|Sqtdt��)NzBInput transformation still changing after %d iterations. Aborting.)�range�TRANSFORM_LOOP_LIMITr��RuntimeError)r%rr<�changedrrr�do_token_transforms�s��z&TransformerManager.do_token_transforms�cellr@cCsL|�d�s	|d7}|jdd�}|j|jD]}||�}q|�|�}d�|�S)zTransforms a cell of input codernTr�r')rOr�r�r�r�r:)r%r�rrdrrr�transform_cell�s
 270  
 271  
 272  
 273  z!TransformerManager.transform_cellc
 274  
 275  Cs�d}t|�D]}|dkrd}n|��rnq|s|d7}|jdd�}|s'dSt|�D]}|��s2q+|�d��d�rBdt|�fSz|jD]}t|dd�sS||�}qGWn
 276  ty_Yd	Sw|d
 277  �d�ru|d��rsdt|�fSdSz|j	D]}t|dd�s�||�}qy|�
 278  |�}Wn
 279  ty�Yd	Swt|�}t|�d
 280  kr�|r�t
 281  |d
 282  �r�d	S|s�dt|�fS|ddjtjkr�|ddjtjkr�dt|�fStjtjtjh}d}	dd�|dD�tjtjhtjhfvr�t|�d
 283  kr�|��}	|d�r|ddj|v�r|d��|d�r|ddj|v�s|d�s#dt|�fS|ddjdk�r^d
 284  }
 285  |d|
 286  jtjtjhv�rO|
 287  d
 288  7}
 289  |d|
 290  jtjtjhv�s=|d|
 291  jd
 292  }d|dfS|dd
 293  j�d��rkdSz&t���t�dt�td�|�dd�}Wd�n	1�s�wYWntt t!t"t#tf�y�Yd	Sw|du�r�dt|�fS|	�r�|	d
 294  jtjk�r�|�r�dSdt|�fS|d���s�dSdS)aReturn whether a block of code is ready to execute, or should be continued
 295  
 296          Parameters
 297          ----------
 298          cell : string
 299              Python input code, which can be multiline.
 300  
 301          Returns
 302          -------
 303          status : str
 304              One of 'complete', 'incomplete', or 'invalid' if source is not a
 305              prefix of valid code.
 306          indent_spaces : int or None
 307              The number of spaces by which to indent the next line of code. If
 308              status is not 'incomplete', this is None.
 309          FrnTr�)ZcompleteN�\�
 310  incompleteZhas_side_effects)�invalidNrr4rSr(NcSsh|]}|j�qSr)ri)r�trrr�	<setcomp>�sz4TransformerManager.check_complete.<locals>.<setcomp>�:�)r�N�errorr'�exec)�symbol)$�reversedrzr�rO�find_last_indentr��getattrr�rr�r�r�rr�rirj�	ENDMARKERr{r��COMMENTr�r�rJr�rRr
 311  r��catch_warnings�simplefilter�
 312  SyntaxWarning�compile_commandr:�
 313  OverflowError�
 314  ValueError�	TypeError�MemoryError)
 315  r%r�Zends_with_newline�	characterrr
 316  rdrbZ
 317  newline_typesZlast_token_liner|r��resrrr�check_complete�s�
 318  ���
 319  ���
 320  �
 321  ��
 322  �
 323  ��
 324  ��
 325  z!TransformerManager.check_completeN)
 326  r0r1r2r3r&r�r�rfr�rrrrrr�Qs 	r�cCs,t�|d�}|sdSt|�d��dd��S)NrSr�	z    )rrrr�replacerrrrr�*sr�cseZdZd�fdd�	Z�ZS)�MaybeAsyncCompilercst���|j|O_dSr)r�r&�flags�r%�extra_flagsr�rrr&2s
 327  zMaybeAsyncCompile.__init__�r)r0r1r2r&r�rrr�rr1src@seZdZddd�ZdS)�MaybeAsyncCommandCompilerrcCst|d�|_dS)N�r)r�compilerrrrrr&8r\z"MaybeAsyncCommandCompiler.__init__Nr)r0r1r2r&rrrrr7srr	)Dr3�astZcodeoprrr7r�rj�typingrrrrr�Z
 328  IPython.utilsr�compilerrr r!r��VERBOSEr�r?�intrMrQrVrWrgrwr�Z
 329  ESC_SH_CAPZESC_HELPZ	ESC_HELP2r�Z
 330  ESC_MAGIC2Z	ESC_QUOTEZ
 331  ESC_QUOTE2Z	ESC_PARENr�r�r�r�r�r�r�r�r�ror�r�r�r�rfr��	TokenInfor�r�r�r�r�rr�PyCF_ALLOW_TOP_LEVEL_AWAITZ_extra_flagsr�rrrr�<module>s�
 332  
 333  %���3 ?
 334  
 335  
 336  �	,�.0
 337  Z