1  """Completion for IPython.
   2  
   3  This module started as fork of the rlcompleter module in the Python standard
   4  library.  The original enhancements made to rlcompleter have been sent
   5  upstream and were accepted as of Python 2.3,
   6  
   7  This module now support a wide variety of completion mechanism both available
   8  for normal classic Python code, as well as completer for IPython specific
   9  Syntax like magics.
  10  
  11  Latex and Unicode completion
  12  ============================
  13  
  14  IPython and compatible frontends not only can complete your code, but can help
  15  you to input a wide range of characters. In particular we allow you to insert
  16  a unicode character using the tab completion mechanism.
  17  
  18  Forward latex/unicode completion
  19  --------------------------------
  20  
  21  Forward completion allows you to easily type a unicode character using its latex
  22  name, or unicode long description. To do so type a backslash follow by the
  23  relevant name and press tab:
  24  
  25  
  26  Using latex completion:
  27  
  28  .. code::
  29  
  30      \\alpha<tab>
  31      α
  32  
  33  or using unicode completion:
  34  
  35  
  36  .. code::
  37  
  38      \\GREEK SMALL LETTER ALPHA<tab>
  39      α
  40  
  41  
  42  Only valid Python identifiers will complete. Combining characters (like arrow or
  43  dots) are also available, unlike latex they need to be put after the their
  44  counterpart that is to say, ``F\\\\vec<tab>`` is correct, not ``\\\\vec<tab>F``.
  45  
  46  Some browsers are known to display combining characters incorrectly.
  47  
  48  Backward latex completion
  49  -------------------------
  50  
  51  It is sometime challenging to know how to type a character, if you are using
  52  IPython, or any compatible frontend you can prepend backslash to the character
  53  and press :kbd:`Tab` to expand it to its latex form.
  54  
  55  .. code::
  56  
  57      \\α<tab>
  58      \\alpha
  59  
  60  
  61  Both forward and backward completions can be deactivated by setting the
  62  :std:configtrait:`Completer.backslash_combining_completions` option to
  63  ``False``.
  64  
  65  
  66  Experimental
  67  ============
  68  
  69  Starting with IPython 6.0, this module can make use of the Jedi library to
  70  generate completions both using static analysis of the code, and dynamically
  71  inspecting multiple namespaces. Jedi is an autocompletion and static analysis
  72  for Python. The APIs attached to this new mechanism is unstable and will
  73  raise unless use in an :any:`provisionalcompleter` context manager.
  74  
  75  You will find that the following are experimental:
  76  
  77      - :any:`provisionalcompleter`
  78      - :any:`IPCompleter.completions`
  79      - :any:`Completion`
  80      - :any:`rectify_completions`
  81  
  82  .. note::
  83  
  84      better name for :any:`rectify_completions` ?
  85  
  86  We welcome any feedback on these new API, and we also encourage you to try this
  87  module in debug mode (start IPython with ``--Completer.debug=True``) in order
  88  to have extra logging information if :any:`jedi` is crashing, or if current
  89  IPython completer pending deprecations are returning results not yet handled
  90  by :any:`jedi`
  91  
  92  Using Jedi for tab completion allow snippets like the following to work without
  93  having to execute any code:
  94  
  95     >>> myvar = ['hello', 42]
  96     ... myvar[1].bi<tab>
  97  
  98  Tab completion will be able to infer that ``myvar[1]`` is a real number without
  99  executing almost any code unlike the deprecated :any:`IPCompleter.greedy`
 100  option.
 101  
 102  Be sure to update :any:`jedi` to the latest stable version or to try the
 103  current development version to get better completions.
 104  
 105  Matchers
 106  ========
 107  
 108  All completions routines are implemented using unified *Matchers* API.
 109  The matchers API is provisional and subject to change without notice.
 110  
 111  The built-in matchers include:
 112  
 113  - :any:`IPCompleter.dict_key_matcher`:  dictionary key completions,
 114  - :any:`IPCompleter.magic_matcher`: completions for magics,
 115  - :any:`IPCompleter.unicode_name_matcher`,
 116    :any:`IPCompleter.fwd_unicode_matcher`
 117    and :any:`IPCompleter.latex_name_matcher`: see `Forward latex/unicode completion`_,
 118  - :any:`back_unicode_name_matcher` and :any:`back_latex_name_matcher`: see `Backward latex completion`_,
 119  - :any:`IPCompleter.file_matcher`: paths to files and directories,
 120  - :any:`IPCompleter.python_func_kw_matcher` - function keywords,
 121  - :any:`IPCompleter.python_matches` - globals and attributes (v1 API),
 122  - ``IPCompleter.jedi_matcher`` - static analysis with Jedi,
 123  - :any:`IPCompleter.custom_completer_matcher` - pluggable completer with a default
 124    implementation in :any:`InteractiveShell` which uses IPython hooks system
 125    (`complete_command`) with string dispatch (including regular expressions).
 126    Differently to other matchers, ``custom_completer_matcher`` will not suppress
 127    Jedi results to match behaviour in earlier IPython versions.
 128  
 129  Custom matchers can be added by appending to ``IPCompleter.custom_matchers`` list.
 130  
 131  Matcher API
 132  -----------
 133  
 134  Simplifying some details, the ``Matcher`` interface can described as
 135  
 136  .. code-block::
 137  
 138      MatcherAPIv1 = Callable[[str], list[str]]
 139      MatcherAPIv2 = Callable[[CompletionContext], SimpleMatcherResult]
 140  
 141      Matcher = MatcherAPIv1 | MatcherAPIv2
 142  
 143  The ``MatcherAPIv1`` reflects the matcher API as available prior to IPython 8.6.0
 144  and remains supported as a simplest way for generating completions. This is also
 145  currently the only API supported by the IPython hooks system `complete_command`.
 146  
 147  To distinguish between matcher versions ``matcher_api_version`` attribute is used.
 148  More precisely, the API allows to omit ``matcher_api_version`` for v1 Matchers,
 149  and requires a literal ``2`` for v2 Matchers.
 150  
 151  Once the API stabilises future versions may relax the requirement for specifying
 152  ``matcher_api_version`` by switching to :any:`functools.singledispatch`, therefore
 153  please do not rely on the presence of ``matcher_api_version`` for any purposes.
 154  
 155  Suppression of competing matchers
 156  ---------------------------------
 157  
 158  By default results from all matchers are combined, in the order determined by
 159  their priority. Matchers can request to suppress results from subsequent
 160  matchers by setting ``suppress`` to ``True`` in the ``MatcherResult``.
 161  
 162  When multiple matchers simultaneously request suppression, the results from of
 163  the matcher with higher priority will be returned.
 164  
 165  Sometimes it is desirable to suppress most but not all other matchers;
 166  this can be achieved by adding a set of identifiers of matchers which
 167  should not be suppressed to ``MatcherResult`` under ``do_not_suppress`` key.
 168  
 169  The suppression behaviour can is user-configurable via
 170  :std:configtrait:`IPCompleter.suppress_competing_matchers`.
 171  """
 172  
 173  
 174  # Copyright (c) IPython Development Team.
 175  # Distributed under the terms of the Modified BSD License.
 176  #
 177  # Some of this code originated from rlcompleter in the Python standard library
 178  # Copyright (C) 2001 Python Software Foundation, www.python.org
 179  
 180  from __future__ import annotations
 181  import builtins as builtin_mod
 182  import enum
 183  import glob
 184  import inspect
 185  import itertools
 186  import keyword
 187  import ast
 188  import os
 189  import re
 190  import string
 191  import sys
 192  import tokenize
 193  import time
 194  import unicodedata
 195  import uuid
 196  import warnings
 197  from ast import literal_eval
 198  from collections import defaultdict
 199  from contextlib import contextmanager
 200  from dataclasses import dataclass
 201  from functools import cached_property, partial
 202  from types import SimpleNamespace
 203  from typing import (
 204      Iterable,
 205      Iterator,
 206      List,
 207      Tuple,
 208      Union,
 209      Any,
 210      Sequence,
 211      Dict,
 212      Optional,
 213      TYPE_CHECKING,
 214      Set,
 215      Sized,
 216      TypeVar,
 217      Literal,
 218  )
 219  
 220  from IPython.core.guarded_eval import guarded_eval, EvaluationContext
 221  from IPython.core.error import TryNext
 222  from IPython.core.inputtransformer2 import ESC_MAGIC
 223  from IPython.core.latex_symbols import latex_symbols, reverse_latex_symbol
 224  from IPython.core.oinspect import InspectColors
 225  from IPython.testing.skipdoctest import skip_doctest
 226  from IPython.utils import generics
 227  from IPython.utils.decorators import sphinx_options
 228  from IPython.utils.dir2 import dir2, get_real_method
 229  from IPython.utils.docs import GENERATING_DOCUMENTATION
 230  from IPython.utils.path import ensure_dir_exists
 231  from IPython.utils.process import arg_split
 232  from traitlets import (
 233      Bool,
 234      Enum,
 235      Int,
 236      List as ListTrait,
 237      Unicode,
 238      Dict as DictTrait,
 239      Union as UnionTrait,
 240      observe,
 241  )
 242  from traitlets.config.configurable import Configurable
 243  
 244  import __main__
 245  
 246  # skip module docstests
 247  __skip_doctest__ = True
 248  
 249  
 250  try:
 251      import jedi
 252      jedi.settings.case_insensitive_completion = False
 253      import jedi.api.helpers
 254      import jedi.api.classes
 255      JEDI_INSTALLED = True
 256  except ImportError:
 257      JEDI_INSTALLED = False
 258  
 259  
 260  if TYPE_CHECKING or GENERATING_DOCUMENTATION and sys.version_info >= (3, 11):
 261      from typing import cast
 262      from typing_extensions import TypedDict, NotRequired, Protocol, TypeAlias, TypeGuard
 263  else:
 264      from typing import Generic
 265  
 266      def cast(type_, obj):
 267          """Workaround for `TypeError: MatcherAPIv2() takes no arguments`"""
 268          return obj
 269  
 270      # do not require on runtime
 271      NotRequired = Tuple  # requires Python >=3.11
 272      TypedDict = Dict  # by extension of `NotRequired` requires 3.11 too
 273      Protocol = object  # requires Python >=3.8
 274      TypeAlias = Any  # requires Python >=3.10
 275      TypeGuard = Generic  # requires Python >=3.10
 276  if GENERATING_DOCUMENTATION:
 277      from typing import TypedDict
 278  
 279  # -----------------------------------------------------------------------------
 280  # Globals
 281  #-----------------------------------------------------------------------------
 282  
 283  # ranges where we have most of the valid unicode names. We could be more finer
 284  # grained but is it worth it for performance  While unicode have character in the
 285  # range 0, 0x110000, we seem to have name for about 10% of those. (131808 as I
 286  # write this). With below range we cover them all, with a density of ~67%
 287  # biggest next gap we consider only adds up about 1% density and there are 600
 288  # gaps that would need hard coding.
 289  _UNICODE_RANGES = [(32, 0x323B0), (0xE0001, 0xE01F0)]
 290  
 291  # Public API
 292  __all__ = ["Completer", "IPCompleter"]
 293  
 294  if sys.platform == 'win32':
 295      PROTECTABLES = ' '
 296  else:
 297      PROTECTABLES = ' ()[]{}?=\\|;:\'#*"^&'
 298  
 299  # Protect against returning an enormous number of completions which the frontend
 300  # may have trouble processing.
 301  MATCHES_LIMIT = 500
 302  
 303  # Completion type reported when no type can be inferred.
 304  _UNKNOWN_TYPE = "<unknown>"
 305  
 306  # sentinel value to signal lack of a match
 307  not_found = object()
 308  
 309  class ProvisionalCompleterWarning(FutureWarning):
 310      """
 311      Exception raise by an experimental feature in this module.
 312  
 313      Wrap code in :any:`provisionalcompleter` context manager if you
 314      are certain you want to use an unstable feature.
 315      """
 316      pass
 317  
 318  warnings.filterwarnings('error', category=ProvisionalCompleterWarning)
 319  
 320  
 321  @skip_doctest
 322  @contextmanager
 323  def provisionalcompleter(action='ignore'):
 324      """
 325      This context manager has to be used in any place where unstable completer
 326      behavior and API may be called.
 327  
 328      >>> with provisionalcompleter():
 329      ...     completer.do_experimental_things() # works
 330  
 331      >>> completer.do_experimental_things() # raises.
 332  
 333      .. note::
 334  
 335          Unstable
 336  
 337          By using this context manager you agree that the API in use may change
 338          without warning, and that you won't complain if they do so.
 339  
 340          You also understand that, if the API is not to your liking, you should report
 341          a bug to explain your use case upstream.
 342  
 343          We'll be happy to get your feedback, feature requests, and improvements on
 344          any of the unstable APIs!
 345      """
 346      with warnings.catch_warnings():
 347          warnings.filterwarnings(action, category=ProvisionalCompleterWarning)
 348          yield
 349  
 350  
 351  def has_open_quotes(s: str) -> Union[str, bool]:
 352      """Return whether a string has open quotes.
 353  
 354      This simply counts whether the number of quote characters of either type in
 355      the string is odd.
 356  
 357      Returns
 358      -------
 359      If there is an open quote, the quote character is returned.  Else, return
 360      False.
 361      """
 362      # We check " first, then ', so complex cases with nested quotes will get
 363      # the " to take precedence.
 364      if s.count('"') % 2:
 365          return '"'
 366      elif s.count("'") % 2:
 367          return "'"
 368      else:
 369          return False
 370  
 371  
 372  def protect_filename(s: str, protectables: str = PROTECTABLES) -> str:
 373      """Escape a string to protect certain characters."""
 374      if set(s) & set(protectables):
 375          if sys.platform == "win32":
 376              return '"' + s + '"'
 377          else:
 378              return "".join(("\\" + c if c in protectables else c) for c in s)
 379      else:
 380          return s
 381  
 382  
 383  def expand_user(path:str) -> Tuple[str, bool, str]:
 384      """Expand ``~``-style usernames in strings.
 385  
 386      This is similar to :func:`os.path.expanduser`, but it computes and returns
 387      extra information that will be useful if the input was being used in
 388      computing completions, and you wish to return the completions with the
 389      original '~' instead of its expanded value.
 390  
 391      Parameters
 392      ----------
 393      path : str
 394          String to be expanded.  If no ~ is present, the output is the same as the
 395          input.
 396  
 397      Returns
 398      -------
 399      newpath : str
 400          Result of ~ expansion in the input path.
 401      tilde_expand : bool
 402          Whether any expansion was performed or not.
 403      tilde_val : str
 404          The value that ~ was replaced with.
 405      """
 406      # Default values
 407      tilde_expand = False
 408      tilde_val = ''
 409      newpath = path
 410  
 411      if path.startswith('~'):
 412          tilde_expand = True
 413          rest = len(path)-1
 414          newpath = os.path.expanduser(path)
 415          if rest:
 416              tilde_val = newpath[:-rest]
 417          else:
 418              tilde_val = newpath
 419  
 420      return newpath, tilde_expand, tilde_val
 421  
 422  
 423  def compress_user(path:str, tilde_expand:bool, tilde_val:str) -> str:
 424      """Does the opposite of expand_user, with its outputs.
 425      """
 426      if tilde_expand:
 427          return path.replace(tilde_val, '~')
 428      else:
 429          return path
 430  
 431  
 432  def completions_sorting_key(word):
 433      """key for sorting completions
 434  
 435      This does several things:
 436  
 437      - Demote any completions starting with underscores to the end
 438      - Insert any %magic and %%cellmagic completions in the alphabetical order
 439        by their name
 440      """
 441      prio1, prio2 = 0, 0
 442  
 443      if word.startswith('__'):
 444          prio1 = 2
 445      elif word.startswith('_'):
 446          prio1 = 1
 447  
 448      if word.endswith('='):
 449          prio1 = -1
 450  
 451      if word.startswith('%%'):
 452          # If there's another % in there, this is something else, so leave it alone
 453          if "%" not in word[2:]:
 454              word = word[2:]
 455              prio2 = 2
 456      elif word.startswith('%'):
 457          if "%" not in word[1:]:
 458              word = word[1:]
 459              prio2 = 1
 460  
 461      return prio1, word, prio2
 462  
 463  
 464  class _FakeJediCompletion:
 465      """
 466      This is a workaround to communicate to the UI that Jedi has crashed and to
 467      report a bug. Will be used only id :any:`IPCompleter.debug` is set to true.
 468  
 469      Added in IPython 6.0 so should likely be removed for 7.0
 470  
 471      """
 472  
 473      def __init__(self, name):
 474  
 475          self.name = name
 476          self.complete = name
 477          self.type = 'crashed'
 478          self.name_with_symbols = name
 479          self.signature = ""
 480          self._origin = "fake"
 481          self.text = "crashed"
 482  
 483      def __repr__(self):
 484          return '<Fake completion object jedi has crashed>'
 485  
 486  
 487  _JediCompletionLike = Union["jedi.api.Completion", _FakeJediCompletion]
 488  
 489  
 490  class Completion:
 491      """
 492      Completion object used and returned by IPython completers.
 493  
 494      .. warning::
 495  
 496          Unstable
 497  
 498          This function is unstable, API may change without warning.
 499          It will also raise unless use in proper context manager.
 500  
 501      This act as a middle ground :any:`Completion` object between the
 502      :any:`jedi.api.classes.Completion` object and the Prompt Toolkit completion
 503      object. While Jedi need a lot of information about evaluator and how the
 504      code should be ran/inspected, PromptToolkit (and other frontend) mostly
 505      need user facing information.
 506  
 507      - Which range should be replaced replaced by what.
 508      - Some metadata (like completion type), or meta information to displayed to
 509        the use user.
 510  
 511      For debugging purpose we can also store the origin of the completion (``jedi``,
 512      ``IPython.python_matches``, ``IPython.magics_matches``...).
 513      """
 514  
 515      __slots__ = ['start', 'end', 'text', 'type', 'signature', '_origin']
 516  
 517      def __init__(
 518          self,
 519          start: int,
 520          end: int,
 521          text: str,
 522          *,
 523          type: Optional[str] = None,
 524          _origin="",
 525          signature="",
 526      ) -> None:
 527          warnings.warn(
 528              "``Completion`` is a provisional API (as of IPython 6.0). "
 529              "It may change without warnings. "
 530              "Use in corresponding context manager.",
 531              category=ProvisionalCompleterWarning,
 532              stacklevel=2,
 533          )
 534  
 535          self.start = start
 536          self.end = end
 537          self.text = text
 538          self.type = type
 539          self.signature = signature
 540          self._origin = _origin
 541  
 542      def __repr__(self):
 543          return '<Completion start=%s end=%s text=%r type=%r, signature=%r,>' % \
 544                  (self.start, self.end, self.text, self.type or '?', self.signature or '?')
 545  
 546      def __eq__(self, other) -> bool:
 547          """
 548          Equality and hash do not hash the type (as some completer may not be
 549          able to infer the type), but are use to (partially) de-duplicate
 550          completion.
 551  
 552          Completely de-duplicating completion is a bit tricker that just
 553          comparing as it depends on surrounding text, which Completions are not
 554          aware of.
 555          """
 556          return self.start == other.start and \
 557              self.end == other.end and \
 558              self.text == other.text
 559  
 560      def __hash__(self):
 561          return hash((self.start, self.end, self.text))
 562  
 563  
 564  class SimpleCompletion:
 565      """Completion item to be included in the dictionary returned by new-style Matcher (API v2).
 566  
 567      .. warning::
 568  
 569          Provisional
 570  
 571          This class is used to describe the currently supported attributes of
 572          simple completion items, and any additional implementation details
 573          should not be relied on. Additional attributes may be included in
 574          future versions, and meaning of text disambiguated from the current
 575          dual meaning of "text to insert" and "text to used as a label".
 576      """
 577  
 578      __slots__ = ["text", "type"]
 579  
 580      def __init__(self, text: str, *, type: Optional[str] = None):
 581          self.text = text
 582          self.type = type
 583  
 584      def __repr__(self):
 585          return f"<SimpleCompletion text={self.text!r} type={self.type!r}>"
 586  
 587  
 588  class _MatcherResultBase(TypedDict):
 589      """Definition of dictionary to be returned by new-style Matcher (API v2)."""
 590  
 591      #: Suffix of the provided ``CompletionContext.token``, if not given defaults to full token.
 592      matched_fragment: NotRequired[str]
 593  
 594      #: Whether to suppress results from all other matchers (True), some
 595      #: matchers (set of identifiers) or none (False); default is False.
 596      suppress: NotRequired[Union[bool, Set[str]]]
 597  
 598      #: Identifiers of matchers which should NOT be suppressed when this matcher
 599      #: requests to suppress all other matchers; defaults to an empty set.
 600      do_not_suppress: NotRequired[Set[str]]
 601  
 602      #: Are completions already ordered and should be left as-is? default is False.
 603      ordered: NotRequired[bool]
 604  
 605  
 606  @sphinx_options(show_inherited_members=True, exclude_inherited_from=["dict"])
 607  class SimpleMatcherResult(_MatcherResultBase, TypedDict):
 608      """Result of new-style completion matcher."""
 609  
 610      # note: TypedDict is added again to the inheritance chain
 611      # in order to get __orig_bases__ for documentation
 612  
 613      #: List of candidate completions
 614      completions: Sequence[SimpleCompletion] | Iterator[SimpleCompletion]
 615  
 616  
 617  class _JediMatcherResult(_MatcherResultBase):
 618      """Matching result returned by Jedi (will be processed differently)"""
 619  
 620      #: list of candidate completions
 621      completions: Iterator[_JediCompletionLike]
 622  
 623  
 624  AnyMatcherCompletion = Union[_JediCompletionLike, SimpleCompletion]
 625  AnyCompletion = TypeVar("AnyCompletion", AnyMatcherCompletion, Completion)
 626  
 627  
 628  @dataclass
 629  class CompletionContext:
 630      """Completion context provided as an argument to matchers in the Matcher API v2."""
 631  
 632      # rationale: many legacy matchers relied on completer state (`self.text_until_cursor`)
 633      # which was not explicitly visible as an argument of the matcher, making any refactor
 634      # prone to errors; by explicitly passing `cursor_position` we can decouple the matchers
 635      # from the completer, and make substituting them in sub-classes easier.
 636  
 637      #: Relevant fragment of code directly preceding the cursor.
 638      #: The extraction of token is implemented via splitter heuristic
 639      #: (following readline behaviour for legacy reasons), which is user configurable
 640      #: (by switching the greedy mode).
 641      token: str
 642  
 643      #: The full available content of the editor or buffer
 644      full_text: str
 645  
 646      #: Cursor position in the line (the same for ``full_text`` and ``text``).
 647      cursor_position: int
 648  
 649      #: Cursor line in ``full_text``.
 650      cursor_line: int
 651  
 652      #: The maximum number of completions that will be used downstream.
 653      #: Matchers can use this information to abort early.
 654      #: The built-in Jedi matcher is currently excepted from this limit.
 655      # If not given, return all possible completions.
 656      limit: Optional[int]
 657  
 658      @cached_property
 659      def text_until_cursor(self) -> str:
 660          return self.line_with_cursor[: self.cursor_position]
 661  
 662      @cached_property
 663      def line_with_cursor(self) -> str:
 664          return self.full_text.split("\n")[self.cursor_line]
 665  
 666  
 667  #: Matcher results for API v2.
 668  MatcherResult = Union[SimpleMatcherResult, _JediMatcherResult]
 669  
 670  
 671  class _MatcherAPIv1Base(Protocol):
 672      def __call__(self, text: str) -> List[str]:
 673          """Call signature."""
 674          ...
 675  
 676      #: Used to construct the default matcher identifier
 677      __qualname__: str
 678  
 679  
 680  class _MatcherAPIv1Total(_MatcherAPIv1Base, Protocol):
 681      #: API version
 682      matcher_api_version: Optional[Literal[1]]
 683  
 684      def __call__(self, text: str) -> List[str]:
 685          """Call signature."""
 686          ...
 687  
 688  
 689  #: Protocol describing Matcher API v1.
 690  MatcherAPIv1: TypeAlias = Union[_MatcherAPIv1Base, _MatcherAPIv1Total]
 691  
 692  
 693  class MatcherAPIv2(Protocol):
 694      """Protocol describing Matcher API v2."""
 695  
 696      #: API version
 697      matcher_api_version: Literal[2] = 2
 698  
 699      def __call__(self, context: CompletionContext) -> MatcherResult:
 700          """Call signature."""
 701          ...
 702  
 703      #: Used to construct the default matcher identifier
 704      __qualname__: str
 705  
 706  
 707  Matcher: TypeAlias = Union[MatcherAPIv1, MatcherAPIv2]
 708  
 709  
 710  def _is_matcher_v1(matcher: Matcher) -> TypeGuard[MatcherAPIv1]:
 711      api_version = _get_matcher_api_version(matcher)
 712      return api_version == 1
 713  
 714  
 715  def _is_matcher_v2(matcher: Matcher) -> TypeGuard[MatcherAPIv2]:
 716      api_version = _get_matcher_api_version(matcher)
 717      return api_version == 2
 718  
 719  
 720  def _is_sizable(value: Any) -> TypeGuard[Sized]:
 721      """Determines whether objects is sizable"""
 722      return hasattr(value, "__len__")
 723  
 724  
 725  def _is_iterator(value: Any) -> TypeGuard[Iterator]:
 726      """Determines whether objects is sizable"""
 727      return hasattr(value, "__next__")
 728  
 729  
 730  def has_any_completions(result: MatcherResult) -> bool:
 731      """Check if any result includes any completions."""
 732      completions = result["completions"]
 733      if _is_sizable(completions):
 734          return len(completions) != 0
 735      if _is_iterator(completions):
 736          try:
 737              old_iterator = completions
 738              first = next(old_iterator)
 739              result["completions"] = cast(
 740                  Iterator[SimpleCompletion],
 741                  itertools.chain([first], old_iterator),
 742              )
 743              return True
 744          except StopIteration:
 745              return False
 746      raise ValueError(
 747          "Completions returned by matcher need to be an Iterator or a Sizable"
 748      )
 749  
 750  
 751  def completion_matcher(
 752      *,
 753      priority: Optional[float] = None,
 754      identifier: Optional[str] = None,
 755      api_version: int = 1,
 756  ) -> Callable[[Matcher], Matcher]:
 757      """Adds attributes describing the matcher.
 758  
 759      Parameters
 760      ----------
 761      priority : Optional[float]
 762          The priority of the matcher, determines the order of execution of matchers.
 763          Higher priority means that the matcher will be executed first. Defaults to 0.
 764      identifier : Optional[str]
 765          identifier of the matcher allowing users to modify the behaviour via traitlets,
 766          and also used to for debugging (will be passed as ``origin`` with the completions).
 767  
 768          Defaults to matcher function's ``__qualname__`` (for example,
 769          ``IPCompleter.file_matcher`` for the built-in matched defined
 770          as a ``file_matcher`` method of the ``IPCompleter`` class).
 771      api_version: Optional[int]
 772          version of the Matcher API used by this matcher.
 773          Currently supported values are 1 and 2.
 774          Defaults to 1.
 775      """
 776  
 777      def wrapper(func: Matcher):
 778          func.matcher_priority = priority or 0  # type: ignore
 779          func.matcher_identifier = identifier or func.__qualname__  # type: ignore
 780          func.matcher_api_version = api_version  # type: ignore
 781          if TYPE_CHECKING:
 782              if api_version == 1:
 783                  func = cast(MatcherAPIv1, func)
 784              elif api_version == 2:
 785                  func = cast(MatcherAPIv2, func)
 786          return func
 787  
 788      return wrapper
 789  
 790  
 791  def _get_matcher_priority(matcher: Matcher):
 792      return getattr(matcher, "matcher_priority", 0)
 793  
 794  
 795  def _get_matcher_id(matcher: Matcher):
 796      return getattr(matcher, "matcher_identifier", matcher.__qualname__)
 797  
 798  
 799  def _get_matcher_api_version(matcher):
 800      return getattr(matcher, "matcher_api_version", 1)
 801  
 802  
 803  context_matcher = partial(completion_matcher, api_version=2)
 804  
 805  
 806  _IC = Iterable[Completion]
 807  
 808  
 809  def _deduplicate_completions(text: str, completions: _IC)-> _IC:
 810      """
 811      Deduplicate a set of completions.
 812  
 813      .. warning::
 814  
 815          Unstable
 816  
 817          This function is unstable, API may change without warning.
 818  
 819      Parameters
 820      ----------
 821      text : str
 822          text that should be completed.
 823      completions : Iterator[Completion]
 824          iterator over the completions to deduplicate
 825  
 826      Yields
 827      ------
 828      `Completions` objects
 829      Completions coming from multiple sources, may be different but end up having
 830      the same effect when applied to ``text``. If this is the case, this will
 831      consider completions as equal and only emit the first encountered.
 832      Not folded in `completions()` yet for debugging purpose, and to detect when
 833      the IPython completer does return things that Jedi does not, but should be
 834      at some point.
 835      """
 836      completions = list(completions)
 837      if not completions:
 838          return
 839  
 840      new_start = min(c.start for c in completions)
 841      new_end = max(c.end for c in completions)
 842  
 843      seen = set()
 844      for c in completions:
 845          new_text = text[new_start:c.start] + c.text + text[c.end:new_end]
 846          if new_text not in seen:
 847              yield c
 848              seen.add(new_text)
 849  
 850  
 851  def rectify_completions(text: str, completions: _IC, *, _debug: bool = False) -> _IC:
 852      """
 853      Rectify a set of completions to all have the same ``start`` and ``end``
 854  
 855      .. warning::
 856  
 857          Unstable
 858  
 859          This function is unstable, API may change without warning.
 860          It will also raise unless use in proper context manager.
 861  
 862      Parameters
 863      ----------
 864      text : str
 865          text that should be completed.
 866      completions : Iterator[Completion]
 867          iterator over the completions to rectify
 868      _debug : bool
 869          Log failed completion
 870  
 871      Notes
 872      -----
 873      :any:`jedi.api.classes.Completion` s returned by Jedi may not have the same start and end, though
 874      the Jupyter Protocol requires them to behave like so. This will readjust
 875      the completion to have the same ``start`` and ``end`` by padding both
 876      extremities with surrounding text.
 877  
 878      During stabilisation should support a ``_debug`` option to log which
 879      completion are return by the IPython completer and not found in Jedi in
 880      order to make upstream bug report.
 881      """
 882      warnings.warn("`rectify_completions` is a provisional API (as of IPython 6.0). "
 883                   "It may change without warnings. "
 884                   "Use in corresponding context manager.",
 885                    category=ProvisionalCompleterWarning, stacklevel=2)
 886  
 887      completions = list(completions)
 888      if not completions:
 889          return
 890      starts = (c.start for c in completions)
 891      ends = (c.end for c in completions)
 892  
 893      new_start = min(starts)
 894      new_end = max(ends)
 895  
 896      seen_jedi = set()
 897      seen_python_matches = set()
 898      for c in completions:
 899          new_text = text[new_start:c.start] + c.text + text[c.end:new_end]
 900          if c._origin == 'jedi':
 901              seen_jedi.add(new_text)
 902          elif c._origin == "IPCompleter.python_matcher":
 903              seen_python_matches.add(new_text)
 904          yield Completion(new_start, new_end, new_text, type=c.type, _origin=c._origin, signature=c.signature)
 905      diff = seen_python_matches.difference(seen_jedi)
 906      if diff and _debug:
 907          print('IPython.python matches have extras:', diff)
 908  
 909  
 910  if sys.platform == 'win32':
 911      DELIMS = ' \t\n`!@#$^&*()=+[{]}|;\'",<>?'
 912  else:
 913      DELIMS = ' \t\n`!@#$^&*()=+[{]}\\|;:\'",<>?'
 914  
 915  GREEDY_DELIMS = ' =\r\n'
 916  
 917  
 918  class CompletionSplitter(object):
 919      """An object to split an input line in a manner similar to readline.
 920  
 921      By having our own implementation, we can expose readline-like completion in
 922      a uniform manner to all frontends.  This object only needs to be given the
 923      line of text to be split and the cursor position on said line, and it
 924      returns the 'word' to be completed on at the cursor after splitting the
 925      entire line.
 926  
 927      What characters are used as splitting delimiters can be controlled by
 928      setting the ``delims`` attribute (this is a property that internally
 929      automatically builds the necessary regular expression)"""
 930  
 931      # Private interface
 932  
 933      # A string of delimiter characters.  The default value makes sense for
 934      # IPython's most typical usage patterns.
 935      _delims = DELIMS
 936  
 937      # The expression (a normal string) to be compiled into a regular expression
 938      # for actual splitting.  We store it as an attribute mostly for ease of
 939      # debugging, since this type of code can be so tricky to debug.
 940      _delim_expr = None
 941  
 942      # The regular expression that does the actual splitting
 943      _delim_re = None
 944  
 945      def __init__(self, delims=None):
 946          delims = CompletionSplitter._delims if delims is None else delims
 947          self.delims = delims
 948  
 949      @property
 950      def delims(self):
 951          """Return the string of delimiter characters."""
 952          return self._delims
 953  
 954      @delims.setter
 955      def delims(self, delims):
 956          """Set the delimiters for line splitting."""
 957          expr = '[' + ''.join('\\'+ c for c in delims) + ']'
 958          self._delim_re = re.compile(expr)
 959          self._delims = delims
 960          self._delim_expr = expr
 961  
 962      def split_line(self, line, cursor_pos=None):
 963          """Split a line of text with a cursor at the given position.
 964          """
 965          cut_line = line if cursor_pos is None else line[:cursor_pos]
 966          return self._delim_re.split(cut_line)[-1]
 967  
 968  
 969  
 970  class Completer(Configurable):
 971  
 972      greedy = Bool(
 973          False,
 974          help="""Activate greedy completion.
 975  
 976          .. deprecated:: 8.8
 977              Use :std:configtrait:`Completer.evaluation` and :std:configtrait:`Completer.auto_close_dict_keys` instead.
 978  
 979          When enabled in IPython 8.8 or newer, changes configuration as follows:
 980  
 981          - ``Completer.evaluation = 'unsafe'``
 982          - ``Completer.auto_close_dict_keys = True``
 983          """,
 984      ).tag(config=True)
 985  
 986      evaluation = Enum(
 987          ("forbidden", "minimal", "limited", "unsafe", "dangerous"),
 988          default_value="limited",
 989          help="""Policy for code evaluation under completion.
 990  
 991          Successive options allow to enable more eager evaluation for better
 992          completion suggestions, including for nested dictionaries, nested lists,
 993          or even results of function calls.
 994          Setting ``unsafe`` or higher can lead to evaluation of arbitrary user
 995          code on :kbd:`Tab` with potentially unwanted or dangerous side effects.
 996  
 997          Allowed values are:
 998  
 999          - ``forbidden``: no evaluation of code is permitted,
1000          - ``minimal``: evaluation of literals and access to built-in namespace;
1001            no item/attribute evaluationm no access to locals/globals,
1002            no evaluation of any operations or comparisons.
1003          - ``limited``: access to all namespaces, evaluation of hard-coded methods
1004            (for example: :any:`dict.keys`, :any:`object.__getattr__`,
1005            :any:`object.__getitem__`) on allow-listed objects (for example:
1006            :any:`dict`, :any:`list`, :any:`tuple`, ``pandas.Series``),
1007          - ``unsafe``: evaluation of all methods and function calls but not of
1008            syntax with side-effects like `del x`,
1009          - ``dangerous``: completely arbitrary evaluation.
1010          """,
1011      ).tag(config=True)
1012  
1013      use_jedi = Bool(default_value=JEDI_INSTALLED,
1014                      help="Experimental: Use Jedi to generate autocompletions. "
1015                      "Default to True if jedi is installed.").tag(config=True)
1016  
1017      jedi_compute_type_timeout = Int(default_value=400,
1018          help="""Experimental: restrict time (in milliseconds) during which Jedi can compute types.
1019          Set to 0 to stop computing types. Non-zero value lower than 100ms may hurt
1020          performance by preventing jedi to build its cache.
1021          """).tag(config=True)
1022  
1023      debug = Bool(default_value=False,
1024                   help='Enable debug for the Completer. Mostly print extra '
1025                        'information for experimental jedi integration.')\
1026                        .tag(config=True)
1027  
1028      backslash_combining_completions = Bool(True,
1029          help="Enable unicode completions, e.g. \\alpha<tab> . "
1030               "Includes completion of latex commands, unicode names, and expanding "
1031               "unicode characters back to latex commands.").tag(config=True)
1032  
1033      auto_close_dict_keys = Bool(
1034          False,
1035          help="""
1036          Enable auto-closing dictionary keys.
1037  
1038          When enabled string keys will be suffixed with a final quote
1039          (matching the opening quote), tuple keys will also receive a
1040          separating comma if needed, and keys which are final will
1041          receive a closing bracket (``]``).
1042          """,
1043      ).tag(config=True)
1044  
1045      def __init__(self, namespace=None, global_namespace=None, **kwargs):
1046          """Create a new completer for the command line.
1047  
1048          Completer(namespace=ns, global_namespace=ns2) -> completer instance.
1049  
1050          If unspecified, the default namespace where completions are performed
1051          is __main__ (technically, __main__.__dict__). Namespaces should be
1052          given as dictionaries.
1053  
1054          An optional second namespace can be given.  This allows the completer
1055          to handle cases where both the local and global scopes need to be
1056          distinguished.
1057          """
1058  
1059          # Don't bind to namespace quite yet, but flag whether the user wants a
1060          # specific namespace or to use __main__.__dict__. This will allow us
1061          # to bind to __main__.__dict__ at completion time, not now.
1062          if namespace is None:
1063              self.use_main_ns = True
1064          else:
1065              self.use_main_ns = False
1066              self.namespace = namespace
1067  
1068          # The global namespace, if given, can be bound directly
1069          if global_namespace is None:
1070              self.global_namespace = {}
1071          else:
1072              self.global_namespace = global_namespace
1073  
1074          self.custom_matchers = []
1075  
1076          super(Completer, self).__init__(**kwargs)
1077  
1078      def complete(self, text, state):
1079          """Return the next possible completion for 'text'.
1080  
1081          This is called successively with state == 0, 1, 2, ... until it
1082          returns None.  The completion should begin with 'text'.
1083  
1084          """
1085          if self.use_main_ns:
1086              self.namespace = __main__.__dict__
1087  
1088          if state == 0:
1089              if "." in text:
1090                  self.matches = self.attr_matches(text)
1091              else:
1092                  self.matches = self.global_matches(text)
1093          try:
1094              return self.matches[state]
1095          except IndexError:
1096              return None
1097  
1098      def global_matches(self, text):
1099          """Compute matches when text is a simple name.
1100  
1101          Return a list of all keywords, built-in functions and names currently
1102          defined in self.namespace or self.global_namespace that match.
1103  
1104          """
1105          matches = []
1106          match_append = matches.append
1107          n = len(text)
1108          for lst in [
1109              keyword.kwlist,
1110              builtin_mod.__dict__.keys(),
1111              list(self.namespace.keys()),
1112              list(self.global_namespace.keys()),
1113          ]:
1114              for word in lst:
1115                  if word[:n] == text and word != "__builtins__":
1116                      match_append(word)
1117  
1118          snake_case_re = re.compile(r"[^_]+(_[^_]+)+?\Z")
1119          for lst in [list(self.namespace.keys()), list(self.global_namespace.keys())]:
1120              shortened = {
1121                  "_".join([sub[0] for sub in word.split("_")]): word
1122                  for word in lst
1123                  if snake_case_re.match(word)
1124              }
1125              for word in shortened.keys():
1126                  if word[:n] == text and word != "__builtins__":
1127                      match_append(shortened[word])
1128          return matches
1129  
1130      def attr_matches(self, text):
1131          """Compute matches when text contains a dot.
1132  
1133          Assuming the text is of the form NAME.NAME....[NAME], and is
1134          evaluatable in self.namespace or self.global_namespace, it will be
1135          evaluated and its attributes (as revealed by dir()) are used as
1136          possible completions.  (For class instances, class members are
1137          also considered.)
1138  
1139          WARNING: this can still invoke arbitrary C code, if an object
1140          with a __getattr__ hook is evaluated.
1141  
1142          """
1143          return self._attr_matches(text)[0]
1144  
1145      # we simple attribute matching with normal identifiers.
1146      _ATTR_MATCH_RE = re.compile(r"(.+)\.(\w*)$")
1147  
1148      def _attr_matches(
1149          self, text: str, include_prefix: bool = True
1150      ) -> Tuple[Sequence[str], str]:
1151          m2 = self._ATTR_MATCH_RE.match(text)
1152          if not m2:
1153              return [], ""
1154          expr, attr = m2.group(1, 2)
1155  
1156          obj = self._evaluate_expr(expr)
1157  
1158          if obj is not_found:
1159              return [], ""
1160  
1161          if self.limit_to__all__ and hasattr(obj, '__all__'):
1162              words = get__all__entries(obj)
1163          else:
1164              words = dir2(obj)
1165  
1166          try:
1167              words = generics.complete_object(obj, words)
1168          except TryNext:
1169              pass
1170          except AssertionError:
1171              raise
1172          except Exception:
1173              # Silence errors from completion function
1174              pass
1175          # Build match list to return
1176          n = len(attr)
1177  
1178          # Note: ideally we would just return words here and the prefix
1179          # reconciliator would know that we intend to append to rather than
1180          # replace the input text; this requires refactoring to return range
1181          # which ought to be replaced (as does jedi).
1182          if include_prefix:
1183              tokens = _parse_tokens(expr)
1184              rev_tokens = reversed(tokens)
1185              skip_over = {tokenize.ENDMARKER, tokenize.NEWLINE}
1186              name_turn = True
1187  
1188              parts = []
1189              for token in rev_tokens:
1190                  if token.type in skip_over:
1191                      continue
1192                  if token.type == tokenize.NAME and name_turn:
1193                      parts.append(token.string)
1194                      name_turn = False
1195                  elif (
1196                      token.type == tokenize.OP and token.string == "." and not name_turn
1197                  ):
1198                      parts.append(token.string)
1199                      name_turn = True
1200                  else:
1201                      # short-circuit if not empty nor name token
1202                      break
1203  
1204              prefix_after_space = "".join(reversed(parts))
1205          else:
1206              prefix_after_space = ""
1207  
1208          return (
1209              ["%s.%s" % (prefix_after_space, w) for w in words if w[:n] == attr],
1210              "." + attr,
1211          )
1212  
1213      def _trim_expr(self, code: str) -> str:
1214          """
1215          Trim the code until it is a valid expression and not a tuple;
1216  
1217          return the trimmed expression for guarded_eval.
1218          """
1219          while code:
1220              code = code[1:]
1221              try:
1222                  res = ast.parse(code)
1223              except SyntaxError:
1224                  continue
1225  
1226              assert res is not None
1227              if len(res.body) != 1:
1228                  continue
1229              expr = res.body[0].value
1230              if isinstance(expr, ast.Tuple) and not code[-1] == ")":
1231                  # we skip implicit tuple, like when trimming `fun(a,b`<completion>
1232                  # as `a,b` would be a tuple, and we actually expect to get only `b`
1233                  continue
1234              return code
1235          return ""
1236  
1237      def _evaluate_expr(self, expr):
1238          obj = not_found
1239          done = False
1240          while not done and expr:
1241              try:
1242                  obj = guarded_eval(
1243                      expr,
1244                      EvaluationContext(
1245                          globals=self.global_namespace,
1246                          locals=self.namespace,
1247                          evaluation=self.evaluation,
1248                      ),
1249                  )
1250                  done = True
1251              except Exception as e:
1252                  if self.debug:
1253                      print("Evaluation exception", e)
1254                  # trim the expression to remove any invalid prefix
1255                  # e.g. user starts `(d[`, so we get `expr = '(d'`,
1256                  # where parenthesis is not closed.
1257                  # TODO: make this faster by reusing parts of the computation?
1258                  expr = self._trim_expr(expr)
1259          return obj
1260  
1261  def get__all__entries(obj):
1262      """returns the strings in the __all__ attribute"""
1263      try:
1264          words = getattr(obj, '__all__')
1265      except Exception:
1266          return []
1267  
1268      return [w for w in words if isinstance(w, str)]
1269  
1270  
1271  class _DictKeyState(enum.Flag):
1272      """Represent state of the key match in context of other possible matches.
1273  
1274      - given `d1 = {'a': 1}` completion on `d1['<tab>` will yield `{'a': END_OF_ITEM}` as there is no tuple.
1275      - given `d2 = {('a', 'b'): 1}`: `d2['a', '<tab>` will yield `{'b': END_OF_TUPLE}` as there is no tuple members to add beyond `'b'`.
1276      - given `d3 = {('a', 'b'): 1}`: `d3['<tab>` will yield `{'a': IN_TUPLE}` as `'a'` can be added.
1277      - given `d4 = {'a': 1, ('a', 'b'): 2}`: `d4['<tab>` will yield `{'a': END_OF_ITEM & END_OF_TUPLE}`
1278      """
1279  
1280      BASELINE = 0
1281      END_OF_ITEM = enum.auto()
1282      END_OF_TUPLE = enum.auto()
1283      IN_TUPLE = enum.auto()
1284  
1285  
1286  def _parse_tokens(c):
1287      """Parse tokens even if there is an error."""
1288      tokens = []
1289      token_generator = tokenize.generate_tokens(iter(c.splitlines()).__next__)
1290      while True:
1291          try:
1292              tokens.append(next(token_generator))
1293          except tokenize.TokenError:
1294              return tokens
1295          except StopIteration:
1296              return tokens
1297  
1298  
1299  def _match_number_in_dict_key_prefix(prefix: str) -> Union[str, None]:
1300      """Match any valid Python numeric literal in a prefix of dictionary keys.
1301  
1302      References:
1303      - https://docs.python.org/3/reference/lexical_analysis.html#numeric-literals
1304      - https://docs.python.org/3/library/tokenize.html
1305      """
1306      if prefix[-1].isspace():
1307          # if user typed a space we do not have anything to complete
1308          # even if there was a valid number token before
1309          return None
1310      tokens = _parse_tokens(prefix)
1311      rev_tokens = reversed(tokens)
1312      skip_over = {tokenize.ENDMARKER, tokenize.NEWLINE}
1313      number = None
1314      for token in rev_tokens:
1315          if token.type in skip_over:
1316              continue
1317          if number is None:
1318              if token.type == tokenize.NUMBER:
1319                  number = token.string
1320                  continue
1321              else:
1322                  # we did not match a number
1323                  return None
1324          if token.type == tokenize.OP:
1325              if token.string == ",":
1326                  break
1327              if token.string in {"+", "-"}:
1328                  number = token.string + number
1329          else:
1330              return None
1331      return number
1332  
1333  
1334  _INT_FORMATS = {
1335      "0b": bin,
1336      "0o": oct,
1337      "0x": hex,
1338  }
1339  
1340  
1341  def match_dict_keys(
1342      keys: List[Union[str, bytes, Tuple[Union[str, bytes], ...]]],
1343      prefix: str,
1344      delims: str,
1345      extra_prefix: Optional[Tuple[Union[str, bytes], ...]] = None,
1346  ) -> Tuple[str, int, Dict[str, _DictKeyState]]:
1347      """Used by dict_key_matches, matching the prefix to a list of keys
1348  
1349      Parameters
1350      ----------
1351      keys
1352          list of keys in dictionary currently being completed.
1353      prefix
1354          Part of the text already typed by the user. E.g. `mydict[b'fo`
1355      delims
1356          String of delimiters to consider when finding the current key.
1357      extra_prefix : optional
1358          Part of the text already typed in multi-key index cases. E.g. for
1359          `mydict['foo', "bar", 'b`, this would be `('foo', 'bar')`.
1360  
1361      Returns
1362      -------
1363      A tuple of three elements: ``quote``, ``token_start``, ``matched``, with
1364      ``quote`` being the quote that need to be used to close current string.
1365      ``token_start`` the position where the replacement should start occurring,
1366      ``matches`` a dictionary of replacement/completion keys on keys and values
1367          indicating whether the state.
1368      """
1369      prefix_tuple = extra_prefix if extra_prefix else ()
1370  
1371      prefix_tuple_size = sum(
1372          [
1373              # for pandas, do not count slices as taking space
1374              not isinstance(k, slice)
1375              for k in prefix_tuple
1376          ]
1377      )
1378      text_serializable_types = (str, bytes, int, float, slice)
1379  
1380      def filter_prefix_tuple(key):
1381          # Reject too short keys
1382          if len(key) <= prefix_tuple_size:
1383              return False
1384          # Reject keys which cannot be serialised to text
1385          for k in key:
1386              if not isinstance(k, text_serializable_types):
1387                  return False
1388          # Reject keys that do not match the prefix
1389          for k, pt in zip(key, prefix_tuple):
1390              if k != pt and not isinstance(pt, slice):
1391                  return False
1392          # All checks passed!
1393          return True
1394  
1395      filtered_key_is_final: Dict[Union[str, bytes, int, float], _DictKeyState] = (
1396          defaultdict(lambda: _DictKeyState.BASELINE)
1397      )
1398  
1399      for k in keys:
1400          # If at least one of the matches is not final, mark as undetermined.
1401          # This can happen with `d = {111: 'b', (111, 222): 'a'}` where
1402          # `111` appears final on first match but is not final on the second.
1403  
1404          if isinstance(k, tuple):
1405              if filter_prefix_tuple(k):
1406                  key_fragment = k[prefix_tuple_size]
1407                  filtered_key_is_final[key_fragment] |= (
1408                      _DictKeyState.END_OF_TUPLE
1409                      if len(k) == prefix_tuple_size + 1
1410                      else _DictKeyState.IN_TUPLE
1411                  )
1412          elif prefix_tuple_size > 0:
1413              # we are completing a tuple but this key is not a tuple,
1414              # so we should ignore it
1415              pass
1416          else:
1417              if isinstance(k, text_serializable_types):
1418                  filtered_key_is_final[k] |= _DictKeyState.END_OF_ITEM
1419  
1420      filtered_keys = filtered_key_is_final.keys()
1421  
1422      if not prefix:
1423          return "", 0, {repr(k): v for k, v in filtered_key_is_final.items()}
1424  
1425      quote_match = re.search("(?:\"|')", prefix)
1426      is_user_prefix_numeric = False
1427  
1428      if quote_match:
1429          quote = quote_match.group()
1430          valid_prefix = prefix + quote
1431          try:
1432              prefix_str = literal_eval(valid_prefix)
1433          except Exception:
1434              return "", 0, {}
1435      else:
1436          # If it does not look like a string, let's assume
1437          # we are dealing with a number or variable.
1438          number_match = _match_number_in_dict_key_prefix(prefix)
1439  
1440          # We do not want the key matcher to suggest variable names so we yield:
1441          if number_match is None:
1442              # The alternative would be to assume that user forgort the quote
1443              # and if the substring matches, suggest adding it at the start.
1444              return "", 0, {}
1445  
1446          prefix_str = number_match
1447          is_user_prefix_numeric = True
1448          quote = ""
1449  
1450      pattern = '[^' + ''.join('\\' + c for c in delims) + ']*$'
1451      token_match = re.search(pattern, prefix, re.UNICODE)
1452      assert token_match is not None # silence mypy
1453      token_start = token_match.start()
1454      token_prefix = token_match.group()
1455  
1456      matched: Dict[str, _DictKeyState] = {}
1457  
1458      str_key: Union[str, bytes]
1459  
1460      for key in filtered_keys:
1461          if isinstance(key, (int, float)):
1462              # User typed a number but this key is not a number.
1463              if not is_user_prefix_numeric:
1464                  continue
1465              str_key = str(key)
1466              if isinstance(key, int):
1467                  int_base = prefix_str[:2].lower()
1468                  # if user typed integer using binary/oct/hex notation:
1469                  if int_base in _INT_FORMATS:
1470                      int_format = _INT_FORMATS[int_base]
1471                      str_key = int_format(key)
1472          else:
1473              # User typed a string but this key is a number.
1474              if is_user_prefix_numeric:
1475                  continue
1476              str_key = key
1477          try:
1478              if not str_key.startswith(prefix_str):
1479                  continue
1480          except (AttributeError, TypeError, UnicodeError):
1481              # Python 3+ TypeError on b'a'.startswith('a') or vice-versa
1482              continue
1483  
1484          # reformat remainder of key to begin with prefix
1485          rem = str_key[len(prefix_str) :]
1486          # force repr wrapped in '
1487          rem_repr = repr(rem + '"') if isinstance(rem, str) else repr(rem + b'"')
1488          rem_repr = rem_repr[1 + rem_repr.index("'"):-2]
1489          if quote == '"':
1490              # The entered prefix is quoted with ",
1491              # but the match is quoted with '.
1492              # A contained " hence needs escaping for comparison:
1493              rem_repr = rem_repr.replace('"', '\\"')
1494  
1495          # then reinsert prefix from start of token
1496          match = "%s%s" % (token_prefix, rem_repr)
1497  
1498          matched[match] = filtered_key_is_final[key]
1499      return quote, token_start, matched
1500  
1501  
1502  def cursor_to_position(text:str, line:int, column:int)->int:
1503      """
1504      Convert the (line,column) position of the cursor in text to an offset in a
1505      string.
1506  
1507      Parameters
1508      ----------
1509      text : str
1510          The text in which to calculate the cursor offset
1511      line : int
1512          Line of the cursor; 0-indexed
1513      column : int
1514          Column of the cursor 0-indexed
1515  
1516      Returns
1517      -------
1518      Position of the cursor in ``text``, 0-indexed.
1519  
1520      See Also
1521      --------
1522      position_to_cursor : reciprocal of this function
1523  
1524      """
1525      lines = text.split('\n')
1526      assert line <= len(lines), '{} <= {}'.format(str(line), str(len(lines)))
1527  
1528      return sum(len(line) + 1 for line in lines[:line]) + column
1529  
1530  def position_to_cursor(text:str, offset:int)->Tuple[int, int]:
1531      """
1532      Convert the position of the cursor in text (0 indexed) to a line
1533      number(0-indexed) and a column number (0-indexed) pair
1534  
1535      Position should be a valid position in ``text``.
1536  
1537      Parameters
1538      ----------
1539      text : str
1540          The text in which to calculate the cursor offset
1541      offset : int
1542          Position of the cursor in ``text``, 0-indexed.
1543  
1544      Returns
1545      -------
1546      (line, column) : (int, int)
1547          Line of the cursor; 0-indexed, column of the cursor 0-indexed
1548  
1549      See Also
1550      --------
1551      cursor_to_position : reciprocal of this function
1552  
1553      """
1554  
1555      assert 0 <= offset <= len(text) , "0 <= %s <= %s" % (offset , len(text))
1556  
1557      before = text[:offset]
1558      blines = before.split('\n')  # ! splitnes trim trailing \n
1559      line = before.count('\n')
1560      col = len(blines[-1])
1561      return line, col
1562  
1563  
1564  def _safe_isinstance(obj, module, class_name, *attrs):
1565      """Checks if obj is an instance of module.class_name if loaded
1566      """
1567      if module in sys.modules:
1568          m = sys.modules[module]
1569          for attr in [class_name, *attrs]:
1570              m = getattr(m, attr)
1571          return isinstance(obj, m)
1572  
1573  
1574  @context_matcher()
1575  def back_unicode_name_matcher(context: CompletionContext):
1576      """Match Unicode characters back to Unicode name
1577  
1578      Same as :any:`back_unicode_name_matches`, but adopted to new Matcher API.
1579      """
1580      fragment, matches = back_unicode_name_matches(context.text_until_cursor)
1581      return _convert_matcher_v1_result_to_v2(
1582          matches, type="unicode", fragment=fragment, suppress_if_matches=True
1583      )
1584  
1585  
1586  def back_unicode_name_matches(text: str) -> Tuple[str, Sequence[str]]:
1587      """Match Unicode characters back to Unicode name
1588  
1589      This does  ``☃`` -> ``\\snowman``
1590  
1591      Note that snowman is not a valid python3 combining character but will be expanded.
1592      Though it will not recombine back to the snowman character by the completion machinery.
1593  
1594      This will not either back-complete standard sequences like \\n, \\b ...
1595  
1596      .. deprecated:: 8.6
1597          You can use :meth:`back_unicode_name_matcher` instead.
1598  
1599      Returns
1600      =======
1601  
1602      Return a tuple with two elements:
1603  
1604      - The Unicode character that was matched (preceded with a backslash), or
1605          empty string,
1606      - a sequence (of 1), name for the match Unicode character, preceded by
1607          backslash, or empty if no match.
1608      """
1609      if len(text)<2:
1610          return '', ()
1611      maybe_slash = text[-2]
1612      if maybe_slash != '\\':
1613          return '', ()
1614  
1615      char = text[-1]
1616      # no expand on quote for completion in strings.
1617      # nor backcomplete standard ascii keys
1618      if char in string.ascii_letters or char in ('"',"'"):
1619          return '', ()
1620      try :
1621          unic = unicodedata.name(char)
1622          return '\\'+char,('\\'+unic,)
1623      except KeyError:
1624          pass
1625      return '', ()
1626  
1627  
1628  @context_matcher()
1629  def back_latex_name_matcher(context: CompletionContext):
1630      """Match latex characters back to unicode name
1631  
1632      Same as :any:`back_latex_name_matches`, but adopted to new Matcher API.
1633      """
1634      fragment, matches = back_latex_name_matches(context.text_until_cursor)
1635      return _convert_matcher_v1_result_to_v2(
1636          matches, type="latex", fragment=fragment, suppress_if_matches=True
1637      )
1638  
1639  
1640  def back_latex_name_matches(text: str) -> Tuple[str, Sequence[str]]:
1641      """Match latex characters back to unicode name
1642  
1643      This does ``\\ℵ`` -> ``\\aleph``
1644  
1645      .. deprecated:: 8.6
1646          You can use :meth:`back_latex_name_matcher` instead.
1647      """
1648      if len(text)<2:
1649          return '', ()
1650      maybe_slash = text[-2]
1651      if maybe_slash != '\\':
1652          return '', ()
1653  
1654  
1655      char = text[-1]
1656      # no expand on quote for completion in strings.
1657      # nor backcomplete standard ascii keys
1658      if char in string.ascii_letters or char in ('"',"'"):
1659          return '', ()
1660      try :
1661          latex = reverse_latex_symbol[char]
1662          # '\\' replace the \ as well
1663          return '\\'+char,[latex]
1664      except KeyError:
1665          pass
1666      return '', ()
1667  
1668  
1669  def _formatparamchildren(parameter) -> str:
1670      """
1671      Get parameter name and value from Jedi Private API
1672  
1673      Jedi does not expose a simple way to get `param=value` from its API.
1674  
1675      Parameters
1676      ----------
1677      parameter
1678          Jedi's function `Param`
1679  
1680      Returns
1681      -------
1682      A string like 'a', 'b=1', '*args', '**kwargs'
1683  
1684      """
1685      description = parameter.description
1686      if not description.startswith('param '):
1687          raise ValueError('Jedi function parameter description have change format.'
1688                           'Expected "param ...", found %r".' % description)
1689      return description[6:]
1690  
1691  def _make_signature(completion)-> str:
1692      """
1693      Make the signature from a jedi completion
1694  
1695      Parameters
1696      ----------
1697      completion : jedi.Completion
1698          object does not complete a function type
1699  
1700      Returns
1701      -------
1702      a string consisting of the function signature, with the parenthesis but
1703      without the function name. example:
1704      `(a, *args, b=1, **kwargs)`
1705  
1706      """
1707  
1708      # it looks like this might work on jedi 0.17
1709      if hasattr(completion, 'get_signatures'):
1710          signatures = completion.get_signatures()
1711          if not signatures:
1712              return  '(?)'
1713  
1714          c0 = completion.get_signatures()[0]
1715          return '('+c0.to_string().split('(', maxsplit=1)[1]
1716  
1717      return '(%s)'% ', '.join([f for f in (_formatparamchildren(p) for signature in completion.get_signatures()
1718                                            for p in signature.defined_names()) if f])
1719  
1720  
1721  _CompleteResult = Dict[str, MatcherResult]
1722  
1723  
1724  DICT_MATCHER_REGEX = re.compile(
1725      r"""(?x)
1726  (  # match dict-referring - or any get item object - expression
1727      .+
1728  )
1729  \[   # open bracket
1730  \s*  # and optional whitespace
1731  # Capture any number of serializable objects (e.g. "a", "b", 'c')
1732  # and slices
1733  ((?:(?:
1734      (?: # closed string
1735          [uUbB]?  # string prefix (r not handled)
1736          (?:
1737              '(?:[^']|(?<!\\)\\')*'
1738          |
1739              "(?:[^"]|(?<!\\)\\")*"
1740          )
1741      )
1742      |
1743          # capture integers and slices
1744          (?:[-+]?\d+)?(?::(?:[-+]?\d+)?){0,2}
1745      |
1746          # integer in bin/hex/oct notation
1747          0[bBxXoO]_?(?:\w|\d)+
1748      )
1749      \s*,\s*
1750  )*)
1751  ((?:
1752      (?: # unclosed string
1753          [uUbB]?  # string prefix (r not handled)
1754          (?:
1755              '(?:[^']|(?<!\\)\\')*
1756              |
1757              "(?:[^"]|(?<!\\)\\")*
1758          )
1759      )
1760      |
1761          # unfinished integer
1762          (?:[-+]?\d+)
1763      |
1764          # integer in bin/hex/oct notation
1765          0[bBxXoO]_?(?:\w|\d)+
1766      )
1767  )?
1768  $
1769  """
1770  )
1771  
1772  
1773  def _convert_matcher_v1_result_to_v2(
1774      matches: Sequence[str],
1775      type: str,
1776      fragment: Optional[str] = None,
1777      suppress_if_matches: bool = False,
1778  ) -> SimpleMatcherResult:
1779      """Utility to help with transition"""
1780      result = {
1781          "completions": [SimpleCompletion(text=match, type=type) for match in matches],
1782          "suppress": (True if matches else False) if suppress_if_matches else False,
1783      }
1784      if fragment is not None:
1785          result["matched_fragment"] = fragment
1786      return cast(SimpleMatcherResult, result)
1787  
1788  
1789  class IPCompleter(Completer):
1790      """Extension of the completer class with IPython-specific features"""
1791  
1792      @observe('greedy')
1793      def _greedy_changed(self, change):
1794          """update the splitter and readline delims when greedy is changed"""
1795          if change["new"]:
1796              self.evaluation = "unsafe"
1797              self.auto_close_dict_keys = True
1798              self.splitter.delims = GREEDY_DELIMS
1799          else:
1800              self.evaluation = "limited"
1801              self.auto_close_dict_keys = False
1802              self.splitter.delims = DELIMS
1803  
1804      dict_keys_only = Bool(
1805          False,
1806          help="""
1807          Whether to show dict key matches only.
1808  
1809          (disables all matchers except for `IPCompleter.dict_key_matcher`).
1810          """,
1811      )
1812  
1813      suppress_competing_matchers = UnionTrait(
1814          [Bool(allow_none=True), DictTrait(Bool(None, allow_none=True))],
1815          default_value=None,
1816          help="""
1817          Whether to suppress completions from other *Matchers*.
1818  
1819          When set to ``None`` (default) the matchers will attempt to auto-detect
1820          whether suppression of other matchers is desirable. For example, at
1821          the beginning of a line followed by `%` we expect a magic completion
1822          to be the only applicable option, and after ``my_dict['`` we usually
1823          expect a completion with an existing dictionary key.
1824  
1825          If you want to disable this heuristic and see completions from all matchers,
1826          set ``IPCompleter.suppress_competing_matchers = False``.
1827          To disable the heuristic for specific matchers provide a dictionary mapping:
1828          ``IPCompleter.suppress_competing_matchers = {'IPCompleter.dict_key_matcher': False}``.
1829  
1830          Set ``IPCompleter.suppress_competing_matchers = True`` to limit
1831          completions to the set of matchers with the highest priority;
1832          this is equivalent to ``IPCompleter.merge_completions`` and
1833          can be beneficial for performance, but will sometimes omit relevant
1834          candidates from matchers further down the priority list.
1835          """,
1836      ).tag(config=True)
1837  
1838      merge_completions = Bool(
1839          True,
1840          help="""Whether to merge completion results into a single list
1841  
1842          If False, only the completion results from the first non-empty
1843          completer will be returned.
1844  
1845          As of version 8.6.0, setting the value to ``False`` is an alias for:
1846          ``IPCompleter.suppress_competing_matchers = True.``.
1847          """,
1848      ).tag(config=True)
1849  
1850      disable_matchers = ListTrait(
1851          Unicode(),
1852          help="""List of matchers to disable.
1853  
1854          The list should contain matcher identifiers (see :any:`completion_matcher`).
1855          """,
1856      ).tag(config=True)
1857  
1858      omit__names = Enum(
1859          (0, 1, 2),
1860          default_value=2,
1861          help="""Instruct the completer to omit private method names
1862  
1863          Specifically, when completing on ``object.<tab>``.
1864  
1865          When 2 [default]: all names that start with '_' will be excluded.
1866  
1867          When 1: all 'magic' names (``__foo__``) will be excluded.
1868  
1869          When 0: nothing will be excluded.
1870          """
1871      ).tag(config=True)
1872      limit_to__all__ = Bool(False,
1873          help="""
1874          DEPRECATED as of version 5.0.
1875  
1876          Instruct the completer to use __all__ for the completion
1877  
1878          Specifically, when completing on ``object.<tab>``.
1879  
1880          When True: only those names in obj.__all__ will be included.
1881  
1882          When False [default]: the __all__ attribute is ignored
1883          """,
1884      ).tag(config=True)
1885  
1886      profile_completions = Bool(
1887          default_value=False,
1888          help="If True, emit profiling data for completion subsystem using cProfile."
1889      ).tag(config=True)
1890  
1891      profiler_output_dir = Unicode(
1892          default_value=".completion_profiles",
1893          help="Template for path at which to output profile data for completions."
1894      ).tag(config=True)
1895  
1896      @observe('limit_to__all__')
1897      def _limit_to_all_changed(self, change):
1898          warnings.warn('`IPython.core.IPCompleter.limit_to__all__` configuration '
1899              'value has been deprecated since IPython 5.0, will be made to have '
1900              'no effects and then removed in future version of IPython.',
1901              UserWarning)
1902  
1903      def __init__(
1904          self, shell=None, namespace=None, global_namespace=None, config=None, **kwargs
1905      ):
1906          """IPCompleter() -> completer
1907  
1908          Return a completer object.
1909  
1910          Parameters
1911          ----------
1912          shell
1913              a pointer to the ipython shell itself.  This is needed
1914              because this completer knows about magic functions, and those can
1915              only be accessed via the ipython instance.
1916          namespace : dict, optional
1917              an optional dict where completions are performed.
1918          global_namespace : dict, optional
1919              secondary optional dict for completions, to
1920              handle cases (such as IPython embedded inside functions) where
1921              both Python scopes are visible.
1922          config : Config
1923              traitlet's config object
1924          **kwargs
1925              passed to super class unmodified.
1926          """
1927  
1928          self.magic_escape = ESC_MAGIC
1929          self.splitter = CompletionSplitter()
1930  
1931          # _greedy_changed() depends on splitter and readline being defined:
1932          super().__init__(
1933              namespace=namespace,
1934              global_namespace=global_namespace,
1935              config=config,
1936              **kwargs,
1937          )
1938  
1939          # List where completion matches will be stored
1940          self.matches = []
1941          self.shell = shell
1942          # Regexp to split filenames with spaces in them
1943          self.space_name_re = re.compile(r'([^\\] )')
1944          # Hold a local ref. to glob.glob for speed
1945          self.glob = glob.glob
1946  
1947          # Determine if we are running on 'dumb' terminals, like (X)Emacs
1948          # buffers, to avoid completion problems.
1949          term = os.environ.get('TERM','xterm')
1950          self.dumb_terminal = term in ['dumb','emacs']
1951  
1952          # Special handling of backslashes needed in win32 platforms
1953          if sys.platform == "win32":
1954              self.clean_glob = self._clean_glob_win32
1955          else:
1956              self.clean_glob = self._clean_glob
1957  
1958          #regexp to parse docstring for function signature
1959          self.docstring_sig_re = re.compile(r'^[\w|\s.]+\(([^)]*)\).*')
1960          self.docstring_kwd_re = re.compile(r'[\s|\[]*(\w+)(?:\s*=\s*.*)')
1961          #use this if positional argument name is also needed
1962          #= re.compile(r'[\s|\[]*(\w+)(?:\s*=?\s*.*)')
1963  
1964          self.magic_arg_matchers = [
1965              self.magic_config_matcher,
1966              self.magic_color_matcher,
1967          ]
1968  
1969          # This is set externally by InteractiveShell
1970          self.custom_completers = None
1971  
1972          # This is a list of names of unicode characters that can be completed
1973          # into their corresponding unicode value. The list is large, so we
1974          # lazily initialize it on first use. Consuming code should access this
1975          # attribute through the `@unicode_names` property.
1976          self._unicode_names = None
1977  
1978          self._backslash_combining_matchers = [
1979              self.latex_name_matcher,
1980              self.unicode_name_matcher,
1981              back_latex_name_matcher,
1982              back_unicode_name_matcher,
1983              self.fwd_unicode_matcher,
1984          ]
1985  
1986          if not self.backslash_combining_completions:
1987              for matcher in self._backslash_combining_matchers:
1988                  self.disable_matchers.append(_get_matcher_id(matcher))
1989  
1990          if not self.merge_completions:
1991              self.suppress_competing_matchers = True
1992  
1993      @property
1994      def matchers(self) -> List[Matcher]:
1995          """All active matcher routines for completion"""
1996          if self.dict_keys_only:
1997              return [self.dict_key_matcher]
1998  
1999          if self.use_jedi:
2000              return [
2001                  *self.custom_matchers,
2002                  *self._backslash_combining_matchers,
2003                  *self.magic_arg_matchers,
2004                  self.custom_completer_matcher,
2005                  self.magic_matcher,
2006                  self._jedi_matcher,
2007                  self.dict_key_matcher,
2008                  self.file_matcher,
2009              ]
2010          else:
2011              return [
2012                  *self.custom_matchers,
2013                  *self._backslash_combining_matchers,
2014                  *self.magic_arg_matchers,
2015                  self.custom_completer_matcher,
2016                  self.dict_key_matcher,
2017                  self.magic_matcher,
2018                  self.python_matcher,
2019                  self.file_matcher,
2020                  self.python_func_kw_matcher,
2021              ]
2022  
2023      def all_completions(self, text:str) -> List[str]:
2024          """
2025          Wrapper around the completion methods for the benefit of emacs.
2026          """
2027          prefix = text.rpartition('.')[0]
2028          with provisionalcompleter():
2029              return ['.'.join([prefix, c.text]) if prefix and self.use_jedi else c.text
2030                      for c in self.completions(text, len(text))]
2031  
2032          return self.complete(text)[1]
2033  
2034      def _clean_glob(self, text:str):
2035          return self.glob("%s*" % text)
2036  
2037      def _clean_glob_win32(self, text:str):
2038          return [f.replace("\\","/")
2039                  for f in self.glob("%s*" % text)]
2040  
2041      @context_matcher()
2042      def file_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2043          """Same as :any:`file_matches`, but adopted to new Matcher API."""
2044          matches = self.file_matches(context.token)
2045          # TODO: add a heuristic for suppressing (e.g. if it has OS-specific delimiter,
2046          #  starts with `/home/`, `C:\`, etc)
2047          return _convert_matcher_v1_result_to_v2(matches, type="path")
2048  
2049      def file_matches(self, text: str) -> List[str]:
2050          """Match filenames, expanding ~USER type strings.
2051  
2052          Most of the seemingly convoluted logic in this completer is an
2053          attempt to handle filenames with spaces in them.  And yet it's not
2054          quite perfect, because Python's readline doesn't expose all of the
2055          GNU readline details needed for this to be done correctly.
2056  
2057          For a filename with a space in it, the printed completions will be
2058          only the parts after what's already been typed (instead of the
2059          full completions, as is normally done).  I don't think with the
2060          current (as of Python 2.3) Python readline it's possible to do
2061          better.
2062  
2063          .. deprecated:: 8.6
2064              You can use :meth:`file_matcher` instead.
2065          """
2066  
2067          # chars that require escaping with backslash - i.e. chars
2068          # that readline treats incorrectly as delimiters, but we
2069          # don't want to treat as delimiters in filename matching
2070          # when escaped with backslash
2071          if text.startswith('!'):
2072              text = text[1:]
2073              text_prefix = u'!'
2074          else:
2075              text_prefix = u''
2076  
2077          text_until_cursor = self.text_until_cursor
2078          # track strings with open quotes
2079          open_quotes = has_open_quotes(text_until_cursor)
2080  
2081          if '(' in text_until_cursor or '[' in text_until_cursor:
2082              lsplit = text
2083          else:
2084              try:
2085                  # arg_split ~ shlex.split, but with unicode bugs fixed by us
2086                  lsplit = arg_split(text_until_cursor)[-1]
2087              except ValueError:
2088                  # typically an unmatched ", or backslash without escaped char.
2089                  if open_quotes:
2090                      lsplit = text_until_cursor.split(open_quotes)[-1]
2091                  else:
2092                      return []
2093              except IndexError:
2094                  # tab pressed on empty line
2095                  lsplit = ""
2096  
2097          if not open_quotes and lsplit != protect_filename(lsplit):
2098              # if protectables are found, do matching on the whole escaped name
2099              has_protectables = True
2100              text0,text = text,lsplit
2101          else:
2102              has_protectables = False
2103              text = os.path.expanduser(text)
2104  
2105          if text == "":
2106              return [text_prefix + protect_filename(f) for f in self.glob("*")]
2107  
2108          # Compute the matches from the filesystem
2109          if sys.platform == 'win32':
2110              m0 = self.clean_glob(text)
2111          else:
2112              m0 = self.clean_glob(text.replace('\\', ''))
2113  
2114          if has_protectables:
2115              # If we had protectables, we need to revert our changes to the
2116              # beginning of filename so that we don't double-write the part
2117              # of the filename we have so far
2118              len_lsplit = len(lsplit)
2119              matches = [text_prefix + text0 +
2120                         protect_filename(f[len_lsplit:]) for f in m0]
2121          else:
2122              if open_quotes:
2123                  # if we have a string with an open quote, we don't need to
2124                  # protect the names beyond the quote (and we _shouldn't_, as
2125                  # it would cause bugs when the filesystem call is made).
2126                  matches = m0 if sys.platform == "win32" else\
2127                      [protect_filename(f, open_quotes) for f in m0]
2128              else:
2129                  matches = [text_prefix +
2130                             protect_filename(f) for f in m0]
2131  
2132          # Mark directories in input list by appending '/' to their names.
2133          return [x+'/' if os.path.isdir(x) else x for x in matches]
2134  
2135      @context_matcher()
2136      def magic_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2137          """Match magics."""
2138          text = context.token
2139          matches = self.magic_matches(text)
2140          result = _convert_matcher_v1_result_to_v2(matches, type="magic")
2141          is_magic_prefix = len(text) > 0 and text[0] == "%"
2142          result["suppress"] = is_magic_prefix and bool(result["completions"])
2143          return result
2144  
2145      def magic_matches(self, text: str) -> List[str]:
2146          """Match magics.
2147  
2148          .. deprecated:: 8.6
2149              You can use :meth:`magic_matcher` instead.
2150          """
2151          # Get all shell magics now rather than statically, so magics loaded at
2152          # runtime show up too.
2153          lsm = self.shell.magics_manager.lsmagic()
2154          line_magics = lsm['line']
2155          cell_magics = lsm['cell']
2156          pre = self.magic_escape
2157          pre2 = pre+pre
2158  
2159          explicit_magic = text.startswith(pre)
2160  
2161          # Completion logic:
2162          # - user gives %%: only do cell magics
2163          # - user gives %: do both line and cell magics
2164          # - no prefix: do both
2165          # In other words, line magics are skipped if the user gives %% explicitly
2166          #
2167          # We also exclude magics that match any currently visible names:
2168          # https://github.com/ipython/ipython/issues/4877, unless the user has
2169          # typed a %:
2170          # https://github.com/ipython/ipython/issues/10754
2171          bare_text = text.lstrip(pre)
2172          global_matches = self.global_matches(bare_text)
2173          if not explicit_magic:
2174              def matches(magic):
2175                  """
2176                  Filter magics, in particular remove magics that match
2177                  a name present in global namespace.
2178                  """
2179                  return ( magic.startswith(bare_text) and
2180                           magic not in global_matches )
2181          else:
2182              def matches(magic):
2183                  return magic.startswith(bare_text)
2184  
2185          comp = [ pre2+m for m in cell_magics if matches(m)]
2186          if not text.startswith(pre2):
2187              comp += [ pre+m for m in line_magics if matches(m)]
2188  
2189          return comp
2190  
2191      @context_matcher()
2192      def magic_config_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2193          """Match class names and attributes for %config magic."""
2194          # NOTE: uses `line_buffer` equivalent for compatibility
2195          matches = self.magic_config_matches(context.line_with_cursor)
2196          return _convert_matcher_v1_result_to_v2(matches, type="param")
2197  
2198      def magic_config_matches(self, text: str) -> List[str]:
2199          """Match class names and attributes for %config magic.
2200  
2201          .. deprecated:: 8.6
2202              You can use :meth:`magic_config_matcher` instead.
2203          """
2204          texts = text.strip().split()
2205  
2206          if len(texts) > 0 and (texts[0] == 'config' or texts[0] == '%config'):
2207              # get all configuration classes
2208              classes = sorted(set([ c for c in self.shell.configurables
2209                                     if c.__class__.class_traits(config=True)
2210                                     ]), key=lambda x: x.__class__.__name__)
2211              classnames = [ c.__class__.__name__ for c in classes ]
2212  
2213              # return all classnames if config or %config is given
2214              if len(texts) == 1:
2215                  return classnames
2216  
2217              # match classname
2218              classname_texts = texts[1].split('.')
2219              classname = classname_texts[0]
2220              classname_matches = [ c for c in classnames
2221                                    if c.startswith(classname) ]
2222  
2223              # return matched classes or the matched class with attributes
2224              if texts[1].find('.') < 0:
2225                  return classname_matches
2226              elif len(classname_matches) == 1 and \
2227                              classname_matches[0] == classname:
2228                  cls = classes[classnames.index(classname)].__class__
2229                  help = cls.class_get_help()
2230                  # strip leading '--' from cl-args:
2231                  help = re.sub(re.compile(r'^--', re.MULTILINE), '', help)
2232                  return [ attr.split('=')[0]
2233                           for attr in help.strip().splitlines()
2234                           if attr.startswith(texts[1]) ]
2235          return []
2236  
2237      @context_matcher()
2238      def magic_color_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2239          """Match color schemes for %colors magic."""
2240          # NOTE: uses `line_buffer` equivalent for compatibility
2241          matches = self.magic_color_matches(context.line_with_cursor)
2242          return _convert_matcher_v1_result_to_v2(matches, type="param")
2243  
2244      def magic_color_matches(self, text: str) -> List[str]:
2245          """Match color schemes for %colors magic.
2246  
2247          .. deprecated:: 8.6
2248              You can use :meth:`magic_color_matcher` instead.
2249          """
2250          texts = text.split()
2251          if text.endswith(' '):
2252              # .split() strips off the trailing whitespace. Add '' back
2253              # so that: '%colors ' -> ['%colors', '']
2254              texts.append('')
2255  
2256          if len(texts) == 2 and (texts[0] == 'colors' or texts[0] == '%colors'):
2257              prefix = texts[1]
2258              return [ color for color in InspectColors.keys()
2259                       if color.startswith(prefix) ]
2260          return []
2261  
2262      @context_matcher(identifier="IPCompleter.jedi_matcher")
2263      def _jedi_matcher(self, context: CompletionContext) -> _JediMatcherResult:
2264          matches = self._jedi_matches(
2265              cursor_column=context.cursor_position,
2266              cursor_line=context.cursor_line,
2267              text=context.full_text,
2268          )
2269          return {
2270              "completions": matches,
2271              # static analysis should not suppress other matchers
2272              "suppress": False,
2273          }
2274  
2275      def _jedi_matches(
2276          self, cursor_column: int, cursor_line: int, text: str
2277      ) -> Iterator[_JediCompletionLike]:
2278          """
2279          Return a list of :any:`jedi.api.Completion`\\s object from a ``text`` and
2280          cursor position.
2281  
2282          Parameters
2283          ----------
2284          cursor_column : int
2285              column position of the cursor in ``text``, 0-indexed.
2286          cursor_line : int
2287              line position of the cursor in ``text``, 0-indexed
2288          text : str
2289              text to complete
2290  
2291          Notes
2292          -----
2293          If ``IPCompleter.debug`` is ``True`` may return a :any:`_FakeJediCompletion`
2294          object containing a string with the Jedi debug information attached.
2295  
2296          .. deprecated:: 8.6
2297              You can use :meth:`_jedi_matcher` instead.
2298          """
2299          namespaces = [self.namespace]
2300          if self.global_namespace is not None:
2301              namespaces.append(self.global_namespace)
2302  
2303          completion_filter = lambda x:x
2304          offset = cursor_to_position(text, cursor_line, cursor_column)
2305          # filter output if we are completing for object members
2306          if offset:
2307              pre = text[offset-1]
2308              if pre == '.':
2309                  if self.omit__names == 2:
2310                      completion_filter = lambda c:not c.name.startswith('_')
2311                  elif self.omit__names == 1:
2312                      completion_filter = lambda c:not (c.name.startswith('__') and c.name.endswith('__'))
2313                  elif self.omit__names == 0:
2314                      completion_filter = lambda x:x
2315                  else:
2316                      raise ValueError("Don't understand self.omit__names == {}".format(self.omit__names))
2317  
2318          interpreter = jedi.Interpreter(text[:offset], namespaces)
2319          try_jedi = True
2320  
2321          try:
2322              # find the first token in the current tree -- if it is a ' or " then we are in a string
2323              completing_string = False
2324              try:
2325                  first_child = next(c for c in interpreter._get_module().tree_node.children if hasattr(c, 'value'))
2326              except StopIteration:
2327                  pass
2328              else:
2329                  # note the value may be ', ", or it may also be ''' or """, or
2330                  # in some cases, """what/you/typed..., but all of these are
2331                  # strings.
2332                  completing_string = len(first_child.value) > 0 and first_child.value[0] in {"'", '"'}
2333  
2334              # if we are in a string jedi is likely not the right candidate for
2335              # now. Skip it.
2336              try_jedi = not completing_string
2337          except Exception as e:
2338              # many of things can go wrong, we are using private API just don't crash.
2339              if self.debug:
2340                  print("Error detecting if completing a non-finished string :", e, '|')
2341  
2342          if not try_jedi:
2343              return iter([])
2344          try:
2345              return filter(completion_filter, interpreter.complete(column=cursor_column, line=cursor_line + 1))
2346          except Exception as e:
2347              if self.debug:
2348                  return iter(
2349                      [
2350                          _FakeJediCompletion(
2351                              'Oops Jedi has crashed, please report a bug with the following:\n"""\n%s\ns"""'
2352                              % (e)
2353                          )
2354                      ]
2355                  )
2356              else:
2357                  return iter([])
2358  
2359      class _CompletionContextType(enum.Enum):
2360          ATTRIBUTE = "attribute"  # For attribute completion
2361          GLOBAL = "global"  # For global completion
2362  
2363      def _determine_completion_context(self, line):
2364          """
2365          Determine whether the cursor is in an attribute or global completion context.
2366          """
2367          # Cursor in string/comment → GLOBAL.
2368          is_string, is_in_expression = self._is_in_string_or_comment(line)
2369          if is_string and not is_in_expression:
2370              return self._CompletionContextType.GLOBAL
2371  
2372          # If we're in a template string expression, handle specially
2373          if is_string and is_in_expression:
2374              # Extract the expression part - look for the last { that isn't closed
2375              expr_start = line.rfind("{")
2376              if expr_start >= 0:
2377                  # We're looking at the expression inside a template string
2378                  expr = line[expr_start + 1 :]
2379                  # Recursively determine the context of the expression
2380                  return self._determine_completion_context(expr)
2381  
2382          # Handle plain number literals - should be global context
2383          # Ex: 3. -42.14 but not 3.1.
2384          if re.search(r"(?<!\w)(?<!\d\.)([-+]?\d+\.(\d+)?)(?!\w)$", line):
2385              return self._CompletionContextType.GLOBAL
2386  
2387          # Handle all other attribute matches np.ran, d[0].k, (a,b).count
2388          chain_match = re.search(r".*(.+\.(?:[a-zA-Z]\w*)?)$", line)
2389          if chain_match:
2390              return self._CompletionContextType.ATTRIBUTE
2391  
2392          return self._CompletionContextType.GLOBAL
2393  
2394      def _is_in_string_or_comment(self, text):
2395          """
2396          Determine if the cursor is inside a string or comment.
2397          Returns (is_string, is_in_expression) tuple:
2398          - is_string: True if in any kind of string
2399          - is_in_expression: True if inside an f-string/t-string expression
2400          """
2401          in_single_quote = False
2402          in_double_quote = False
2403          in_triple_single = False
2404          in_triple_double = False
2405          in_template_string = False  # Covers both f-strings and t-strings
2406          in_expression = False  # For expressions in f/t-strings
2407          expression_depth = 0  # Track nested braces in expressions
2408          i = 0
2409  
2410          while i < len(text):
2411              # Check for f-string or t-string start
2412              if (
2413                  i + 1 < len(text)
2414                  and text[i] in ("f", "t")
2415                  and (text[i + 1] == '"' or text[i + 1] == "'")
2416                  and not (
2417                      in_single_quote
2418                      or in_double_quote
2419                      or in_triple_single
2420                      or in_triple_double
2421                  )
2422              ):
2423                  in_template_string = True
2424                  i += 1  # Skip the 'f' or 't'
2425  
2426              # Handle triple quotes
2427              if i + 2 < len(text):
2428                  if (
2429                      text[i : i + 3] == '"""'
2430                      and not in_single_quote
2431                      and not in_triple_single
2432                  ):
2433                      in_triple_double = not in_triple_double
2434                      if not in_triple_double:
2435                          in_template_string = False
2436                      i += 3
2437                      continue
2438                  if (
2439                      text[i : i + 3] == "'''"
2440                      and not in_double_quote
2441                      and not in_triple_double
2442                  ):
2443                      in_triple_single = not in_triple_single
2444                      if not in_triple_single:
2445                          in_template_string = False
2446                      i += 3
2447                      continue
2448  
2449              # Handle escapes
2450              if text[i] == "\\" and i + 1 < len(text):
2451                  i += 2
2452                  continue
2453  
2454              # Handle nested braces within f-strings
2455              if in_template_string:
2456                  # Special handling for consecutive opening braces
2457                  if i + 1 < len(text) and text[i : i + 2] == "{{":
2458                      i += 2
2459                      continue
2460  
2461                  # Detect start of an expression
2462                  if text[i] == "{":
2463                      # Only increment depth and mark as expression if not already in an expression
2464                      # or if we're at a top-level nested brace
2465                      if not in_expression or (in_expression and expression_depth == 0):
2466                          in_expression = True
2467                          expression_depth += 1
2468                      i += 1
2469                      continue
2470  
2471                  # Detect end of an expression
2472                  if text[i] == "}":
2473                      expression_depth -= 1
2474                      if expression_depth <= 0:
2475                          in_expression = False
2476                          expression_depth = 0
2477                      i += 1
2478                      continue
2479  
2480              in_triple_quote = in_triple_single or in_triple_double
2481  
2482              # Handle quotes - also reset template string when closing quotes are encountered
2483              if text[i] == '"' and not in_single_quote and not in_triple_quote:
2484                  in_double_quote = not in_double_quote
2485                  if not in_double_quote and not in_triple_quote:
2486                      in_template_string = False
2487              elif text[i] == "'" and not in_double_quote and not in_triple_quote:
2488                  in_single_quote = not in_single_quote
2489                  if not in_single_quote and not in_triple_quote:
2490                      in_template_string = False
2491  
2492              # Check for comment
2493              if text[i] == "#" and not (
2494                  in_single_quote or in_double_quote or in_triple_quote
2495              ):
2496                  return True, False
2497  
2498              i += 1
2499  
2500          is_string = (
2501              in_single_quote or in_double_quote or in_triple_single or in_triple_double
2502          )
2503  
2504          # Return tuple (is_string, is_in_expression)
2505          return (
2506              is_string or (in_template_string and not in_expression),
2507              in_expression and expression_depth > 0,
2508          )
2509  
2510      @context_matcher()
2511      def python_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2512          """Match attributes or global python names"""
2513          text = context.text_until_cursor
2514          completion_type = self._determine_completion_context(text)
2515          if completion_type == self._CompletionContextType.ATTRIBUTE:
2516              try:
2517                  matches, fragment = self._attr_matches(text, include_prefix=False)
2518                  if text.endswith(".") and self.omit__names:
2519                      if self.omit__names == 1:
2520                          # true if txt is _not_ a __ name, false otherwise:
2521                          no__name = lambda txt: re.match(r".*\.__.*?__", txt) is None
2522                      else:
2523                          # true if txt is _not_ a _ name, false otherwise:
2524                          no__name = (
2525                              lambda txt: re.match(r"\._.*?", txt[txt.rindex(".") :])
2526                              is None
2527                          )
2528                      matches = filter(no__name, matches)
2529                  return _convert_matcher_v1_result_to_v2(
2530                      matches, type="attribute", fragment=fragment
2531                  )
2532              except NameError:
2533                  # catches <undefined attributes>.<tab>
2534                  matches = []
2535                  return _convert_matcher_v1_result_to_v2(matches, type="attribute")
2536          else:
2537              matches = self.global_matches(context.token)
2538              # TODO: maybe distinguish between functions, modules and just "variables"
2539              return _convert_matcher_v1_result_to_v2(matches, type="variable")
2540  
2541      @completion_matcher(api_version=1)
2542      def python_matches(self, text: str) -> Iterable[str]:
2543          """Match attributes or global python names.
2544  
2545          .. deprecated:: 8.27
2546              You can use :meth:`python_matcher` instead."""
2547          if "." in text:
2548              try:
2549                  matches = self.attr_matches(text)
2550                  if text.endswith('.') and self.omit__names:
2551                      if self.omit__names == 1:
2552                          # true if txt is _not_ a __ name, false otherwise:
2553                          no__name = (lambda txt:
2554                                      re.match(r'.*\.__.*?__',txt) is None)
2555                      else:
2556                          # true if txt is _not_ a _ name, false otherwise:
2557                          no__name = (lambda txt:
2558                                      re.match(r'\._.*?',txt[txt.rindex('.'):]) is None)
2559                      matches = filter(no__name, matches)
2560              except NameError:
2561                  # catches <undefined attributes>.<tab>
2562                  matches = []
2563          else:
2564              matches = self.global_matches(text)
2565          return matches
2566  
2567      def _default_arguments_from_docstring(self, doc):
2568          """Parse the first line of docstring for call signature.
2569  
2570          Docstring should be of the form 'min(iterable[, key=func])\n'.
2571          It can also parse cython docstring of the form
2572          'Minuit.migrad(self, int ncall=10000, resume=True, int nsplit=1)'.
2573          """
2574          if doc is None:
2575              return []
2576  
2577          #care only the firstline
2578          line = doc.lstrip().splitlines()[0]
2579  
2580          #p = re.compile(r'^[\w|\s.]+\(([^)]*)\).*')
2581          #'min(iterable[, key=func])\n' -> 'iterable[, key=func]'
2582          sig = self.docstring_sig_re.search(line)
2583          if sig is None:
2584              return []
2585          # iterable[, key=func]' -> ['iterable[' ,' key=func]']
2586          sig = sig.groups()[0].split(',')
2587          ret = []
2588          for s in sig:
2589              #re.compile(r'[\s|\[]*(\w+)(?:\s*=\s*.*)')
2590              ret += self.docstring_kwd_re.findall(s)
2591          return ret
2592  
2593      def _default_arguments(self, obj):
2594          """Return the list of default arguments of obj if it is callable,
2595          or empty list otherwise."""
2596          call_obj = obj
2597          ret = []
2598          if inspect.isbuiltin(obj):
2599              pass
2600          elif not (inspect.isfunction(obj) or inspect.ismethod(obj)):
2601              if inspect.isclass(obj):
2602                  #for cython embedsignature=True the constructor docstring
2603                  #belongs to the object itself not __init__
2604                  ret += self._default_arguments_from_docstring(
2605                              getattr(obj, '__doc__', ''))
2606                  # for classes, check for __init__,__new__
2607                  call_obj = (getattr(obj, '__init__', None) or
2608                         getattr(obj, '__new__', None))
2609              # for all others, check if they are __call__able
2610              elif hasattr(obj, '__call__'):
2611                  call_obj = obj.__call__
2612          ret += self._default_arguments_from_docstring(
2613                   getattr(call_obj, '__doc__', ''))
2614  
2615          _keeps = (inspect.Parameter.KEYWORD_ONLY,
2616                    inspect.Parameter.POSITIONAL_OR_KEYWORD)
2617  
2618          try:
2619              sig = inspect.signature(obj)
2620              ret.extend(k for k, v in sig.parameters.items() if
2621                         v.kind in _keeps)
2622          except ValueError:
2623              pass
2624  
2625          return list(set(ret))
2626  
2627      @context_matcher()
2628      def python_func_kw_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2629          """Match named parameters (kwargs) of the last open function."""
2630          matches = self.python_func_kw_matches(context.token)
2631          return _convert_matcher_v1_result_to_v2(matches, type="param")
2632  
2633      def python_func_kw_matches(self, text):
2634          """Match named parameters (kwargs) of the last open function.
2635  
2636          .. deprecated:: 8.6
2637              You can use :meth:`python_func_kw_matcher` instead.
2638          """
2639  
2640          if "." in text: # a parameter cannot be dotted
2641              return []
2642          try: regexp = self.__funcParamsRegex
2643          except AttributeError:
2644              regexp = self.__funcParamsRegex = re.compile(r'''
2645                  '.*?(?<!\\)' |    # single quoted strings or
2646                  ".*?(?<!\\)" |    # double quoted strings or
2647                  \w+          |    # identifier
2648                  \S                # other characters
2649                  ''', re.VERBOSE | re.DOTALL)
2650          # 1. find the nearest identifier that comes before an unclosed
2651          # parenthesis before the cursor
2652          # e.g. for "foo (1+bar(x), pa<cursor>,a=1)", the candidate is "foo"
2653          tokens = regexp.findall(self.text_until_cursor)
2654          iterTokens = reversed(tokens)
2655          openPar = 0
2656  
2657          for token in iterTokens:
2658              if token == ')':
2659                  openPar -= 1
2660              elif token == '(':
2661                  openPar += 1
2662                  if openPar > 0:
2663                      # found the last unclosed parenthesis
2664                      break
2665          else:
2666              return []
2667          # 2. Concatenate dotted names ("foo.bar" for "foo.bar(x, pa" )
2668          ids = []
2669          isId = re.compile(r'\w+$').match
2670  
2671          while True:
2672              try:
2673                  ids.append(next(iterTokens))
2674                  if not isId(ids[-1]):
2675                      ids.pop()
2676                      break
2677                  if not next(iterTokens) == '.':
2678                      break
2679              except StopIteration:
2680                  break
2681  
2682          # Find all named arguments already assigned to, as to avoid suggesting
2683          # them again
2684          usedNamedArgs = set()
2685          par_level = -1
2686          for token, next_token in zip(tokens, tokens[1:]):
2687              if token == '(':
2688                  par_level += 1
2689              elif token == ')':
2690                  par_level -= 1
2691  
2692              if par_level != 0:
2693                  continue
2694  
2695              if next_token != '=':
2696                  continue
2697  
2698              usedNamedArgs.add(token)
2699  
2700          argMatches = []
2701          try:
2702              callableObj = '.'.join(ids[::-1])
2703              namedArgs = self._default_arguments(eval(callableObj,
2704                                                      self.namespace))
2705  
2706              # Remove used named arguments from the list, no need to show twice
2707              for namedArg in set(namedArgs) - usedNamedArgs:
2708                  if namedArg.startswith(text):
2709                      argMatches.append("%s=" %namedArg)
2710          except:
2711              pass
2712  
2713          return argMatches
2714  
2715      @staticmethod
2716      def _get_keys(obj: Any) -> List[Any]:
2717          # Objects can define their own completions by defining an
2718          # _ipy_key_completions_() method.
2719          method = get_real_method(obj, '_ipython_key_completions_')
2720          if method is not None:
2721              return method()
2722  
2723          # Special case some common in-memory dict-like types
2724          if isinstance(obj, dict) or _safe_isinstance(obj, "pandas", "DataFrame"):
2725              try:
2726                  return list(obj.keys())
2727              except Exception:
2728                  return []
2729          elif _safe_isinstance(obj, "pandas", "core", "indexing", "_LocIndexer"):
2730              try:
2731                  return list(obj.obj.keys())
2732              except Exception:
2733                  return []
2734          elif _safe_isinstance(obj, 'numpy', 'ndarray') or\
2735               _safe_isinstance(obj, 'numpy', 'void'):
2736              return obj.dtype.names or []
2737          return []
2738  
2739      @context_matcher()
2740      def dict_key_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2741          """Match string keys in a dictionary, after e.g. ``foo[``."""
2742          matches = self.dict_key_matches(context.token)
2743          return _convert_matcher_v1_result_to_v2(
2744              matches, type="dict key", suppress_if_matches=True
2745          )
2746  
2747      def dict_key_matches(self, text: str) -> List[str]:
2748          """Match string keys in a dictionary, after e.g. ``foo[``.
2749  
2750          .. deprecated:: 8.6
2751              You can use :meth:`dict_key_matcher` instead.
2752          """
2753  
2754          # Short-circuit on closed dictionary (regular expression would
2755          # not match anyway, but would take quite a while).
2756          if self.text_until_cursor.strip().endswith("]"):
2757              return []
2758  
2759          match = DICT_MATCHER_REGEX.search(self.text_until_cursor)
2760  
2761          if match is None:
2762              return []
2763  
2764          expr, prior_tuple_keys, key_prefix = match.groups()
2765  
2766          obj = self._evaluate_expr(expr)
2767  
2768          if obj is not_found:
2769              return []
2770  
2771          keys = self._get_keys(obj)
2772          if not keys:
2773              return keys
2774  
2775          tuple_prefix = guarded_eval(
2776              prior_tuple_keys,
2777              EvaluationContext(
2778                  globals=self.global_namespace,
2779                  locals=self.namespace,
2780                  evaluation=self.evaluation,  # type: ignore
2781                  in_subscript=True,
2782              ),
2783          )
2784  
2785          closing_quote, token_offset, matches = match_dict_keys(
2786              keys, key_prefix, self.splitter.delims, extra_prefix=tuple_prefix
2787          )
2788          if not matches:
2789              return []
2790  
2791          # get the cursor position of
2792          # - the text being completed
2793          # - the start of the key text
2794          # - the start of the completion
2795          text_start = len(self.text_until_cursor) - len(text)
2796          if key_prefix:
2797              key_start = match.start(3)
2798              completion_start = key_start + token_offset
2799          else:
2800              key_start = completion_start = match.end()
2801  
2802          # grab the leading prefix, to make sure all completions start with `text`
2803          if text_start > key_start:
2804              leading = ''
2805          else:
2806              leading = text[text_start:completion_start]
2807  
2808          # append closing quote and bracket as appropriate
2809          # this is *not* appropriate if the opening quote or bracket is outside
2810          # the text given to this method, e.g. `d["""a\nt
2811          can_close_quote = False
2812          can_close_bracket = False
2813  
2814          continuation = self.line_buffer[len(self.text_until_cursor) :].strip()
2815  
2816          if continuation.startswith(closing_quote):
2817              # do not close if already closed, e.g. `d['a<tab>'`
2818              continuation = continuation[len(closing_quote) :]
2819          else:
2820              can_close_quote = True
2821  
2822          continuation = continuation.strip()
2823  
2824          # e.g. `pandas.DataFrame` has different tuple indexer behaviour,
2825          # handling it is out of scope, so let's avoid appending suffixes.
2826          has_known_tuple_handling = isinstance(obj, dict)
2827  
2828          can_close_bracket = (
2829              not continuation.startswith("]") and self.auto_close_dict_keys
2830          )
2831          can_close_tuple_item = (
2832              not continuation.startswith(",")
2833              and has_known_tuple_handling
2834              and self.auto_close_dict_keys
2835          )
2836          can_close_quote = can_close_quote and self.auto_close_dict_keys
2837  
2838          # fast path if closing quote should be appended but not suffix is allowed
2839          if not can_close_quote and not can_close_bracket and closing_quote:
2840              return [leading + k for k in matches]
2841  
2842          results = []
2843  
2844          end_of_tuple_or_item = _DictKeyState.END_OF_TUPLE | _DictKeyState.END_OF_ITEM
2845  
2846          for k, state_flag in matches.items():
2847              result = leading + k
2848              if can_close_quote and closing_quote:
2849                  result += closing_quote
2850  
2851              if state_flag == end_of_tuple_or_item:
2852                  # We do not know which suffix to add,
2853                  # e.g. both tuple item and string
2854                  # match this item.
2855                  pass
2856  
2857              if state_flag in end_of_tuple_or_item and can_close_bracket:
2858                  result += "]"
2859              if state_flag == _DictKeyState.IN_TUPLE and can_close_tuple_item:
2860                  result += ", "
2861              results.append(result)
2862          return results
2863  
2864      @context_matcher()
2865      def unicode_name_matcher(self, context: CompletionContext):
2866          """Same as :any:`unicode_name_matches`, but adopted to new Matcher API."""
2867          fragment, matches = self.unicode_name_matches(context.text_until_cursor)
2868          return _convert_matcher_v1_result_to_v2(
2869              matches, type="unicode", fragment=fragment, suppress_if_matches=True
2870          )
2871  
2872      @staticmethod
2873      def unicode_name_matches(text: str) -> Tuple[str, List[str]]:
2874          """Match Latex-like syntax for unicode characters base
2875          on the name of the character.
2876  
2877          This does  ``\\GREEK SMALL LETTER ETA`` -> ``η``
2878  
2879          Works only on valid python 3 identifier, or on combining characters that
2880          will combine to form a valid identifier.
2881          """
2882          slashpos = text.rfind('\\')
2883          if slashpos > -1:
2884              s = text[slashpos+1:]
2885              try :
2886                  unic = unicodedata.lookup(s)
2887                  # allow combining chars
2888                  if ('a'+unic).isidentifier():
2889                      return '\\'+s,[unic]
2890              except KeyError:
2891                  pass
2892          return '', []
2893  
2894      @context_matcher()
2895      def latex_name_matcher(self, context: CompletionContext):
2896          """Match Latex syntax for unicode characters.
2897  
2898          This does both ``\\alp`` -> ``\\alpha`` and ``\\alpha`` -> ``α``
2899          """
2900          fragment, matches = self.latex_matches(context.text_until_cursor)
2901          return _convert_matcher_v1_result_to_v2(
2902              matches, type="latex", fragment=fragment, suppress_if_matches=True
2903          )
2904  
2905      def latex_matches(self, text: str) -> Tuple[str, Sequence[str]]:
2906          """Match Latex syntax for unicode characters.
2907  
2908          This does both ``\\alp`` -> ``\\alpha`` and ``\\alpha`` -> ``α``
2909  
2910          .. deprecated:: 8.6
2911              You can use :meth:`latex_name_matcher` instead.
2912          """
2913          slashpos = text.rfind('\\')
2914          if slashpos > -1:
2915              s = text[slashpos:]
2916              if s in latex_symbols:
2917                  # Try to complete a full latex symbol to unicode
2918                  # \\alpha -> α
2919                  return s, [latex_symbols[s]]
2920              else:
2921                  # If a user has partially typed a latex symbol, give them
2922                  # a full list of options \al -> [\aleph, \alpha]
2923                  matches = [k for k in latex_symbols if k.startswith(s)]
2924                  if matches:
2925                      return s, matches
2926          return '', ()
2927  
2928      @context_matcher()
2929      def custom_completer_matcher(self, context):
2930          """Dispatch custom completer.
2931  
2932          If a match is found, suppresses all other matchers except for Jedi.
2933          """
2934          matches = self.dispatch_custom_completer(context.token) or []
2935          result = _convert_matcher_v1_result_to_v2(
2936              matches, type=_UNKNOWN_TYPE, suppress_if_matches=True
2937          )
2938          result["ordered"] = True
2939          result["do_not_suppress"] = {_get_matcher_id(self._jedi_matcher)}
2940          return result
2941  
2942      def dispatch_custom_completer(self, text):
2943          """
2944          .. deprecated:: 8.6
2945              You can use :meth:`custom_completer_matcher` instead.
2946          """
2947          if not self.custom_completers:
2948              return
2949  
2950          line = self.line_buffer
2951          if not line.strip():
2952              return None
2953  
2954          # Create a little structure to pass all the relevant information about
2955          # the current completion to any custom completer.
2956          event = SimpleNamespace()
2957          event.line = line
2958          event.symbol = text
2959          cmd = line.split(None,1)[0]
2960          event.command = cmd
2961          event.text_until_cursor = self.text_until_cursor
2962  
2963          # for foo etc, try also to find completer for %foo
2964          if not cmd.startswith(self.magic_escape):
2965              try_magic = self.custom_completers.s_matches(
2966                  self.magic_escape + cmd)
2967          else:
2968              try_magic = []
2969  
2970          for c in itertools.chain(self.custom_completers.s_matches(cmd),
2971                   try_magic,
2972                   self.custom_completers.flat_matches(self.text_until_cursor)):
2973              try:
2974                  res = c(event)
2975                  if res:
2976                      # first, try case sensitive match
2977                      withcase = [r for r in res if r.startswith(text)]
2978                      if withcase:
2979                          return withcase
2980                      # if none, then case insensitive ones are ok too
2981                      text_low = text.lower()
2982                      return [r for r in res if r.lower().startswith(text_low)]
2983              except TryNext:
2984                  pass
2985              except KeyboardInterrupt:
2986                  """
2987                  If custom completer take too long,
2988                  let keyboard interrupt abort and return nothing.
2989                  """
2990                  break
2991  
2992          return None
2993  
2994      def completions(self, text: str, offset: int)->Iterator[Completion]:
2995          """
2996          Returns an iterator over the possible completions
2997  
2998          .. warning::
2999  
3000              Unstable
3001  
3002              This function is unstable, API may change without warning.
3003              It will also raise unless use in proper context manager.
3004  
3005          Parameters
3006          ----------
3007          text : str
3008              Full text of the current input, multi line string.
3009          offset : int
3010              Integer representing the position of the cursor in ``text``. Offset
3011              is 0-based indexed.
3012  
3013          Yields
3014          ------
3015          Completion
3016  
3017          Notes
3018          -----
3019          The cursor on a text can either be seen as being "in between"
3020          characters or "On" a character depending on the interface visible to
3021          the user. For consistency the cursor being on "in between" characters X
3022          and Y is equivalent to the cursor being "on" character Y, that is to say
3023          the character the cursor is on is considered as being after the cursor.
3024  
3025          Combining characters may span more that one position in the
3026          text.
3027  
3028          .. note::
3029  
3030              If ``IPCompleter.debug`` is :any:`True` will yield a ``--jedi/ipython--``
3031              fake Completion token to distinguish completion returned by Jedi
3032              and usual IPython completion.
3033  
3034          .. note::
3035  
3036              Completions are not completely deduplicated yet. If identical
3037              completions are coming from different sources this function does not
3038              ensure that each completion object will only be present once.
3039          """
3040          warnings.warn("_complete is a provisional API (as of IPython 6.0). "
3041                        "It may change without warnings. "
3042                        "Use in corresponding context manager.",
3043                        category=ProvisionalCompleterWarning, stacklevel=2)
3044  
3045          seen = set()
3046          profiler:Optional[cProfile.Profile]
3047          try:
3048              if self.profile_completions:
3049                  import cProfile
3050                  profiler = cProfile.Profile()
3051                  profiler.enable()
3052              else:
3053                  profiler = None
3054  
3055              for c in self._completions(text, offset, _timeout=self.jedi_compute_type_timeout/1000):
3056                  if c and (c in seen):
3057                      continue
3058                  yield c
3059                  seen.add(c)
3060          except KeyboardInterrupt:
3061              """if completions take too long and users send keyboard interrupt,
3062              do not crash and return ASAP. """
3063              pass
3064          finally:
3065              if profiler is not None:
3066                  profiler.disable()
3067                  ensure_dir_exists(self.profiler_output_dir)
3068                  output_path = os.path.join(self.profiler_output_dir, str(uuid.uuid4()))
3069                  print("Writing profiler output to", output_path)
3070                  profiler.dump_stats(output_path)
3071  
3072      def _completions(self, full_text: str, offset: int, *, _timeout) -> Iterator[Completion]:
3073          """
3074          Core completion module.Same signature as :any:`completions`, with the
3075          extra `timeout` parameter (in seconds).
3076  
3077          Computing jedi's completion ``.type`` can be quite expensive (it is a
3078          lazy property) and can require some warm-up, more warm up than just
3079          computing the ``name`` of a completion. The warm-up can be :
3080  
3081              - Long warm-up the first time a module is encountered after
3082              install/update: actually build parse/inference tree.
3083  
3084              - first time the module is encountered in a session: load tree from
3085              disk.
3086  
3087          We don't want to block completions for tens of seconds so we give the
3088          completer a "budget" of ``_timeout`` seconds per invocation to compute
3089          completions types, the completions that have not yet been computed will
3090          be marked as "unknown" an will have a chance to be computed next round
3091          are things get cached.
3092  
3093          Keep in mind that Jedi is not the only thing treating the completion so
3094          keep the timeout short-ish as if we take more than 0.3 second we still
3095          have lots of processing to do.
3096  
3097          """
3098          deadline = time.monotonic() + _timeout
3099  
3100          before = full_text[:offset]
3101          cursor_line, cursor_column = position_to_cursor(full_text, offset)
3102  
3103          jedi_matcher_id = _get_matcher_id(self._jedi_matcher)
3104  
3105          def is_non_jedi_result(
3106              result: MatcherResult, identifier: str
3107          ) -> TypeGuard[SimpleMatcherResult]:
3108              return identifier != jedi_matcher_id
3109  
3110          results = self._complete(
3111              full_text=full_text, cursor_line=cursor_line, cursor_pos=cursor_column
3112          )
3113  
3114          non_jedi_results: Dict[str, SimpleMatcherResult] = {
3115              identifier: result
3116              for identifier, result in results.items()
3117              if is_non_jedi_result(result, identifier)
3118          }
3119  
3120          jedi_matches = (
3121              cast(_JediMatcherResult, results[jedi_matcher_id])["completions"]
3122              if jedi_matcher_id in results
3123              else ()
3124          )
3125  
3126          iter_jm = iter(jedi_matches)
3127          if _timeout:
3128              for jm in iter_jm:
3129                  try:
3130                      type_ = jm.type
3131                  except Exception:
3132                      if self.debug:
3133                          print("Error in Jedi getting type of ", jm)
3134                      type_ = None
3135                  delta = len(jm.name_with_symbols) - len(jm.complete)
3136                  if type_ == 'function':
3137                      signature = _make_signature(jm)
3138                  else:
3139                      signature = ''
3140                  yield Completion(start=offset - delta,
3141                                   end=offset,
3142                                   text=jm.name_with_symbols,
3143                                   type=type_,
3144                                   signature=signature,
3145                                   _origin='jedi')
3146  
3147                  if time.monotonic() > deadline:
3148                      break
3149  
3150          for jm in iter_jm:
3151              delta = len(jm.name_with_symbols) - len(jm.complete)
3152              yield Completion(
3153                  start=offset - delta,
3154                  end=offset,
3155                  text=jm.name_with_symbols,
3156                  type=_UNKNOWN_TYPE,  # don't compute type for speed
3157                  _origin="jedi",
3158                  signature="",
3159              )
3160  
3161          # TODO:
3162          # Suppress this, right now just for debug.
3163          if jedi_matches and non_jedi_results and self.debug:
3164              some_start_offset = before.rfind(
3165                  next(iter(non_jedi_results.values()))["matched_fragment"]
3166              )
3167              yield Completion(
3168                  start=some_start_offset,
3169                  end=offset,
3170                  text="--jedi/ipython--",
3171                  _origin="debug",
3172                  type="none",
3173                  signature="",
3174              )
3175  
3176          ordered: List[Completion] = []
3177          sortable: List[Completion] = []
3178  
3179          for origin, result in non_jedi_results.items():
3180              matched_text = result["matched_fragment"]
3181              start_offset = before.rfind(matched_text)
3182              is_ordered = result.get("ordered", False)
3183              container = ordered if is_ordered else sortable
3184  
3185              # I'm unsure if this is always true, so let's assert and see if it
3186              # crash
3187              assert before.endswith(matched_text)
3188  
3189              for simple_completion in result["completions"]:
3190                  completion = Completion(
3191                      start=start_offset,
3192                      end=offset,
3193                      text=simple_completion.text,
3194                      _origin=origin,
3195                      signature="",
3196                      type=simple_completion.type or _UNKNOWN_TYPE,
3197                  )
3198                  container.append(completion)
3199  
3200          yield from list(self._deduplicate(ordered + self._sort(sortable)))[
3201              :MATCHES_LIMIT
3202          ]
3203  
3204      def complete(self, text=None, line_buffer=None, cursor_pos=None) -> Tuple[str, Sequence[str]]:
3205          """Find completions for the given text and line context.
3206  
3207          Note that both the text and the line_buffer are optional, but at least
3208          one of them must be given.
3209  
3210          Parameters
3211          ----------
3212          text : string, optional
3213              Text to perform the completion on.  If not given, the line buffer
3214              is split using the instance's CompletionSplitter object.
3215          line_buffer : string, optional
3216              If not given, the completer attempts to obtain the current line
3217              buffer via readline.  This keyword allows clients which are
3218              requesting for text completions in non-readline contexts to inform
3219              the completer of the entire text.
3220          cursor_pos : int, optional
3221              Index of the cursor in the full line buffer.  Should be provided by
3222              remote frontends where kernel has no access to frontend state.
3223  
3224          Returns
3225          -------
3226          Tuple of two items:
3227          text : str
3228              Text that was actually used in the completion.
3229          matches : list
3230              A list of completion matches.
3231  
3232          Notes
3233          -----
3234              This API is likely to be deprecated and replaced by
3235              :any:`IPCompleter.completions` in the future.
3236  
3237          """
3238          warnings.warn('`Completer.complete` is pending deprecation since '
3239                  'IPython 6.0 and will be replaced by `Completer.completions`.',
3240                        PendingDeprecationWarning)
3241          # potential todo, FOLD the 3rd throw away argument of _complete
3242          # into the first 2 one.
3243          # TODO: Q: does the above refer to jedi completions (i.e. 0-indexed?)
3244          # TODO: should we deprecate now, or does it stay?
3245  
3246          results = self._complete(
3247              line_buffer=line_buffer, cursor_pos=cursor_pos, text=text, cursor_line=0
3248          )
3249  
3250          jedi_matcher_id = _get_matcher_id(self._jedi_matcher)
3251  
3252          return self._arrange_and_extract(
3253              results,
3254              # TODO: can we confirm that excluding Jedi here was a deliberate choice in previous version?
3255              skip_matchers={jedi_matcher_id},
3256              # this API does not support different start/end positions (fragments of token).
3257              abort_if_offset_changes=True,
3258          )
3259  
3260      def _arrange_and_extract(
3261          self,
3262          results: Dict[str, MatcherResult],
3263          skip_matchers: Set[str],
3264          abort_if_offset_changes: bool,
3265      ):
3266          sortable: List[AnyMatcherCompletion] = []
3267          ordered: List[AnyMatcherCompletion] = []
3268          most_recent_fragment = None
3269          for identifier, result in results.items():
3270              if identifier in skip_matchers:
3271                  continue
3272              if not result["completions"]:
3273                  continue
3274              if not most_recent_fragment:
3275                  most_recent_fragment = result["matched_fragment"]
3276              if (
3277                  abort_if_offset_changes
3278                  and result["matched_fragment"] != most_recent_fragment
3279              ):
3280                  break
3281              if result.get("ordered", False):
3282                  ordered.extend(result["completions"])
3283              else:
3284                  sortable.extend(result["completions"])
3285  
3286          if not most_recent_fragment:
3287              most_recent_fragment = ""  # to satisfy typechecker (and just in case)
3288  
3289          return most_recent_fragment, [
3290              m.text for m in self._deduplicate(ordered + self._sort(sortable))
3291          ]
3292  
3293      def _complete(self, *, cursor_line, cursor_pos, line_buffer=None, text=None,
3294                    full_text=None) -> _CompleteResult:
3295          """
3296          Like complete but can also returns raw jedi completions as well as the
3297          origin of the completion text. This could (and should) be made much
3298          cleaner but that will be simpler once we drop the old (and stateful)
3299          :any:`complete` API.
3300  
3301          With current provisional API, cursor_pos act both (depending on the
3302          caller) as the offset in the ``text`` or ``line_buffer``, or as the
3303          ``column`` when passing multiline strings this could/should be renamed
3304          but would add extra noise.
3305  
3306          Parameters
3307          ----------
3308          cursor_line
3309              Index of the line the cursor is on. 0 indexed.
3310          cursor_pos
3311              Position of the cursor in the current line/line_buffer/text. 0
3312              indexed.
3313          line_buffer : optional, str
3314              The current line the cursor is in, this is mostly due to legacy
3315              reason that readline could only give a us the single current line.
3316              Prefer `full_text`.
3317          text : str
3318              The current "token" the cursor is in, mostly also for historical
3319              reasons. as the completer would trigger only after the current line
3320              was parsed.
3321          full_text : str
3322              Full text of the current cell.
3323  
3324          Returns
3325          -------
3326          An ordered dictionary where keys are identifiers of completion
3327          matchers and values are ``MatcherResult``s.
3328          """
3329  
3330          # if the cursor position isn't given, the only sane assumption we can
3331          # make is that it's at the end of the line (the common case)
3332          if cursor_pos is None:
3333              cursor_pos = len(line_buffer) if text is None else len(text)
3334  
3335          if self.use_main_ns:
3336              self.namespace = __main__.__dict__
3337  
3338          # if text is either None or an empty string, rely on the line buffer
3339          if (not line_buffer) and full_text:
3340              line_buffer = full_text.split('\n')[cursor_line]
3341          if not text:  # issue #11508: check line_buffer before calling split_line
3342              text = (
3343                  self.splitter.split_line(line_buffer, cursor_pos) if line_buffer else ""
3344              )
3345  
3346          # If no line buffer is given, assume the input text is all there was
3347          if line_buffer is None:
3348              line_buffer = text
3349  
3350          # deprecated - do not use `line_buffer` in new code.
3351          self.line_buffer = line_buffer
3352          self.text_until_cursor = self.line_buffer[:cursor_pos]
3353  
3354          if not full_text:
3355              full_text = line_buffer
3356  
3357          context = CompletionContext(
3358              full_text=full_text,
3359              cursor_position=cursor_pos,
3360              cursor_line=cursor_line,
3361              token=text,
3362              limit=MATCHES_LIMIT,
3363          )
3364  
3365          # Start with a clean slate of completions
3366          results: Dict[str, MatcherResult] = {}
3367  
3368          jedi_matcher_id = _get_matcher_id(self._jedi_matcher)
3369  
3370          suppressed_matchers: Set[str] = set()
3371  
3372          matchers = {
3373              _get_matcher_id(matcher): matcher
3374              for matcher in sorted(
3375                  self.matchers, key=_get_matcher_priority, reverse=True
3376              )
3377          }
3378  
3379          for matcher_id, matcher in matchers.items():
3380              matcher_id = _get_matcher_id(matcher)
3381  
3382              if matcher_id in self.disable_matchers:
3383                  continue
3384  
3385              if matcher_id in results:
3386                  warnings.warn(f"Duplicate matcher ID: {matcher_id}.")
3387  
3388              if matcher_id in suppressed_matchers:
3389                  continue
3390  
3391              result: MatcherResult
3392              try:
3393                  if _is_matcher_v1(matcher):
3394                      result = _convert_matcher_v1_result_to_v2(
3395                          matcher(text), type=_UNKNOWN_TYPE
3396                      )
3397                  elif _is_matcher_v2(matcher):
3398                      result = matcher(context)
3399                  else:
3400                      api_version = _get_matcher_api_version(matcher)
3401                      raise ValueError(f"Unsupported API version {api_version}")
3402              except BaseException:
3403                  # Show the ugly traceback if the matcher causes an
3404                  # exception, but do NOT crash the kernel!
3405                  sys.excepthook(*sys.exc_info())
3406                  continue
3407  
3408              # set default value for matched fragment if suffix was not selected.
3409              result["matched_fragment"] = result.get("matched_fragment", context.token)
3410  
3411              if not suppressed_matchers:
3412                  suppression_recommended: Union[bool, Set[str]] = result.get(
3413                      "suppress", False
3414                  )
3415  
3416                  suppression_config = (
3417                      self.suppress_competing_matchers.get(matcher_id, None)
3418                      if isinstance(self.suppress_competing_matchers, dict)
3419                      else self.suppress_competing_matchers
3420                  )
3421                  should_suppress = (
3422                      (suppression_config is True)
3423                      or (suppression_recommended and (suppression_config is not False))
3424                  ) and has_any_completions(result)
3425  
3426                  if should_suppress:
3427                      suppression_exceptions: Set[str] = result.get(
3428                          "do_not_suppress", set()
3429                      )
3430                      if isinstance(suppression_recommended, Iterable):
3431                          to_suppress = set(suppression_recommended)
3432                      else:
3433                          to_suppress = set(matchers)
3434                      suppressed_matchers = to_suppress - suppression_exceptions
3435  
3436                      new_results = {}
3437                      for previous_matcher_id, previous_result in results.items():
3438                          if previous_matcher_id not in suppressed_matchers:
3439                              new_results[previous_matcher_id] = previous_result
3440                      results = new_results
3441  
3442              results[matcher_id] = result
3443  
3444          _, matches = self._arrange_and_extract(
3445              results,
3446              # TODO Jedi completions non included in legacy stateful API; was this deliberate or omission?
3447              #  if it was omission, we can remove the filtering step, otherwise remove this comment.
3448              skip_matchers={jedi_matcher_id},
3449              abort_if_offset_changes=False,
3450          )
3451  
3452          # populate legacy stateful API
3453          self.matches = matches
3454  
3455          return results
3456  
3457      @staticmethod
3458      def _deduplicate(
3459          matches: Sequence[AnyCompletion],
3460      ) -> Iterable[AnyCompletion]:
3461          filtered_matches: Dict[str, AnyCompletion] = {}
3462          for match in matches:
3463              text = match.text
3464              if (
3465                  text not in filtered_matches
3466                  or filtered_matches[text].type == _UNKNOWN_TYPE
3467              ):
3468                  filtered_matches[text] = match
3469  
3470          return filtered_matches.values()
3471  
3472      @staticmethod
3473      def _sort(matches: Sequence[AnyCompletion]):
3474          return sorted(matches, key=lambda x: completions_sorting_key(x.text))
3475  
3476      @context_matcher()
3477      def fwd_unicode_matcher(self, context: CompletionContext):
3478          """Same as :any:`fwd_unicode_match`, but adopted to new Matcher API."""
3479          # TODO: use `context.limit` to terminate early once we matched the maximum
3480          #  number that will be used downstream; can be added as an optional to
3481          #  `fwd_unicode_match(text: str, limit: int = None)` or we could re-implement here.
3482          fragment, matches = self.fwd_unicode_match(context.text_until_cursor)
3483          return _convert_matcher_v1_result_to_v2(
3484              matches, type="unicode", fragment=fragment, suppress_if_matches=True
3485          )
3486  
3487      def fwd_unicode_match(self, text: str) -> Tuple[str, Sequence[str]]:
3488          """
3489          Forward match a string starting with a backslash with a list of
3490          potential Unicode completions.
3491  
3492          Will compute list of Unicode character names on first call and cache it.
3493  
3494          .. deprecated:: 8.6
3495              You can use :meth:`fwd_unicode_matcher` instead.
3496  
3497          Returns
3498          -------
3499          At tuple with:
3500              - matched text (empty if no matches)
3501              - list of potential completions, empty tuple  otherwise)
3502          """
3503          # TODO: self.unicode_names is here a list we traverse each time with ~100k elements.
3504          # We could do a faster match using a Trie.
3505  
3506          # Using pygtrie the following seem to work:
3507  
3508          #     s = PrefixSet()
3509  
3510          #     for c in range(0,0x10FFFF + 1):
3511          #         try:
3512          #             s.add(unicodedata.name(chr(c)))
3513          #         except ValueError:
3514          #             pass
3515          #     [''.join(k) for k in s.iter(prefix)]
3516  
3517          # But need to be timed and adds an extra dependency.
3518  
3519          slashpos = text.rfind('\\')
3520          # if text starts with slash
3521          if slashpos > -1:
3522              # PERF: It's important that we don't access self._unicode_names
3523              # until we're inside this if-block. _unicode_names is lazily
3524              # initialized, and it takes a user-noticeable amount of time to
3525              # initialize it, so we don't want to initialize it unless we're
3526              # actually going to use it.
3527              s = text[slashpos + 1 :]
3528              sup = s.upper()
3529              candidates = [x for x in self.unicode_names if x.startswith(sup)]
3530              if candidates:
3531                  return s, candidates
3532              candidates = [x for x in self.unicode_names if sup in x]
3533              if candidates:
3534                  return s, candidates
3535              splitsup = sup.split(" ")
3536              candidates = [
3537                  x for x in self.unicode_names if all(u in x for u in splitsup)
3538              ]
3539              if candidates:
3540                  return s, candidates
3541  
3542              return "", ()
3543  
3544          # if text does not start with slash
3545          else:
3546              return '', ()
3547  
3548      @property
3549      def unicode_names(self) -> List[str]:
3550          """List of names of unicode code points that can be completed.
3551  
3552          The list is lazily initialized on first access.
3553          """
3554          if self._unicode_names is None:
3555              names = []
3556              for c in range(0,0x10FFFF + 1):
3557                  try:
3558                      names.append(unicodedata.name(chr(c)))
3559                  except ValueError:
3560                      pass
3561              self._unicode_names = _unicode_name_compute(_UNICODE_RANGES)
3562  
3563          return self._unicode_names
3564  
3565  def _unicode_name_compute(ranges:List[Tuple[int,int]]) -> List[str]:
3566      names = []
3567      for start,stop in ranges:
3568          for c in range(start, stop) :
3569              try:
3570                  names.append(unicodedata.name(chr(c)))
3571              except ValueError:
3572                  pass
3573      return names
3574