BUG: Docs won't build (Unexpected exception in doc\source\user_guide\enhancingperf.rst)

[Ravenin7]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

git clone https://github.com/pandas-dev/pandas.git
cd pandas
conda env create --file environment.yml
conda activate pandas-dev
python -m pip install -ve . --no-build-isolation --config-settings editable-verbose=true
cd doc

python make.py html
python make.py clean
python make.py --single user_guide/enhancingperf.rst

Issue Description

I was following the steps to build the documentation as given here, but while running python make.py html it failed when building user_guide/enhancingperf.rst :

WARNING: ources... [ 95%] user_guide/enhancingperf
>>>-------------------------------------------------------------------------
Exception in C:\Users\91942\Documents\GitHub\pandas\doc\source\user_guide\enhancingperf.rst at block ending on line 101
Specify :okexcept: as an option in the ipython:: block to suppress this message
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[6], line 1
----> 1 get_ipython().run_line_magic('prun', '-l 4 df.apply(lambda x: integrate_f(x["a"], x["b"], x["N"]), axis=1)  # noqa E999')

File ~\anaconda3\envs\pandas-dev\lib\site-packages\IPython\core\interactiveshell.py:2480, in InteractiveShell.run_line_magic(self, magic_name, line, _stack_depth)
   2478     kwargs['local_ns'] = self.get_local_scope(stack_depth)
   2479 with self.builtin_trap:
-> 2480     result = fn(*args, **kwargs)
   2482 # The code below prevents the output from being displayed
   2483 # when using magics with decorator @output_can_be_silenced
   2484 # when the last Python token in the expression is a ';'.
   2485 if getattr(fn, magic.MAGIC_OUTPUT_CAN_BE_SILENCED, False):

File ~\anaconda3\envs\pandas-dev\lib\site-packages\IPython\core\magics\execution.py:317, in ExecutionMagics.prun(self, parameter_s, cell)
    315     arg_str += '\n' + cell
    316 arg_str = self.shell.transform_cell(arg_str)
--> 317 return self._run_with_profiler(arg_str, opts, self.shell.user_ns)

File ~\anaconda3\envs\pandas-dev\lib\site-packages\IPython\core\magics\execution.py:339, in ExecutionMagics._run_with_profiler(self, code, opts, namespace)
    337 prof = profile.Profile()
    338 try:
--> 339     prof = prof.runctx(code, namespace, namespace)
    340     sys_exit = ''
    341 except SystemExit:

File ~\anaconda3\envs\pandas-dev\lib\cProfile.py:101, in Profile.runctx(self, cmd, globals, locals)
     99 self.enable()
    100 try:
--> 101     exec(cmd, globals, locals)
    102 finally:
    103     self.disable()

File <string>:1

File ~\Documents\GitHub\pandas\pandas\core\frame.py:10398, in DataFrame.apply(self, func, axis, raw, result_type, args, by_row, engine, engine_kwargs, **kwargs)
  10384 from pandas.core.apply import frame_apply
  10386 op = frame_apply(
  10387     self,
  10388     func=func,
   (...)
  10396     kwargs=kwargs,
  10397 )
> 10398 return op.apply().__finalize__(self, method="apply")

File ~\Documents\GitHub\pandas\pandas\core\apply.py:903, in FrameApply.apply(self)
    900 elif self.raw:
    901     return self.apply_raw(engine=self.engine, engine_kwargs=self.engine_kwargs)
--> 903 return self.apply_standard()

File ~\Documents\GitHub\pandas\pandas\core\apply.py:1054, in FrameApply.apply_standard(self)
   1052 def apply_standard(self):
   1053     if self.engine == "python":
-> 1054         results, res_index = self.apply_series_generator()
   1055     else:
   1056         results, res_index = self.apply_series_numba()

File ~\Documents\GitHub\pandas\pandas\core\apply.py:1070, in FrameApply.apply_series_generator(self)
   1067 results = {}
   1069 for i, v in enumerate(series_gen):
-> 1070     results[i] = self.func(v, *self.args, **self.kwargs)
   1071     if isinstance(results[i], ABCSeries):
   1072         # If we have a view on v, we need to make a copy because
   1073         #  series_generator will swap out the underlying data
   1074         results[i] = results[i].copy(deep=False)

File <string>:1, in <lambda>(x)

File ~\Documents\GitHub\pandas\pandas\core\series.py:969, in Series.__getitem__(self, key)
    966     key = np.asarray(key, dtype=bool)
    967     return self._get_rows_with_mask(key)
--> 969 return self._get_with(key)

File ~\Documents\GitHub\pandas\pandas\core\series.py:981, in Series._get_with(self, key)
    978 elif isinstance(key, tuple):
    979     return self._get_values_tuple(key)
--> 981 return self.loc[key]

File ~\Documents\GitHub\pandas\pandas\core\indexing.py:1195, in _LocationIndexer.__getitem__(self, key)
   1193 maybe_callable = com.apply_if_callable(key, self.obj)
   1194 maybe_callable = self._raise_callable_usage(key, maybe_callable)
-> 1195 return self._getitem_axis(maybe_callable, axis=axis)

File ~\Documents\GitHub\pandas\pandas\core\indexing.py:1424, in _LocIndexer._getitem_axis(self, key, axis)
   1421     if hasattr(key, "ndim") and key.ndim > 1:
   1422         raise ValueError("Cannot index with multidimensional key")
-> 1424     return self._getitem_iterable(key, axis=axis)
   1426 # nested tuple slicing
   1427 if is_nested_tuple(key, labels):

File ~\Documents\GitHub\pandas\pandas\core\indexing.py:1364, in _LocIndexer._getitem_iterable(self, key, axis)
   1361 self._validate_key(key, axis)
   1363 # A collection of keys
-> 1364 keyarr, indexer = self._get_listlike_indexer(key, axis)
   1365 return self.obj._reindex_with_indexers(
   1366     {axis: [keyarr, indexer]}, allow_dups=True
   1367 )

File ~\Documents\GitHub\pandas\pandas\core\indexing.py:1562, in _LocIndexer._get_listlike_indexer(self, key, axis)
   1559 ax = self.obj._get_axis(axis)
   1560 axis_name = self.obj._get_axis_name(axis)
-> 1562 keyarr, indexer = ax._get_indexer_strict(key, axis_name)
   1564 return keyarr, indexer

File ~\Documents\GitHub\pandas\pandas\core\indexes\base.py:6056, in Index._get_indexer_strict(self, key, axis_name)
   6053 else:
   6054     keyarr, indexer, new_indexer = self._reindex_non_unique(keyarr)
-> 6056 self._raise_if_missing(keyarr, indexer, axis_name)
   6058 keyarr = self.take(indexer)
   6059 if isinstance(key, Index):
   6060     # GH 42790 - Preserve name from an Index

File ~\Documents\GitHub\pandas\pandas\core\indexes\base.py:6108, in Index._raise_if_missing(self, key, indexer, axis_name)
   6105     raise KeyError(f"None of [{key}] are in the [{axis_name}]")
   6107 not_found = list(ensure_index(key)[missing_mask.nonzero()[0]].unique())
-> 6108 raise KeyError(f"{not_found} not in index")

KeyError: "['c'] not in index"

<<<-------------------------------------------------------------------------

Exception occurred:
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\IPython\sphinxext\ipython_directive.py", line 584, in process_input
    raise RuntimeError(
RuntimeError: Unexpected exception in `C:\Users\91942\Documents\GitHub\pandas\doc\source\user_guide\enhancingperf.rst` line 101
The full traceback has been saved in C:\Users\91942\AppData\Local\Temp\sphinx-err-5kwabx28.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!

The same error occurs while running python make.py --single user_guide/enhancingperf.rst.

Full traceback given in sphinx-err-5kwabx28.log is:

# Platform:         win32; (Windows-10-10.0.22631-SP0)
# Sphinx version:   8.1.3
# Python version:   3.10.15 (CPython)
# Docutils version: 0.21.2
# Jinja2 version:   3.1.4
# Pygments version: 2.18.0

# Last messages:
#   user_guide/dsintro
#   
#   
#   reading sources... [ 95%]
#   user_guide/duplicates
#   
#   
#   reading sources... [ 95%]
#   user_guide/enhancingperf
#   

# Loaded extensions:
#   sphinx.ext.mathjax (8.1.3)
#   alabaster (1.0.0)
#   sphinxcontrib.applehelp (2.0.0)
#   sphinxcontrib.devhelp (2.0.0)
#   sphinxcontrib.htmlhelp (2.1.0)
#   sphinxcontrib.serializinghtml (1.1.10)
#   sphinxcontrib.qthelp (2.0.0)
#   contributors (0.1)
#   IPython.sphinxext.ipython_directive (unknown version)
#   IPython.sphinxext.ipython_console_highlighting (unknown version)
#   matplotlib.sphinxext.plot_directive (3.9.2)
#   sphinx.ext.autodoc.preserve_defaults (8.1.3)
#   sphinx.ext.autodoc.type_comment (8.1.3)
#   sphinx.ext.autodoc.typehints (8.1.3)
#   sphinx.ext.autodoc (8.1.3)
#   sphinx.ext.autosummary (8.1.3)
#   numpydoc (1.8.0)
#   sphinx_copybutton (0.5.2)
#   sphinx_design (0.6.1)
#   sphinx.ext.coverage (8.1.3)
#   sphinx.ext.doctest (8.1.3)
#   sphinx.ext.extlinks (8.1.3)
#   sphinx.ext.ifconfig (8.1.3)
#   sphinx.ext.intersphinx (8.1.3)
#   sphinx.ext.linkcode (8.1.3)
#   sphinx.ext.todo (8.1.3)
#   nbsphinx (0.9.5)
#   pydata_sphinx_theme (unknown version)

# Traceback:
Traceback (most recent call last):
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\sphinx\cmd\build.py", line 514, in build_main
    app.build(args.force_all, args.filenames)
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\sphinx\application.py", line 381, in build
    self.builder.build_update()
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\sphinx\builders\__init__.py", line 358, in build_update
    self.build(
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\sphinx\builders\__init__.py", line 385, in build
    updated_docnames = set(self.read())
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\sphinx\builders\__init__.py", line 502, in read
    self._read_serial(docnames)
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\sphinx\builders\__init__.py", line 567, in _read_serial
    self.read_doc(docname)
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\sphinx\builders\__init__.py", line 630, in read_doc
    publisher.publish()
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\core.py", line 234, in publish
    self.document = self.reader.read(self.source, self.parser,
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\sphinx\io.py", line 106, in read
    self.parse()
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\readers\__init__.py", line 76, in parse
    self.parser.parse(self.input, document)
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\sphinx\parsers.py", line 85, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\parsers\rst\states.py", line 169, in run
    results = StateMachineWS.run(self, input_lines, input_offset,
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\statemachine.py", line 233, in run
    context, next_state, result = self.check_line(
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\statemachine.py", line 445, in check_line
    return method(match, context, next_state)
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\parsers\rst\states.py", line 3034, in text
    self.section(title.lstrip(), source, style, lineno + 1, messages)
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\parsers\rst\states.py", line 325, in section
    self.new_subsection(title, lineno, messages)
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\parsers\rst\states.py", line 391, in new_subsection
    newabsoffset = self.nested_parse(
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\parsers\rst\states.py", line 279, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\parsers\rst\states.py", line 195, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\statemachine.py", line 233, in run
    context, next_state, result = self.check_line(
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\statemachine.py", line 445, in check_line
    return method(match, context, next_state)
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\parsers\rst\states.py", line 2790, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\parsers\rst\states.py", line 325, in section
    self.new_subsection(title, lineno, messages)
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\parsers\rst\states.py", line 391, in new_subsection
    newabsoffset = self.nested_parse(
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\parsers\rst\states.py", line 279, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\parsers\rst\states.py", line 195, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\statemachine.py", line 233, in run
    context, next_state, result = self.check_line(
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\statemachine.py", line 445, in check_line
    return method(match, context, next_state)
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\parsers\rst\states.py", line 2790, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\parsers\rst\states.py", line 325, in section
    self.new_subsection(title, lineno, messages)
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\parsers\rst\states.py", line 391, in new_subsection
    newabsoffset = self.nested_parse(
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\parsers\rst\states.py", line 279, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\parsers\rst\states.py", line 195, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\statemachine.py", line 233, in run
    context, next_state, result = self.check_line(
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\statemachine.py", line 445, in check_line
    return method(match, context, next_state)
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\parsers\rst\states.py", line 2357, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\parsers\rst\states.py", line 2369, in explicit_construct
    return method(self, expmatch)
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\parsers\rst\states.py", line 2106, in directive
    return self.run_directive(
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\docutils\parsers\rst\states.py", line 2156, in run_directive
    result = directive_instance.run()
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\IPython\sphinxext\ipython_directive.py", line 1033, in run
    rows, figure = self.shell.process_block(block)
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\IPython\sphinxext\ipython_directive.py", line 732, in process_block
    self.process_input(data, input_prompt, lineno)
  File "C:\Users\91942\anaconda3\envs\pandas-dev\lib\site-packages\IPython\sphinxext\ipython_directive.py", line 584, in process_input
    raise RuntimeError(
RuntimeError: Unexpected exception in `C:\Users\91942\Documents\GitHub\pandas\doc\source\user_guide\enhancingperf.rst` line 101

The issue appears to be with this line:

pandas/doc/source/user_guide/enhancingperf.rst

Line 101 in 9cd4a28

%load_ext Cython

Should I add :okexcept: to the block to suppress this message?

Expected Behavior

HTML doc should be successfully built.

Installed Versions

INSTALLED VERSIONS

commit : 7bd594c
python : 3.10.15
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.22631
machine : AMD64
processor : Intel64 Family 6 Model 154 Stepping 4, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : English_India.1252

pandas : 3.0.0.dev0+1600.g7bd594c81a
numpy : 1.26.4
dateutil : 2.9.0
pip : 24.3.1
Cython : 3.0.11
sphinx : 8.1.3
IPython : 8.29.0
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : 4.12.3
blosc : None
bottleneck : 1.4.2
fastparquet : 2024.5.0
fsspec : 2024.10.0
html5lib : 1.1
hypothesis : 6.115.6
gcsfs : 2024.10.0
jinja2 : 3.1.4
lxml.etree : 5.3.0
matplotlib : 3.9.2
numba : 0.60.0
numexpr : 2.10.1
odfpy : None
openpyxl : 3.1.5
psycopg2 : 2.9.9
pymysql : 1.4.6
pyarrow : 18.0.0
pyreadstat : 1.2.8
pytest : 8.3.3
python-calamine : None
pytz : 2024.2
pyxlsb : 1.0.10
s3fs : 2024.10.0
scipy : 1.14.1
sqlalchemy : 2.0.36
tables : 3.10.1
tabulate : 0.9.0
xarray : 2024.9.0
xlrd : 2.0.1
xlsxwriter : 3.2.0
zstandard : 0.23.0
tzdata : 2024.2
qtpy : None
pyqt5 : None

[jorisvandenbossche]

On the dev docs (https://pandas.pydata.org/docs/dev/user_guide/enhancingperf.html#pure-python) it is not failing, so ideally we shouldn't add a okexcept because that could then hide actual errors if it would later start to error (i.e. for the online docs, we never want this to error, so we want to get notified if that would happen).

Now, that said, I don't directly understand what would cause the error you see. Especially the KeyError: "['c'] not in index" error message, because the code example being run there doesn't mention any "c" column.