Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build fails for release and forks, advice? #70

Open
tetov opened this issue Oct 14, 2019 · 27 comments
Open

Build fails for release and forks, advice? #70

tetov opened this issue Oct 14, 2019 · 27 comments

Comments

@tetov
Copy link

tetov commented Oct 14, 2019

Hi!

I understand that the status of sphinxcontrib-versioning is a bit complicated, but I gather from reading from the issues and PRs and from looking at a few doc setups that people have this working. I was wondering if someone could help me with an issue I haven't found reported here on the repo.

Running sphinx-versioning build with the pypi version I get the error documented in issue #66 fixed in PR #69. However, running @z00sts' PR branch quits with a file not found error.

I've also tried @leokoppel's fork, but there it hangs after two runs of doc building and no files found in the reported path (/tmp/tmp*sphinxcontrib_versioning).

Anyone encountered something similar or could give some points towards getting this to work with compas-dev/compas?

General info

$ python --version
Python 3.6.9

$ pip freeze | grep -i sphinx
nbsphinx==0.4.3
Sphinx==2.2.0
sphinx-compas-theme==0.4.1
sphinxcontrib-applehelp==1.0.1
sphinxcontrib-devhelp==1.0.1
sphinxcontrib-htmlhelp==1.0.2
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.2
sphinxcontrib-serializinghtml==1.1.3
sphinxcontrib-versioning==2.2.1

$ git remote -v

origin          [email protected]:compas-dev/compas.git (fetch)
origin          [email protected]:compas-dev/compas.git (push)

Logs

$ pip install git+https://github.com/z00sts/[email protected]
$ sphinx-versioning -L build -i -r master docs /dist/docs
=> Gathering info about the remote git repository...
=> Getting list of all remote branches/tags...
=> Found: docs-pull-req footnote-fix master patch-1 pathlib_support quick-fix-clr-pythonnet sphinx-versioning update_robot worbit-patch-1 0.1.0 0.1.0-drx_numpy_callback 0.1.0-facenetwork_import 0.1.0-subd_ck_example_imports 0.1.0-viewer_glut_title_str 0.1.1 0.1.1-import_error_smoothing_cpp 0.1.1-import_problems_cpp_versions 0.1.1-leaves_after_embedding 0.1.1-rhino_utility_aliases 0.1.1-xfunc_callback_args 0.1.1_ga_bugfix v0.3.0 v0.3.1 v0.3.2 v0.3.3 v0.3.4 v0.3.5 v0.3.6 v0.4.0 v0.4.1 v0.4.10 v0.4.11 v0.4.12 v0.4.13 v0.4.14 v0.4.15 v0.4.16 v0.4.17 v0.4.18 v0.4.19 v0.4.2 v0.4.20 v0.4.21 v0.4.22 v0.4.23 v0.4.3 v0.4.4 v0.4.5 v0.4.6 v0.4.7 v0.4.8 v0.4.9 v0.5.0 v0.5.1 v0.5.2 v0.6.0 v0.6.1 v0.6.2 v0.7.0 v0.7.1 v0.7.2
=> With docs: docs-pull-req footnote-fix master patch-1 pathlib_support quick-fix-clr-pythonnet sphinx-versioning update_robot worbit-patch-1 v0.3.0 v0.3.1 v0.3.2 v0.3.3 v0.3.4 v0.3.5 v0.3.6 v0.4.0 v0.4.1 v0.4.10 v0.4.11 v0.4.12 v0.4.13 v0.4.14 v0.4.15 v0.4.16 v0.4.17 v0.4.18 v0.4.19 v0.4.2 v0.4.20 v0.4.21 v0.4.22 v0.4.23 v0.4.3 v0.4.4 v0.4.5 v0.4.6 v0.4.7 v0.4.8 v0.4.9 v0.5.0 v0.5.1 v0.5.2 v0.6.0 v0.6.1 v0.6.2 v0.7.0 v0.7.1 v0.7.2
=> Root ref is: master
=> Pre-running Sphinx to collect versions' master_doc and other info.
usage: sphinx-versioning [OPTIONS] SOURCEDIR OUTPUTDIR [FILENAMES...]
sphinx-versioning: error: cannot find files ['/tmp/tmpm9pm2hi8sphinxcontrib_versioning']
=> sphinx-build failed for branch/tag: master
Failure.
$ pip install git+https://github.com/leokoppel/sphinxcontrib-versioning
$ sphinx-versioning -L build -i -r master docs /dist/docs
=> Gathering info about the remote git repository...
=> Getting list of all remote branches/tags...

[...]
# Runs twice with Sphinx-warnings that we see with normal runs
[...]

copying static files... ... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded, 27 warnings.

The HTML pages are in ../../../../tmp/tmplibvh55lsphinxcontrib_versioning.

# Freezes
@super-sk
Copy link

Hi Anton !

I encountered the same problem with the sphinxcontrib-versioning 2.2.1 from pypi. Even if I solved my problem in #66 I'm still stuck like you mentioned above.

@Stonesjtu
Copy link

Stonesjtu commented Nov 7, 2019

Hi there, after digging around for 1 hour, I found a simple hack should work for this case.

Change the argv to argv[1:].

Currently by default argv has three elements: sphinx-build, {SOURCE}, {TARGET}. But I suspect the new sphinx accepts only the last two, it will treat {TARGET} as filename option and check if it's really a file, which is a folder of course (Here's the logging line Not files).

@tetov
Copy link
Author

tetov commented Nov 7, 2019

Thanks! Going to test it out later today!

@Stonesjtu
Copy link

@tetov sorry for post a wrong solution, it's for another bug. BTW I encountered the same freezing bug too.

@Stonesjtu
Copy link

There are several things to make it work in my setting up.

  1. add sphinxcontrib.versioning.sphinx_ to the extension list in conf.py
  2. check-out support newer versions of sphinx #46 to fix the version mis-match issue.

It should work now.

@vancheese
Copy link

I've used fresh install from the main by following the two points above (I manually patched the sphinx.py) and then also also the ImportError: cannot import name 'build_main' fix
The program hangs after display

The HTML pages are in /tmp/tmpbuild directory

This error occurs with my real project and also with the tutorials simple example (https://sphinxcontrib-versioning.readthedocs.io/en/latest/tutorial.html#build-all-versions)

@Stonesjtu
Copy link

You can try add some log lines in these callbacks to see if this extension is registered correctly.


def env_updated(cls, app, env):

@GlitchyPSIX
Copy link

Any progress in this? I have this same problem.

@vancheese
Copy link

You can try add some log lines in these callbacks to see if this extension is registered correctly.

def env_updated(cls, app, env):

I added simple print commands to see if the two functions were called and finished.
Both function weren't called before the programm freezed

@Stonesjtu
Copy link

@vancheese Then the extension itself is not registered properly. Can you post the conf.py and command that failed?

@vancheese
Copy link

I've identified where the program freeze
In sphinx_.py in read_config, the program stops at

config = queue.get()

@GlitchyPSIX
Copy link

I've found that applying the patch from #46, then applying the argv[1:] fix and #66 makes the build system work and not stall anymore...

This may be because of sphinx version uncompatibility, because the fixes themselves end up stalling until #46 is applied

@paddy-hack
Copy link

I've been hit by this as well and tried to figure out what broke in which sphinx versions. I've used the Docker sphinx that I maintain for this and used the "quick-and-dirty" documentation from the tutorial for testing.

Version 2.0.1 is the first version where I observed the hang/freeze. Apart from the sphinx version, there are no other differences between my Docker images for 2.0.0 and 2.0.1. The changes between those two don't provide any clue, to me at least, what might have triggered this.

@GlitchyPSIX
Copy link

GlitchyPSIX commented Nov 18, 2019

@paddy-hack The changes I spoke about in my last comment work on sphinx 2.2.1 too

@dlopezlo
Copy link

Hi,

i'm using the fork: https://github.com/Smile-SA/sphinx-versions installed from pip and all of the above patches are already applied but it still hangs from sphinx-2.0.1.

Have anyone tried this library?

@GlitchyPSIX does it work for you with patches you mentioned? #46 and #66

@GlitchyPSIX
Copy link

GlitchyPSIX commented Nov 20, 2019

@dlopezlo Yes (sphinx 2.2.1), I have managed to get sphinxcontrib-versioning building properly by applying #46, #66, and the argv[1:] fix (I did them in this order)

Do remember to add the sphinxcontrib.versioning.sphinx_ extension in conf.py

I made a separate conf.py for when I want to build separately without versioning which doesn't have that extension which I build using sphinx-build

@paddy-hack
Copy link

With some real documentation, I also ran into #48, which is fixed in #51.

@paddy-hack
Copy link

As for adding the sphinxcontrib.versioning.sphinx_ extension in conf.py, while that may be an obvious(?) thing to do, I didn't and it is not in the documentation, yet 😢.
I'm off now giving that a try 🤞

@paddy-hack
Copy link

Adding sphinxcontrib.versioning.sphinx_ to the extensions array in conf.py fixes the hang with this caveat that you can only build documentation for versions that do. If you try to build (older) versions of your documentation where the extension has not been added, the build will hang.

Fix confirmed to work on 2.0.1, 2.1.0, 2.1.2, 2.2.0 and 2.2.1 using the tutorial example with the extension added to conf.py.

@fmmix
Copy link

fmmix commented Nov 27, 2019

After applying all those fixes I get the message:

miniconda3/envs/py35/lib/python3.5/site-packages/sphinxcontrib/versioning/sphinx_.py", line 101, in html_page_context
    cls.VERSIONS.context = context
AttributeError: 'NoneType' object has no attribute 'context'

Does anyone have the same issue? Or did I maybe do something wrong?

@GlitchyPSIX
Copy link

Adding sphinxcontrib.versioning.sphinx_ to the extensions array in conf.py fixes the hang with this caveat that you can only build documentation for versions that do. If you try to build (older) versions of your documentation where the extension has not been added, the build will hang.

My workaround for this is just a second conf.py in a separate directory made only for non-versioned building

@paddy-hack
Copy link

I'm aware of your work around but that doesn't solve thing for documentation versions that have already been committed and pushed 😢

@paddy-hack
Copy link

@fmmix I got the same error when accidently running sphinx-build instead of sphinx-versioning.

@paddy-hack
Copy link

@GlitchyPSIX I've resorted to conditionally added the extension based on an environment variable. Since I normally only want to run sphinx-versionsing in CI (and sphinx-build during "development" of the documentation), I have added it like so

if 'CI' in os.environ and os.environ['CI'] == 'true':
    extensions.append('sphinxcontrib.versioning.sphinx_')

Works For Me 😄

@GlitchyPSIX
Copy link

@paddy-hack Haha, yeah, figured this would be a solution

In the end that's kind of what I achieved with the second conf.py since I don't use CI but batch files to generate the respective documentation types

Maybe your solution can work for others facing this issue? Looks a bit more catch-all than my local-based case.

@ClementMaliet
Copy link

It took me a while but I finally figured it out for good.

I will not make a PR because I find the fix to be especially ugly but I leave it there as a git patch for those encountering the issue.

The bug happens starting from Sphinx 2.0.1 as some noticed and is due to the fact that the CONFIG_FILENAME attribute of the sphinx.application module was deprecated with a 3.0.0 deletion in mind. Now the Sphinx deprecation helper actually wrap the entire module in sys.modules into a _ModuleWrapper to wrap the module __getattribute__ to send a DeprecationWarning if the deprecated attribute is to be fetch. This is all good and all but it does breaks the attempt that sphinxcontrib.versioning makes to monkey patch the sphinx.application.Config object to auto-register sphinxcontrib.versioning.sphinx_ as an extension (which actually fixes the hang as previous comments found out) upon instantiation.

My UGLY hotfix bypasses the _ModuleWrapper explicitly and will not work for sphinx versions before the 2.0.1 and after at least the 3.0.0.

Index: sphinxcontrib/versioning/sphinx_.py
<+>UTF-8
===================================================================
--- sphinxcontrib/versioning/sphinx_.py	(revision f286d43595d75c08cb51a0153a9cdf3b8a1957b3)
+++ sphinxcontrib/versioning/sphinx_.py	(date 1582638010030)
@@ -184,8 +184,13 @@
     :param str current_name: The ref name of the current version being built.
     :param bool is_root: Is this build in the web root?
     """
-    # Patch.
-    application.Config = ConfigInject
+    # Update monkey patch to work with Sphinx 2.0.1+ _ModuleWrapper addition around sphinx.application due to
+    # CONFIG_FILENAME deprecation.
+    # _ModuleWrapper does not redirect __setattr__ calls to the underlying module so we need to bypass it to inject
+    # the extension.
+    # This is ugly and will break starting from at least Sphinx 3.0+ but the idea of monkey patching the extension
+    # silently into sphinx was hackish at most to begin with.
+    application._module.Config = ConfigInject  # noqa
     if config.show_banner:
         EventHandlers.BANNER_GREATEST_TAG = config.banner_greatest_tag
         EventHandlers.BANNER_MAIN_VERSION = config.banner_main_ref

@Holzhaus
Copy link

I needed a versioning tool for Sphinx but all existing projects that I found didn't work properly and their code was messy, so I decided to implement my own solution and published it here: https://holzhaus.github.io/sphinx-multiversion/master/index.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants