"deploy-docs" github action for PR in Autoware Documentation fails

[kminoda]

I am currently working on a PR in Autoware Documentation, and there seems to be an issue in Github actions.

"deploy-docs" github action fails with the following error:

Run autowarefoundation/autoware-github-actions/deploy-docs@v1
Run actions/setup-python@v3
Successfully setup CPython (3.8.12)
Run pip3 install -U wheel
Collecting wheel
  Downloading wheel-0.37.1-py2.py3-none-any.whl ([35](https://github.com/autowarefoundation/autoware-documentation/runs/6380953350?check_suite_focus=true#step:3:35) kB)
Installing collected packages: wheel
Successfully installed wheel-0.37.1
Collecting fontawesome_markdown
  Downloading fontawesome-markdown-0.2.6.tar.gz (5.8 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting markdown
  Downloading Markdown-3.3.7-py3-none-any.whl (97 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.8/97.8 KB 9.6 MB/s eta 0:00:00
Collecting mdx_truly_sane_lists
  Downloading mdx_truly_sane_lists-1.2-py3-none-any.whl (5.0 kB)
Collecting mdx_unimoji
  Downloading mdx_unimoji-1.0.tar.gz (3.9 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'error'
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [20 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-cpyge50x/mdx-unimoji_dccb6da62ad2476f8124620208c0737f/setup.py", line 10, in <module>
          setup(
        File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
          return distutils.core.setup(**attrs)
        File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/distutils/core.py", line 108, in setup
          _setup_distribution = dist = klass(attrs)
        File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/setuptools/dist.py", line [43](https://github.com/autowarefoundation/autoware-documentation/runs/6380953350?check_suite_focus=true#step:3:43)4, in __init__
          _Distribution.__init__(self, {
        File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/distutils/dist.py", line 292, in __init__
          self.finalize_options()
        File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/setuptools/dist.py", line 743, in finalize_options
          ep(self)
        File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/setuptools/dist.py", line 750, in _finalize_setup_keywords
          ep.load()(self, ep.name, value)
        File "/tmp/pip-install-cpyge50x/mdx-unimoji_dccb6da62ad2476f812[46](https://github.com/autowarefoundation/autoware-documentation/runs/6380953350?check_suite_focus=true#step:3:46)20208c0737f/.eggs/setuptools_markdown-0.4.1-py3.8.egg/setuptools_markdown.py", line 43, in long_description_markdown_filename
          output = pypandoc.convert(markdown_filename, 'rst', format='md')
      AttributeError: module 'pypandoc' has no attribute 'convert'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
Error: Process completed with exit code 1.

Here is the PR: autowarefoundation/autoware-documentation#103.
(It appears that the same issue is happening in autowarefoundation/autoware-documentation#104 too)

I understood that the error occurs in installing these python packages, but can not come up with any solution.

Does anyone have any idea how to solve this issue?
Thank you in advance.

[shmpwk]

It looks this line causes error.
https://github.com/msabramo/setuptools-markdown/blob/34ed6276e0c8bcefcd967bd8db138546a270bdf4/setuptools_markdown.py#L43

pypandoc does not have convert function. https://github.com/NicklasTegner/pypandoc/blob/master/pypandoc/__init__.py
pypandoc only has convert_file or convert_text.

Actually pypandoc previously have convert function like https://github.com/NicklasTegner/pypandoc/blob/c06d7b64a58729fa174b7721dac6c600fad166af/pypandoc/__init__.py

I created PR msabramo/setuptools-markdown#21 but the error cannot be helped until the PR will merge and pypi will update.

FYI, there is a similar issue man-group/pytest-plugins#87.

[shmpwk]

Ok, I will, thank you.

[shmpwk]

I created and merged those PR.

• https://github.com/autowarefoundation/autoware-documentation -> mkdocs.yaml
  • ci(deploy-docs): remove mdx_unimoji autoware-documentation#105
• https://github.com/autowarefoundation/autoware.universe -> mkdocs.yaml
  • ci(deploy-docs): remove mdx_unimoji autoware.universe#883
• https://github.com/autowarefoundation/autoware-github-actions -> deploy-docs/mkdocs-requirements.txt
  • fix(deploy-docs): remove mdx_unimoji autoware-github-actions#146

Thank you for your review @kenji-miyake.

[kenji-miyake]

Please wait for the next release of autoware-github-actions. 🙏
It would be within about 30 minutes.

[kenji-miyake]

https://github.com/autowarefoundation/autoware-github-actions/releases/tag/v1.7.1

[shmpwk]

Thank you!
deploy-docs has passed! autowarefoundation/autoware-documentation#103
cc. @kminoda

[shmpwk]

https://github.com/orgs/autowarefoundation/discussions/286#discussioncomment-2727099
↑To sum up the comment above, fixing ci solved the problem by removing mdx_unimoji dependency.
See autowarefoundation/autoware-documentation#105

You can check the ci has passed here.

[JessicaTegner]

Hi there.

I'm the maintainer of PyPandoc, which seems to be the root error in ops post.

In the recent release of PyPandoc (version 1.8), the convert function was removed after a year of being deprecated.

[kenji-miyake]

Thank you for notifying that! 🙏

[shmpwk]

Thank you @NicklasTegner
We decided to temporarily remove PyPandoc depended library.
But when those libraries are updated, we might use them again.

[JessicaTegner]

@shmpwk @kenji-miyake

The solution, to get it to work again, is to switch from pypandoc.convert() to pypandoc.convert_file() or pypandoc.convert_text(), depending on the usage case.

[shmpwk]

Thank you for your advice.
Yes I created such PRs but they are not our libraries.

msabramo/setuptools-markdown#21
man-group/pytest-plugins#205

We hope they’ll be merged.