Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[jsk_interactive_marker] transformable_markers_client.py fails without chainer #800

Open
Naoki-Hiraoka opened this issue Jan 13, 2021 · 8 comments
Open

[jsk_interactive_marker] transformable_markers_client.py fails without chainer #800

Naoki-Hiraoka opened this issue Jan 13, 2021 · 8 comments

Comments

@Naoki-Hiraoka
Copy link
Contributor

Naoki-Hiraoka commented Jan 13, 2021 •
edited
Loading

When I executed transformable_markers_client.py through roslaunch jsk_interactive_markersample_transformable_markers_client.launch on melodic, I got an error.

Traceback (most recent call last):
  File "/opt/ros/melodic/share/jsk_interactive_marker/scripts/transformable_markers_client.py", line 23, in <module>
    from jsk_recognition_utils.color import labelcolormap
  File "/opt/ros/melodic/lib/python2.7/dist-packages/jsk_recognition_utils/__init__.py", line 1, in <module>
    from jsk_recognition_utils import chainermodels
  File "/opt/ros/melodic/lib/python2.7/dist-packages/jsk_recognition_utils/chainermodels/__init__.py", line 1, in <module>
    from jsk_recognition_utils.chainermodels import alexnet
  File "/opt/ros/melodic/lib/python2.7/dist-packages/jsk_recognition_utils/chainermodels/alexnet.py", line 1, in <module>
    import chainer
ImportError: No module named chainer
[transformable_markers_client-3] process has died [pid 29880, exit code 1, cmd /opt/ros/melodic/share/jsk_interactive_marker/scripts/transformable_markers_client.py ~server:=transformable_interactive_server __name:=transformable_markers_client __log:=/home/hiraoka/.ros/log/77914342-5557-11eb-a283-3c7c3f4ccfd6/transformable_markers_client-3.log].

I am not a chainer user and chainer is not automatically installed by default, so chainer is not installed on my PC now.

I am not familiar with installation procedure of chainer, and simple sudo pip install chainer did not worked on my melodic PC.

sudo pip install chainer
  Downloading https://files.pythonhosted.org/packages/a8/ba/32b704e077cb24b4d85260512a5af903e772f06fb58e716301dd51758869/chainer-7.0.0.tar.gz (1.0MB)
    6% |██                              | 61kB 4.3kB/s eta 0:03:40Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 342, in run
    requirement_set.prepare_files(finder)
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 620, in _prepare_file
    session=self.session, hashes=hashes)
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 821, in unpack_url
    hashes=hashes
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 659, in unpack_http_url
    hashes)
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 902, in _download_http_url
    _download_url(resp, link, content_file, hashes)
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 603, in _download_url
    hashes.check_against_chunks(downloaded_chunks)
  File "/usr/lib/python2.7/dist-packages/pip/utils/hashes.py", line 46, in check_against_chunks
    for chunk in chunks:
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 571, in written_chunks
    for chunk in chunks:
  File "/usr/lib/python2.7/dist-packages/pip/utils/ui.py", line 139, in iter
    for x in it:
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 560, in resp_read
    decode_content=False):
  File "/usr/share/python-wheels/urllib3-1.22-py2.py3-none-any.whl/urllib3/response.py", line 436, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/usr/share/python-wheels/urllib3-1.22-py2.py3-none-any.whl/urllib3/response.py", line 401, in read
    raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
  File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/share/python-wheels/urllib3-1.22-py2.py3-none-any.whl/urllib3/response.py", line 316, in _error_catcher
    raise ReadTimeoutError(self._pool, None, 'Read timed out.')
ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.

Does anyone know how to use transformable_markers_client.py?
@Naoki-Hiraoka
Copy link
Contributor Author

I think transformable_markers_client.py is not a chainer specific software so transformable_markers_client.py should not depend on chainer.(?)

@pazeshun
Copy link
Contributor

pazeshun commented Jan 13, 2021 •
edited
Loading

Currently, chainer should be installed by sudo pip install chainer<7.0.0 on Python2.
See chainer/chainer#8545 for more details.
However, you may not be able to install even with this command because your error seems caused by a network problem.
Your network or pip server may have some temporary problem.

@pazeshun
Copy link
Contributor

I think transformable_markers_client.py is not a chainer specific software so transformable_markers_client.py should not depend on chainer.(?)

This is a complicated problem.
transformable_markers_client.py only uses jsk_recognition_utils.color.labelcolormap:

jsk_visualization/jsk_interactive_markers/jsk_interactive_marker/scripts/transformable_markers_client.py

Line 23 in e308bd7

from jsk_recognition_utils.color import labelcolormap

jsk_visualization/jsk_interactive_markers/jsk_interactive_marker/scripts/transformable_markers_client.py

Line 67 in e308bd7

cmap = labelcolormap(n_boxes)

But while it is imported, __init__.py of jsk_recognition_utils is also executed and it imports chainer-dependent feature:
https://github.com/jsk-ros-pkg/jsk_recognition/blob/2c1400e15d79b4f5f9fc7e6784a58bb277ebd8e3/jsk_recognition_utils/python/jsk_recognition_utils/__init__.py#L1

When we want to fix this, we have to check all software using import jsk_recognition_utils because this software may use features automatically imported by __init__.py of jsk_recognition_utils.
That is too troublesome...

@knorth55 knorth55 mentioned this issue Jan 13, 2021
Merged
@knorth55
Copy link
Member

@Naoki-Hiraoka can you try this PR? jsk-ros-pkg/jsk_recognition#2565

@Naoki-Hiraoka
Copy link
Contributor Author

@pazeshun @knorth55
Thank you for your help!

With jsk-ros-pkg/jsk_recognition#2565 , transformable_markers_client.py runs successfully without chainer.

$ roslaunch jsk_interactive_marker sample_transformable_markers_client.launch
...(omitted)...
Please install chainer<7.0.0 to import chainermodels
...(omitted)...

@Naoki-Hiraoka
Copy link
Contributor Author

Currently, chainer should be installed by sudo pip install chainer<7.0.0 on Python2.
See chainer/chainer#8545 for more details.
However, you may not be able to install even with this command because your error seems caused by a network problem.
Your network or pip server may have some temporary problem.

sudo pip install chainer<7.0.0 does not worked. My network or pip server may have some temporary problem as you said.
(Since I can use transformable_markers_client.py with jsk-ros-pkg/jsk_recognition#2565 now, this has become not an urgent problem for me now.)

$ sudo pip install chainer==6.7.0
Collecting chainer==6.7.0
  Downloading https://files.pythonhosted.org/packages/57/c7/8703e3d03825d51a80919e250f6aa5c88f181b8b5cf3cac0d7761dfe0f6d/chainer-6.7.0.tar.gz (876kB)
    3% |█▏                              | 30kB 19.6MB/s eta 0:00:01Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 342, in run
    requirement_set.prepare_files(finder)
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 620, in _prepare_file
    session=self.session, hashes=hashes)
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 821, in unpack_url
    hashes=hashes
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 659, in unpack_http_url
    hashes)
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 902, in _download_http_url
    _download_url(resp, link, content_file, hashes)
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 603, in _download_url
    hashes.check_against_chunks(downloaded_chunks)
  File "/usr/lib/python2.7/dist-packages/pip/utils/hashes.py", line 46, in check_against_chunks
    for chunk in chunks:
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 571, in written_chunks
    for chunk in chunks:
  File "/usr/lib/python2.7/dist-packages/pip/utils/ui.py", line 139, in iter
    for x in it:
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 560, in resp_read
    decode_content=False):
  File "/usr/share/python-wheels/urllib3-1.22-py2.py3-none-any.whl/urllib3/response.py", line 436, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/usr/share/python-wheels/urllib3-1.22-py2.py3-none-any.whl/urllib3/response.py", line 401, in read
    raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
  File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/share/python-wheels/urllib3-1.22-py2.py3-none-any.whl/urllib3/response.py", line 316, in _error_catcher
    raise ReadTimeoutError(self._pool, None, 'Read timed out.')
ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.

@pazeshun
Copy link
Contributor

@knorth55 Thank you for your nice workaround!
@Naoki-Hiraoka Thank you for testing.
I can download and install chainer==6.7.0 now, so pip server should be OK.
Your environment probably includes something wrong, I think.

$ pip --version
pip 20.0.2 from /usr/local/lib/python2.7/dist-packages/pip (python 2.7)
$ pip install chainer==6.7.0 --user --no-cache-dir
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/__init__.py:83: RequestsDependencyWarning: Old version of cryptography ([1, 2, 3]) may cause slowdown.
  warnings.warn(warning, RequestsDependencyWarning)
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Collecting chainer==6.7.0
  Downloading chainer-6.7.0.tar.gz (876 kB)
     |▍                               | 10 kB 13.7 MB/s eta 0:00:0     |▊                               | 20 kB 13.0 MB/s eta 0:00:0     |███▊                            | 102 kB 2.6 MB/s eta 0:00:0     |████                            | 112 kB 2.6 MB/s eta 0:00:0     |████▌                           | 122 kB 2.6 MB/s eta 0:00:0     |████▉                           | 133 kB 2.6 MB/s eta 0:00:0     |█████▎                          | 143 kB 2.6 MB/s eta 0:00:0     |█████▋                          | 153 kB 2.6 MB/s eta 0:00:0     |██████                          | 163 kB 2.6 MB/s eta 0:00:0     |██████▍                         | 174 kB 2.6 MB/s eta 0:00:0     |██████▊                         | 184 kB 2.6 MB/s eta 0:00:0     |███████                         | 194 kB 2.6 MB/s eta 0:00:0     |███████▌                        | 204 kB 2.6 MB/s eta 0:00:0     |███████▉                        | 215 kB 2.6 MB/s eta 0:00:0     |████████▏                       | 225 kB 2.6 MB/s eta 0:00:0     |████████▋                       | 235 kB 2.6 MB/s eta 0:00:0     |█████████                       | 245 kB 2.6 MB/s eta 0:00:0     |█████████▍                      | 256 kB 2.6 MB/s eta 0:00:0     |█████████▊                      | 266 kB 2.6 MB/s eta 0:00:0     |██████████                      | 276 kB 2.6 MB/s eta 0:00:0     |██████████▌                     | 286 kB 2.6 MB/s eta 0:00:0     |██████████▉                     | 296 kB 2.6 MB/s eta 0:00:0     |███████████▏                    | 307 kB 2.6 MB/s eta 0:00:0     |███████████▋                    | 317 kB 2.6 MB/s eta 0:00:0     |████████████                    | 327 kB 2.6 MB/s eta 0:00:0     |████████████▎                   | 337 kB 2.6 MB/s eta 0:00:0     |████████████▊                   | 348 kB 2.6 MB/s eta 0:00:0     |█████████████                   | 358 kB 2.6 MB/s eta 0:00:0     |█████████████▌                  | 368 kB 2.6 MB/s eta 0:00:0     |█████████████▉                  | 378 kB 2.6 MB/s eta 0:00:0     |██████████████▏                 | 389 kB 2.6 MB/s eta 0:00:0     |██████████████▋                 | 399 kB 2.6 MB/s eta 0:00:0     |███████████████                 | 409 kB 2.6 MB/s eta 0:00:0     |███████████████▎                | 419 kB 2.6 MB/s eta 0:00:0     |███████████████▊                | 430 kB 2.6 MB/s eta 0:00:0     |████████████████                | 440 kB 2.6 MB/s eta 0:00:0     |████████████████▍               | 450 kB 2.6 MB/s eta 0:00:0     |████████████████▉               | 460 kB 2.6 MB/s eta 0:00:0     |█████████████████▏              | 471 kB 2.6 MB/s eta 0:00:0     |█████████████████▋              | 481 kB 2.6 MB/s eta 0:00:0     |██████████████████              | 491 kB 2.6 MB/s eta 0:00:0     |██████████████████▎             | 501 kB 2.6 MB/s eta 0:00:0     |██████████████████▊             | 512 kB 2.6 MB/s eta 0:00:0     |███████████████████             | 522 kB 2.6 MB/s eta 0:00:0     |███████████████████▍            | 532 kB 2.6 MB/s eta 0:00:0     |███████████████████▉            | 542 kB 2.6 MB/s eta 0:00:0     |████████████████████▏           | 552 kB 2.6 MB/s eta 0:00:0     |████████████████████▌           | 563 kB 2.6 MB/s eta 0:00:0     |█████████████████████           | 573 kB 2.6 MB/s eta 0:00:0     |█████████████████████▎          | 583 kB 2.6 MB/s eta 0:00:0     |█████████████████████▊          | 593 kB 2.6 MB/s eta 0:00:0     |██████████████████████          | 604 kB 2.6 MB/s eta 0:00:0     |██████████████████████▍         | 614 kB 2.6 MB/s eta 0:00:0     |██████████████████████▉         | 624 kB 2.6 MB/s eta 0:00:0     |███████████████████████▏        | 634 kB 2.6 MB/s eta 0:00:0     |███████████████████████▌        | 645 kB 2.6 MB/s eta 0:00:0     |████████████████████████        | 655 kB 2.6 MB/s eta 0:00:0     |████████████████████████▎       | 665 kB 2.6 MB/s eta 0:00:0     |████████████████████████▋       | 675 kB 2.6 MB/s eta 0:00:0     |█████████████████████████       | 686 kB 2.6 MB/s eta 0:00:0     |█████████████████████████▍      | 696 kB 2.6 MB/s eta 0:00:0     |█████████████████████████▉      | 706 kB 2.6 MB/s eta 0:00:0     |██████████████████████████▏     | 716 kB 2.6 MB/s eta 0:00:0     |██████████████████████████▌     | 727 kB 2.6 MB/s eta 0:00:0     |███████████████████████████     | 737 kB 2.6 MB/s eta 0:00:0     |███████████████████████████▎    | 747 kB 2.6 MB/s eta 0:00:0     |███████████████████████████▋    | 757 kB 2.6 MB/s eta 0:00:0     |████████████████████████████    | 768 kB 2.6 MB/s eta 0:00:0     |████████████████████████████▍   | 778 kB 2.6 MB/s eta 0:00:0     |████████████████████████████▊   | 788 kB 2.6 MB/s eta 0:00:0     |█████████████████████████████▏  | 798 kB 2.6 MB/s eta 0:00:0     |█████████████████████████████▌  | 808 kB 2.6 MB/s eta 0:00:0     |██████████████████████████████  | 819 kB 2.6 MB/s eta 0:00:0     |██████████████████████████████▎ | 829 kB 2.6 MB/s eta 0:00:0     |██████████████████████████████▋ | 839 kB 2.6 MB/s eta 0:00:0     |███████████████████████████████ | 849 kB 2.6 MB/s eta 0:00:0     |███████████████████████████████▍| 860 kB 2.6 MB/s eta 0:00:0     |███████████████████████████████▊| 870 kB 2.6 MB/s eta 0:00:0     |████████████████████████████████| 876 kB 2.6 MB/s 
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from chainer==6.7.0) (20.7.0)
Requirement already satisfied: typing<=3.6.6 in /usr/local/lib/python2.7/dist-packages (from chainer==6.7.0) (3.6.6)
Requirement already satisfied: typing_extensions<=3.6.6 in /usr/local/lib/python2.7/dist-packages (from chainer==6.7.0) (3.6.6)
Requirement already satisfied: filelock in /usr/local/lib/python2.7/dist-packages (from chainer==6.7.0) (3.0.12)
Requirement already satisfied: numpy>=1.9.0 in /usr/lib/python2.7/dist-packages (from chainer==6.7.0) (1.11.0)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python2.7/dist-packages (from chainer==6.7.0) (3.7.1)
Requirement already satisfied: six>=1.9.0 in /usr/lib/python2.7/dist-packages (from chainer==6.7.0) (1.10.0)
Building wheels for collected packages: chainer
  Building wheel for chainer (setup.py) ... done
  Created wheel for chainer: filename=chainer-6.7.0-py2-none-any.whl size=887810 sha256=874fe0c82c933016c38d49b00c8daa7eb762c8e12bac7c7bf9628b33308c6fa5
  Stored in directory: /tmp/pip-ephem-wheel-cache-e7892l/wheels/2a/79/8c/a36c747a259e068df041501f6101135c04edb121df2876499e
Successfully built chainer
Installing collected packages: chainer
Successfully installed chainer-6.7.0
WARNING: You are using pip version 20.0.2; however, version 20.3.3 is available.
You should consider upgrading via the '/usr/bin/python -m pip install --upgrade pip' command.

@Naoki-Hiraoka
Copy link
Contributor Author

@pazeshun Thank you for information.

My environment

$ pip --version
pip 9.0.1 from /usr/lib/python2.7/dist-packages (python 2.7)
$ apt show python-pip
Package: python-pip
Version: 9.0.1-2.3~ubuntu1.18.04.4
Priority: optional
Section: universe/python
Origin: Ubuntu
Maintainer: Ubuntu Developers <[email protected]>
Original-Maintainer: Debian Python Modules Team <[email protected]>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 673 kB
Depends: ca-certificates, python-pip-whl (= 9.0.1-2.3~ubuntu1.18.04.4), python:any (<< 2.8), python:any (>= 2.7.5-5~)
Recommends: build-essential, python-all-dev (>= 2.6), python-setuptools, python-wheel
Homepage: https://pip.pypa.io/en/stable/
Download-Size: 151 kB
APT-Manual-Installed: yes
APT-Sources: http://jp.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages
Description: Python package installer
 pip is the Python package installer.  It integrates with virtualenv, doesn't
 do partial installs, can save package state for replaying, can install from
 non-egg sources, and can install from version control repositories.

Cannot upgrade pip by default.

$ pip install -U pip
Collecting pip
  Downloading https://files.pythonhosted.org/packages/54/eb/4a3642e971f404d69d4f6fa3885559d67562801b99d7592487f1ecc4e017/pip-20.3.3-py2.py3-none-any.whl (1.5MB)
    2% |▋                               | 30kB 35.4MB/s eta 0:00:01Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 353, in run
    wb.build(autobuilding=True)
  File "/usr/lib/python2.7/dist-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 620, in _prepare_file
    session=self.session, hashes=hashes)
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 821, in unpack_url
    hashes=hashes
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 659, in unpack_http_url
    hashes)
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 902, in _download_http_url
    _download_url(resp, link, content_file, hashes)
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 603, in _download_url
    hashes.check_against_chunks(downloaded_chunks)
  File "/usr/lib/python2.7/dist-packages/pip/utils/hashes.py", line 46, in check_against_chunks
    for chunk in chunks:
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 571, in written_chunks
    for chunk in chunks:
  File "/usr/lib/python2.7/dist-packages/pip/utils/ui.py", line 139, in iter
    for x in it:
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 560, in resp_read
    decode_content=False):
  File "/usr/share/python-wheels/urllib3-1.22-py2.py3-none-any.whl/urllib3/response.py", line 436, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/usr/share/python-wheels/urllib3-1.22-py2.py3-none-any.whl/urllib3/response.py", line 401, in read
    raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
  File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/share/python-wheels/urllib3-1.22-py2.py3-none-any.whl/urllib3/response.py", line 316, in _error_catcher
    raise ReadTimeoutError(self._pool, None, 'Read timed out.')
ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out

--default-timeout=100 option solved this problem.
https://stackoverflow.com/questions/43298872/how-to-solve-readtimeouterror-httpsconnectionpoolhost-pypi-python-org-port

$ sudo pip install chainer==6.7.0 --default-timeout=100
$ sudo pip install -U pip --default-timeout=100

In additon to chainer, I found that jsk_recognition_utils/chainermodels also depends on fcn.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@knorth55 @pazeshun @Naoki-Hiraoka