Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mopidy run in docker on armhf hangs with 100% CPU usage #402

Open
mczerski opened this issue Sep 28, 2024 · 11 comments
Open

mopidy run in docker on armhf hangs with 100% CPU usage #402

mczerski opened this issue Sep 28, 2024 · 11 comments

Comments

@mczerski
Copy link

I have a armhf armbian bullseye system with mopidy in docker. It was WORKING with gst-plugin-spotify v0.9.1.
After the issue described here happend I had to upgrade gst-plugin-spotify to v0.13.1. Now it is NOT WORKING. I can go to web interface, select spotify song to play and hit play, but after that mopidy process goes 100% CPU and nothing else happens. Anyone has the same issue ?
I created the minimal deockerfile (below):
Dockerfile:

FROM arm32v7/debian:bullseye-slim

RUN apt update && \
    apt install -y \
        mopidy \
        python3-pip

RUN pip install \
    mopidy \
    mopidy-spotify==5.0.0a2 \
    Mopidy-MusicBox-Webclient

COPY gst-plugin-spotify_0.13.1-1_armhf.deb /
RUN dpkg -i /gst-plugin-spotify_0.13.1-1_armhf.deb

COPY mopidy.conf /etc/mopidy/
RUN mkdir -p /var/lib/mopidy/spotify/credentials-cache/
COPY credentials.json /var/lib/mopidy/spotify/credentials-cache/

CMD ["mopidy", "--config", "/etc/mopidy/mopidy.conf"]

mopidy.conf:

[http]
enabled = true
hostname = 0.0.0.0

[spotify]
enabled = true
username = XXX
password = XXX
client_id = XXX
client_secret = XXX
allow_cache = false

docker build --network=host -t test-mopidy-spotify .
docker run -d --rm --network host --name test-mopidy-spotify test-mopidy-spotify
@kingosticks
Copy link
Member

Can you provide a log ?

@mczerski
Copy link
Author

nothing important in mopidy logs.
In gstreamer logs, with GST_DEBUG=3,spotify:6, after playing the song:

0:00:28.190274119     8  0x15607a0 DEBUG        spotifyaudiosrc audio/spotify/src/spotifyaudiosrc/imp.rs:354:<gstspotify::spotifyaudiosrc::imp::SpotifyAudioSrc as gstreamer::subclass::uri_handler::URIHandlerImpl>::set_uri:<source> set URI: spotify:track:71fSn6OIF7HnCbgCzOBqpj
0:00:28.197646613     8 0xf1605380 DEBUG        spotifyaudiosrc audio/spotify/src/common.rs:144:gstspotify::common::Settings::connect_session::{{closure}}:<source> reuse cached credentials for user xzgfvdc9r165xst3txjd0zzua
0:00:28.199231912     8  0x14e4370 WARN                    alsa conf.c:5200:snd_config_expand: alsalib error: Unknown parameters {AES0 0x02 AES1 0x82 AES2 0x00 AES3 0x02}
0:00:28.199311787     8  0x14e4370 WARN                    alsa pcm.c:2660:snd_pcm_open_noupdate: alsalib error: Unknown PCM default:{AES0 0x02 AES1 0x82 AES2 0x00 AES3 0x02}

@mczerski
Copy link
Author

also i'm watching cpu usage with htop. it shows that most of the usage is RED - Kernel threads

@kingosticks
Copy link
Member

And non-spotify tracks don't have this issue? Do they still have the alsa param errors?

@mczerski
Copy link
Author

local files are working and produce such log

0:00:19.831573348     8  0x10fa7a0 WARN                 basesrc gstbasesrc.c:3688:gst_base_src_start_complete:<source> pad not activated yet
0:00:19.837045208     8  0x10fa7a0 WARN                 basesrc gstbasesrc.c:3688:gst_base_src_start_complete:<source> pad not activated yet
0:00:19.849541519     8  0x107e370 WARN                    alsa conf.c:5200:snd_config_expand: alsalib error: Unknown parameters {AES0 0x02 AES1 0x82 AES2 0x00 AES3 0x02}
0:00:19.849691269     8  0x107e370 WARN                    alsa pcm.c:2660:snd_pcm_open_noupdate: alsalib error: Unknown PCM default:{AES0 0x02 AES1 0x82 AES2 0x00 AES3 0x02}
0:00:19.869566657     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:141:gst_tag_from_id3_tag: Cannot map ID3v2 tag 'TCMP' to GStreamer tag
0:00:19.869673616     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:141:gst_tag_from_id3_tag: Cannot map ID3v2 tag 'TMED' to GStreamer tag
0:00:19.869730491     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:141:gst_tag_from_id3_tag: Cannot map ID3v2 tag 'TLAN' to GStreamer tag
0:00:19.869759492     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:141:gst_tag_from_id3_tag: Cannot map ID3v2 tag 'TDOR' to GStreamer tag
0:00:19.869785325     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:141:gst_tag_from_id3_tag: Cannot map ID3v2 tag 'USLT' to GStreamer tag
0:00:19.869937492     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:141:gst_tag_from_id3_tag: Cannot map ID3v2 tag 'TIPL' to GStreamer tag
0:00:19.869970909     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:141:gst_tag_from_id3_tag: Cannot map ID3v2 tag 'TXXX' to GStreamer tag
0:00:19.870002826     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:213:gst_tag_from_id3_user_tag: Cannot map ID3v2 user tag 'Script' of type 'TXXX' to GStreamer tag
0:00:19.870073868     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:141:gst_tag_from_id3_tag: Cannot map ID3v2 tag 'TXXX' to GStreamer tag
0:00:19.870097076     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:213:gst_tag_from_id3_user_tag: Cannot map ID3v2 user tag 'ASIN' of type 'TXXX' to GStreamer tag
0:00:19.870124993     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:141:gst_tag_from_id3_tag: Cannot map ID3v2 tag 'TXXX' to GStreamer tag
0:00:19.870192910     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:213:gst_tag_from_id3_user_tag: Cannot map ID3v2 user tag 'Artist Credit' of type 'TXXX' to GStreamer tag
0:00:19.870246827     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:141:gst_tag_from_id3_tag: Cannot map ID3v2 tag 'TXXX' to GStreamer tag
0:00:19.870269994     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:213:gst_tag_from_id3_user_tag: Cannot map ID3v2 user tag 'ALBUMARTISTSORT' of type 'TXXX' to GStreamer tag
0:00:19.870435495     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:141:gst_tag_from_id3_tag: Cannot map ID3v2 tag 'TXXX' to GStreamer tag
0:00:19.870463912     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:213:gst_tag_from_id3_user_tag: Cannot map ID3v2 user tag 'Album Artist Credit' of type 'TXXX' to GStreamer tag
0:00:19.870514287     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:141:gst_tag_from_id3_tag: Cannot map ID3v2 tag 'TXXX' to GStreamer tag
0:00:19.870538662     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:213:gst_tag_from_id3_user_tag: Cannot map ID3v2 user tag 'MusicBrainz Album Status' of type 'TXXX' to GStreamer tag
0:00:19.870589746     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:141:gst_tag_from_id3_tag: Cannot map ID3v2 tag 'TXXX' to GStreamer tag
0:00:19.870614912     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:213:gst_tag_from_id3_user_tag: Cannot map ID3v2 user tag 'MusicBrainz Album Type' of type 'TXXX' to GStreamer tag
0:00:19.870644996     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:141:gst_tag_from_id3_tag: Cannot map ID3v2 tag 'TXXX' to GStreamer tag
0:00:19.870667454     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:213:gst_tag_from_id3_user_tag: Cannot map ID3v2 user tag 'MusicBrainz Album Release Country' of type 'TXXX' to GStreamer tag
0:00:19.870770746     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:141:gst_tag_from_id3_tag: Cannot map ID3v2 tag 'TXXX' to GStreamer tag
0:00:19.870937414     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:141:gst_tag_from_id3_tag: Cannot map ID3v2 tag 'UFID' to GStreamer tag
0:00:19.870981706     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:141:gst_tag_from_id3_tag: Cannot map ID3v2 tag 'TXXX' to GStreamer tag
0:00:19.871013748     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:141:gst_tag_from_id3_tag: Cannot map ID3v2 tag 'TXXX' to GStreamer tag
0:00:19.871043873     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:141:gst_tag_from_id3_tag: Cannot map ID3v2 tag 'TXXX' to GStreamer tag
0:00:19.871075373     8 0xf2a056c0 FIXME                  id3v2 gstid3tag.c:141:gst_tag_from_id3_tag: Cannot map ID3v2 tag 'TXXX' to GStreamer tag
0:00:19.935132641     8 0xf2009598 WARN         audio-resampler audio-resampler.c:274:convert_taps_gint16_c: can't find exact taps
0:00:19.941701465     8 0xf2a5cc58 WARN                    alsa pcm_hw.c:1359:snd_pcm_hw_get_chmap: alsalib error: Cannot read Channel Map ctl

@mczerski
Copy link
Author

I added htop and gdb to docker image to track down what is going on, while the CPU goes 100% this is the state:
htop:
mopidy_spotify_htop
gdb:

(gdb) c
Continuing.
^C
Thread 1 "mopidy" received signal SIGINT, Interrupt.
[Switching to Thread 0xf71ae310 (LWP 1)]
__libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
46	in ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S
(gdb) info thread
  Id   Target Id                                    Frame 
* 1    Thread 0xf71ae310 (LWP 1) "mopidy"           __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
  2    Thread 0xf47ba460 (LWP 10) "gmain"           __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
  3    Thread 0xf3dff460 (LWP 11) "mopidy"          __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
  4    Thread 0xf33ff460 (LWP 12) "mopidy"          __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
  5    Thread 0xf29ff460 (LWP 13) "mopidy"          __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
  6    Thread 0xf1fff460 (LWP 14) "mopidy"          __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
  7    Thread 0xf15ff460 (LWP 15) "mopidy"          __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
  8    Thread 0xf0bff460 (LWP 16) "mopidy"          __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
  9    Thread 0xef7fe460 (LWP 24) "mopidy"          __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
  10   Thread 0xeeffd460 (LWP 25) "mopidy"          __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
  11   Thread 0xee7fc460 (LWP 26) "mopidy"          __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:47
  14   Thread 0xeffff460 (LWP 44) "mopidy"          0xed64d908 in ?? () from /usr/lib/arm-linux-gnueabihf/gstreamer-1.0/libgstspotify.so
  15   Thread 0xed431460 (LWP 45) "source:src"      __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
  16   Thread 0xecc30460 (LWP 46) "tokio-runtime-w" __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:47
  17   Thread 0xec9d6460 (LWP 47) "source:src"      syscall () at ../sysdeps/unix/sysv/linux/arm/syscall.S:37
(gdb) thread 14
[Switching to thread 14 (Thread 0xeffff460 (LWP 44))]
#0  0xed64d908 in ?? () from /usr/lib/arm-linux-gnueabihf/gstreamer-1.0/libgstspotify.so
(gdb) bt
#0  0xed64d908 in ?? () from /usr/lib/arm-linux-gnueabihf/gstreamer-1.0/libgstspotify.so
#1  0x00000000 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) 

``

@mczerski
Copy link
Author

I can reproduce the issue with gst-launch command.

First something that works (http stream):

# gst-launch-1.0 playbin uri=http://mp3.polskieradio.pl:8900/
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Got context from element 'source': gst.soup.session=context, session=(SoupSession)NULL, force=(boolean)false;
Redistribute latency...
Pipeline is PREROLLED ...
Prerolled, waiting for buffering to finish...
Setting pipeline to PLAYING ...
New clock: GstAudioSinkClock

and spotify that hangs with 100% CPU usage:

# gst-launch-1.0 playbin uri=spotify:track:3i3P1mGpV9eRlfKccjDjwi?username=$USERNAME\&password=$PASSWORD
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...

On the other armhf system, without docker:

# gst-launch-1.0 playbin uri=spotify:track:3i3P1mGpV9eRlfKccjDjwi?username=$USERNAME\&password=$PASSWORD
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
ERROR: from element /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstSpotifyAudioSrc:source: Could not get/set settings from/on resource.
Additional debug info:
audio/spotify/src/spotifyaudiosrc/imp.rs(280): <gstspotify::spotifyaudiosrc::imp::SpotifyAudioSrc as gstreamer_base::subclass::push_src::PushSrcImpl>::create (): /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstSpotifyAudioSrc:source:
Permission denied { Login failed with reason: Bad credentials }
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
ERROR: from element /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstSpotifyAudioSrc:source: Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstSpotifyAudioSrc:source:
streaming stopped, reason error (-5)
ERROR: pipeline doesn't want to preroll.
Freeing pipeline ...

This does not hang and fails with credentials issue as expected.

@mczerski
Copy link
Author

and log with GST_DEBUG=5:
log.txt

after launching the command this is the part of log that keeps repeating endlessly:

0:00:00.405050467 15195 0xac544200 DEBUG                  query gstquery.c:679:gst_query_new_custom: creating new query 0xac611120 position
0:00:00.405152633 15195 0xac544200 DEBUG       GST_ELEMENT_PADS gstelement.c:2051:gst_element_query: send query on element playbin0
0:00:00.405217675 15195 0xac544200 DEBUG                    bin gstbin.c:4337:gst_bin_query:<playbin0> Sending query 0xac611120 (type position) to sink children
0:00:00.405304300 15195 0xac544200 DEBUG             GST_STATES gstbin.c:1996:bin_element_is_sink:<playbin0> child uridecodebin0 is not sink
0:00:00.405341549 15195 0xac544200 DEBUG             GST_STATES gstbin.c:1996:bin_element_is_sink:<playbin0> child playsink is sink
0:00:00.405371383 15195 0xac544200 DEBUG       GST_ELEMENT_PADS gstelement.c:2051:gst_element_query: send query on element playsink
0:00:00.405408883 15195 0xac544200 DEBUG                    bin gstbin.c:4337:gst_bin_query:<playsink> Sending query 0xac611120 (type position) to sink children
0:00:00.405441424 15195 0xac544200 DEBUG             GST_STATES gstbin.c:1996:bin_element_is_sink:<playsink> child streamsynchronizer0 is not sink
0:00:00.405479841 15195 0xac544200 DEBUG                    bin gstbin.c:4360:gst_bin_query:<playsink> query 0xac611120 result 0
0:00:00.405518049 15195 0xac544200 DEBUG                    bin gstbin.c:4360:gst_bin_query:<playbin0> query 0xac611120 result 0

@mczerski
Copy link
Author

mczerski commented Oct 6, 2024

I built gst-plugin-spotify v.0.13.2 in debug version. started with rust-gdb and this is what I get after starting gst-launch commad.

(gdb) info thread
  Id   Target Id                                     Frame 
* 1    Thread 0xf77e15f0 (LWP 101) "gst-launch-1.0"  __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
  2    Thread 0xf5e66440 (LWP 104) "gst-launch-1.0"  syscall () at ../sysdeps/unix/sysv/linux/arm/syscall.S:37
  3    Thread 0xf5aff440 (LWP 105) "source:src"      __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
  4    Thread 0xf52fe440 (LWP 106) "tokio-runtime-w" 0xf63266d0 in core::sync::atomic::atomic_compare_exchange_weak<usize> (dst=0xf5b075b8, old=9, new=<optimized out>, 
    success=core::sync::atomic::Ordering::AcqRel, failure=core::sync::atomic::Ordering::Acquire) at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/core/src/sync/atomic.rs:3430
  5    Thread 0xf4eff440 (LWP 107) "gmain"           __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
  6    Thread 0xf44ff440 (LWP 108) "tokio-runtime-w" syscall () at ../sysdeps/unix/sysv/linux/arm/syscall.S:37
  7    Thread 0xf3eff440 (LWP 109) "source:src"      syscall () at ../sysdeps/unix/sysv/linux/arm/syscall.S:37
(gdb) thread 4
[Switching to thread 4 (Thread 0xf52fe440 (LWP 106))]
#0  0xf63266d0 in core::sync::atomic::atomic_compare_exchange_weak<usize> (dst=0xf5b075b8, old=9, new=<optimized out>, success=core::sync::atomic::Ordering::AcqRel, 
    failure=core::sync::atomic::Ordering::Acquire) at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/core/src/sync/atomic.rs:3430
3430	/rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/core/src/sync/atomic.rs: No such file or directory.
(gdb) bt
#0  0xf63266d0 in core::sync::atomic::atomic_compare_exchange_weak<usize> (dst=0xf5b075b8, old=9, new=<optimized out>, success=core::sync::atomic::Ordering::AcqRel, 
    failure=core::sync::atomic::Ordering::Acquire) at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/core/src/sync/atomic.rs:3430
#1  core::sync::atomic::AtomicUsize::compare_exchange_weak (self=0xf5b075b8, current=9, new=<optimized out>, success=core::sync::atomic::Ordering::AcqRel, failure=core::sync::atomic::Ordering::Acquire)
    at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/core/src/sync/atomic.rs:2642
#2  tokio::sync::oneshot::State::set_complete (cell=0xf5b075b8) at src/sync/oneshot.rs:1337
#3  0xf6304f14 in tokio::sync::oneshot::Inner<core::result::Result<http::response::Response<hyper::body::body::Body>, (hyper::error::Error, core::option::Option<http::request::Request<hyper::body::body::Body>>)>>::complete<core::result::Result<http::response::Response<hyper::body::body::Body>, (hyper::error::Error, core::option::Option<http::request::Request<hyper::body::body::Body>>)>> (
    self=0xf5b074f8) at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/sync/oneshot.rs:1126
#4  tokio::sync::oneshot::Sender<core::result::Result<http::response::Response<hyper::body::body::Body>, (hyper::error::Error, core::option::Option<http::request::Request<hyper::body::body::Body>>)>>::send<core::result::Result<http::response::Response<hyper::body::body::Body>, (hyper::error::Error, core::option::Option<http::request::Request<hyper::body::body::Body>>)>> (self=..., t=...)
    at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/sync/oneshot.rs:609
#5  0xf62fbb14 in hyper::client::dispatch::Callback<http::request::Request<hyper::body::body::Body>, http::response::Response<hyper::body::body::Body>>::send<http::request::Request<hyper::body::body::Body>, http::response::Response<hyper::body::body::Body>> (self=..., val=...) at src/client/dispatch.rs:256
#6  0xf61bf7ac in hyper::proto::h1::dispatch::{impl#5}::recv_msg<hyper::body::body::Body> (self=<optimized out>, msg=...)
    at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hyper-0.14.30/src/proto/h1/dispatch.rs:614
#7  0xf61c280c in hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Client<hyper::body::body::Body>, hyper::body::body::Body, tokio::net::tcp::stream::TcpStream, hyper::proto::h1::role::Client>::poll_read_head<hyper::proto::h1::dispatch::Client<hyper::body::body::Body>, hyper::body::body::Body, tokio::net::tcp::stream::TcpStream, hyper::proto::h1::role::Client> (self=0xf5b071f0, 
    cx=<optimized out>) at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hyper-0.14.30/src/proto/h1/dispatch.rs:270
#8  0xf61c40c8 in hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Client<hyper::body::body::Body>, hyper::body::body::Body, tokio::net::tcp::stream::TcpStream, hyper::proto::h1::role::Client>::poll_read<hyper::proto::h1::dispatch::Client<hyper::body::body::Body>, hyper::body::body::Body, tokio::net::tcp::stream::TcpStream, hyper::proto::h1::role::Client> (self=0xf5b071f0, 
    cx=<optimized out>) at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hyper-0.14.30/src/proto/h1/dispatch.rs:191
#9  0xf61c33a8 in hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Client<hyper::body::body::Body>, hyper::body::body::Body, tokio::net::tcp::stream::TcpStream, hyper::proto::h1::role::Client>::poll_loop<hyper::proto::h1::dispatch::Client<hyper::body::body::Body>, hyper::body::body::Body, tokio::net::tcp::stream::TcpStream, hyper::proto::h1::role::Client> (self=0xf5b071f0, 
    cx=0xf52fd948) at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hyper-0.14.30/src/proto/h1/dispatch.rs:163
#10 0xf61c06ac in hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Client<hyper::body::body::Body>, hyper::body::body::Body, tokio::net::tcp::stream::TcpStream, hyper::proto::h1::role::Client>::poll_inner<hyper::proto::h1::dispatch::Client<hyper::body::body::Body>, hyper::body::body::Body, tokio::net::tcp::stream::TcpStream, hyper::proto::h1::role::Client> (self=0xf5b071f0, 
    cx=0xf52fd948, should_shutdown=true) at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hyper-0.14.30/src/proto/h1/dispatch.rs:140
#11 0xf61bfeac in hyper::proto::h1::dispatch::Dispatcher<hyper::proto::h1::dispatch::Client<hyper::body::body::Body>, hyper::body::body::Body, tokio::net::tcp::stream::TcpStream, hyper::proto::h1::role::Client>::poll_catch<hyper::proto::h1::dispatch::Client<hyper::body::body::Body>, hyper::body::body::Body, tokio::net::tcp::stream::TcpStream, hyper::proto::h1::role::Client> (self=0xf5b071f0, cx=0xb, 
    should_shutdown=true) at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hyper-0.14.30/src/proto/h1/dispatch.rs:119
#12 0xf61e7b48 in hyper::proto::h1::dispatch::{impl#1}::poll<hyper::proto::h1::dispatch::Client<hyper::body::body::Body>, hyper::body::body::Body, tokio::net::tcp::stream::TcpStream, hyper::proto::h1::role::Client> (self=..., cx=0xf52fd948) at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hyper-0.14.30/src/proto/h1/dispatch.rs:435
#13 hyper::client::conn::{impl#10}::poll<tokio::net::tcp::stream::TcpStream, hyper::body::body::Body> (cx=0xf52fd948, self=...)
    at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hyper-0.14.30/src/client/conn.rs:1108
#14 hyper::client::conn::{impl#5}::poll<tokio::net::tcp::stream::TcpStream, hyper::body::body::Body> (self=..., cx=0xf52fd948)
    at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hyper-0.14.30/src/client/conn.rs:559
#15 0xf61a8d8c in futures_core::future::{impl#2}::try_poll<hyper::client::conn::Connection<tokio::net::tcp::stream::TcpStream, hyper::body::body::Body>, (), hyper::error::Error> (self=..., cx=0x9)
    at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.30/src/future.rs:82
#16 futures_util::future::try_future::into_future::{impl#2}::poll<hyper::client::conn::Connection<tokio::net::tcp::stream::TcpStream, hyper::body::body::Body>> (self=..., cx=0x9)
    at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/future/try_future/into_future.rs:34
#17 futures_util::future::future::map::{impl#2}::poll<futures_util::future::try_future::into_future::IntoFuture<hyper::client::conn::Connection<tokio::net::tcp::stream::TcpStream, hyper::body::body::Body>>, futures_util::fns::MapErrFn<hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#0}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, core::result::Result<(), ()>> (self=..., cx=0x9)
    at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/future/future/map.rs:55
#18 0xf61a8b30 in futures_util::future::future::map::{impl#2}::poll<futures_util::future::try_future::MapErr<hyper::client::conn::Connection<tokio::net::tcp::stream::TcpStream, hyper::body::body::Body>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#0}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#1}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>, ()> (self=..., cx=0x9) at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/future/future/map.rs:55
#19 0xf61ca500 in tokio::runtime::task::core::{impl#6}::poll::{closure#0}<futures_util::future::future::Map<futures_util::future::try_future::MapErr<hyper::client::conn::Connection<tokio::net::tcp::stream::TcpStream, hyper::body::body::Body>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#0}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#1}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle, alloc::alloc::Global>> (ptr=0xf5b071e8)
    at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/core.rs:331
#20 tokio::loom::std::unsafe_cell::UnsafeCell<tokio::runtime::task::core::Stage<futures_util::future::future::Map<futures_util::future::try_future::MapErr<hyper::client::conn::Connection<tokio::net::tcp:--Type <RET> for more, q to quit, c to continue without paging--
:stream::TcpStream, hyper::body::body::Body>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#0}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#1}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>>>::with_mut<tokio::runtime::task::core::Stage<futures_util::future::future::Map<futures_util::future::try_future::MapErr<hyper::client::conn::Connection<tokio::net::tcp::stream::TcpStream, hyper::body::body::Body>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#0}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#1}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>>, core::task::poll::Poll<()>, tokio::runtime::task::core::{impl#6}::poll::{closure_env#0}<futures_util::future::future::Map<futures_util::future::try_future::MapErr<hyper::client::conn::Connection<tokio::net::tcp::stream::TcpStream, hyper::body::body::Body>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#0}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#1}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle, alloc::alloc::Global>>> (self=0xf5b071e8, f=...)
    at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/loom/std/unsafe_cell.rs:16
#21 tokio::runtime::task::core::Core<futures_util::future::future::Map<futures_util::future::try_future::MapErr<hyper::client::conn::Connection<tokio::net::tcp::stream::TcpStream, hyper::body::body::Body>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#0}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#1}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle, alloc::alloc::Global>>::poll<futures_util::future::future::Map<futures_util::future::try_future::MapErr<hyper::client::conn::Connection<tokio::net::tcp::stream::TcpStream, hyper::body::body::Body>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#0}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#1}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle, alloc::alloc::Global>> (self=0xf5b071d8, cx=...) at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/core.rs:320
#22 0xf61d48a8 in tokio::runtime::task::harness::poll_future::{closure#0}<futures_util::future::future::Map<futures_util::future::try_future::MapErr<hyper::client::conn::Connection<tokio::net::tcp::stream::TcpStream, hyper::body::body::Body>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#0}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#1}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle, alloc::alloc::Global>> ()
    at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/harness.rs:500
#23 core::panic::unwind_safe::{impl#23}::call_once<core::task::poll::Poll<()>, tokio::runtime::task::harness::poll_future::{closure_env#0}<futures_util::future::future::Map<futures_util::future::try_future::MapErr<hyper::client::conn::Connection<tokio::net::tcp::stream::TcpStream, hyper::body::body::Body>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#0}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#1}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle, alloc::alloc::Global>>> (self=...) at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/core/src/panic/unwind_safe.rs:272
#24 std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::{closure_env#0}<futures_util::future::future::Map<futures_util::future::try_future::MapErr<hyper::client::conn::Connection<tokio::net::tcp::stream::TcpStream, hyper::body::body::Body>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#0}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#1}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle, alloc::alloc::Global>>>, core::task::poll::Poll<()>> (data=<optimized out>) at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/panicking.rs:557
#25 std::panicking::try<core::task::poll::Poll<()>, core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::{closure_env#0}<futures_util::future::future::Map<futures_util::future::try_future::MapErr<hyper::client::conn::Connection<tokio::net::tcp::stream::TcpStream, hyper::body::body::Body>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#0}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#1}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle, alloc::alloc::Global>>>> (f=<error reading variable: Cannot access memory at address 0xc>) at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/panicking.rs:521
#26 std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::{closure_env#0}<futures_util::future::future::Map<futures_util::future::try_future::MapErr<hyper::client::conn::Connection<tokio::net::tcp::stream::TcpStream, hyper::body::body::Body>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#0}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#1}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle, alloc::alloc::Global>>>, core::task::poll::Poll<()>> (f=...) at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/panic.rs:350
#27 0xf61b7744 in tokio::runtime::task::harness::poll_future<futures_util::future::future::Map<futures_util::future::try_future::MapErr<hyper::client::conn::Connection<tokio::net::tcp::stream::TcpStream, hyper::body::body::Body>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#0}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#1}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle, alloc::alloc::Global>> (core=0xf5b071d8, cx=...)
    at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/harness.rs:488
#28 0xf61b8aa4 in tokio::runtime::task::harness::Harness<futures_util::future::future::Map<futures_util::future::try_future::MapErr<hyper::client::conn::Connection<tokio::net::tcp::stream::TcpStream, hyper::body::body::Body>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#0}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#1}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle, alloc::alloc::Global>>::poll_inner<futures_util::future::future::Map<futures_util::future::try_future::MapErr<hyper::client::conn::Connection<tokio::net::tcp::stream::TcpStream, hyper::body::body::Body>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{asy--Type <RET> for more, q to quit, c to continue without paging--
nc_block#0}::{closure_env#0}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#1}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle, alloc::alloc::Global>> (self=<optimized out>) at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/harness.rs:209
#29 0xf61bb904 in tokio::runtime::task::harness::Harness<futures_util::future::future::Map<futures_util::future::try_future::MapErr<hyper::client::conn::Connection<tokio::net::tcp::stream::TcpStream, hyper::body::body::Body>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#0}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#1}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle, alloc::alloc::Global>>::poll<futures_util::future::future::Map<futures_util::future::try_future::MapErr<hyper::client::conn::Connection<tokio::net::tcp::stream::TcpStream, hyper::body::body::Body>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#0}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, hyper::client::client::{impl#3}::connect_to::{closure#0}::{closure#0}::{async_block#0}::{closure_env#1}<hyper::client::connect::http::HttpConnector<hyper::client::connect::dns::GaiResolver>, hyper::body::body::Body>>, alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle, alloc::alloc::Global>> (self=...) at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/harness.rs:154
#30 0xf632a578 in tokio::runtime::scheduler::multi_thread::worker::{impl#1}::run_task::{closure#0} () at src/runtime/scheduler/multi_thread/worker.rs:598
#31 tokio::runtime::coop::with_budget<core::result::Result<alloc::boxed::Box<tokio::runtime::scheduler::multi_thread::worker::Core, alloc::alloc::Global>, ()>, tokio::runtime::scheduler::multi_thread::worker::{impl#1}::run_task::{closure_env#0}> (budget=..., f=...) at src/runtime/coop.rs:107
#32 tokio::runtime::coop::budget<core::result::Result<alloc::boxed::Box<tokio::runtime::scheduler::multi_thread::worker::Core, alloc::alloc::Global>, ()>, tokio::runtime::scheduler::multi_thread::worker::{impl#1}::run_task::{closure_env#0}> (f=...) at src/runtime/coop.rs:73
#33 tokio::runtime::scheduler::multi_thread::worker::Context::run_task (self=0xf52fdbac, task=..., core=0xf5b059d8) at src/runtime/scheduler/multi_thread/worker.rs:597
#34 0xf6329448 in tokio::runtime::scheduler::multi_thread::worker::Context::run (self=<optimized out>, core=0xf5b059d8) at src/runtime/scheduler/multi_thread/worker.rs:548
#35 0xf63192cc in tokio::runtime::scheduler::multi_thread::worker::run::{closure#0}::{closure#0} () at src/runtime/scheduler/multi_thread/worker.rs:513
#36 tokio::runtime::context::scoped::Scoped<tokio::runtime::scheduler::Context>::set<tokio::runtime::scheduler::Context, tokio::runtime::scheduler::multi_thread::worker::run::{closure#0}::{closure_env#0}, ()> (self=0xf4500618, t=<optimized out>, f=...) at src/runtime/context/scoped.rs:40
#37 0xf63186ac in tokio::runtime::context::set_scheduler::{closure#0}<(), tokio::runtime::scheduler::multi_thread::worker::run::{closure#0}::{closure_env#0}> (c=0xf45005f8) at src/runtime/context.rs:180
#38 std::thread::local::LocalKey<tokio::runtime::context::Context>::try_with<tokio::runtime::context::Context, tokio::runtime::context::set_scheduler::{closure_env#0}<(), tokio::runtime::scheduler::multi_thread::worker::run::{closure#0}::{closure_env#0}>, ()> (self=<optimized out>, f=...) at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/thread/local.rs:283
#39 std::thread::local::LocalKey<tokio::runtime::context::Context>::with<tokio::runtime::context::Context, tokio::runtime::context::set_scheduler::{closure_env#0}<(), tokio::runtime::scheduler::multi_thread::worker::run::{closure#0}::{closure_env#0}>, ()> (self=<optimized out>, f=...) at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/thread/local.rs:260
#40 0xf631eb6c in tokio::runtime::context::set_scheduler<(), tokio::runtime::scheduler::multi_thread::worker::run::{closure#0}::{closure_env#0}> (v=<optimized out>, f=...) at src/runtime/context.rs:180
#41 0xf6319120 in tokio::runtime::scheduler::multi_thread::worker::run::{closure#0} () at src/runtime/scheduler/multi_thread/worker.rs:508
#42 tokio::runtime::context::runtime::enter_runtime<tokio::runtime::scheduler::multi_thread::worker::run::{closure_env#0}, ()> (handle=<optimized out>, allow_block_in_place=true, f=...)
    at src/runtime/context/runtime.rs:65
#43 0xf6328e54 in tokio::runtime::scheduler::multi_thread::worker::run (worker=...) at src/runtime/scheduler/multi_thread/worker.rs:500
#44 0xf630f83c in tokio::runtime::scheduler::multi_thread::worker::{impl#0}::launch::{closure#0} () at src/runtime/scheduler/multi_thread/worker.rs:466
#45 tokio::runtime::blocking::task::{impl#2}::poll<tokio::runtime::scheduler::multi_thread::worker::{impl#0}::launch::{closure_env#0}, ()> (self=..., _cx=<optimized out>)
    at src/runtime/blocking/task.rs:42
#46 0xf63146e8 in tokio::runtime::task::core::{impl#6}::poll::{closure#0}<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::{impl#0}::launch::{closure_env#0}>, tokio::runtime::blocking::schedule::BlockingSchedule> (ptr=0xf5b05c48) at src/runtime/task/core.rs:331
#47 tokio::loom::std::unsafe_cell::UnsafeCell<tokio::runtime::task::core::Stage<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::{impl#0}::launch::{closure_env#0}>>>::with_mut<tokio::runtime::task::core::Stage<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::{impl#0}::launch::{closure_env#0}>>, core::task::poll::Poll<()>, tokio::runtime::task::core::{impl#6}::poll::{closure_env#0}<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::{impl#0}::launch::{closure_env#0}>, tokio::runtime::blocking::schedule::BlockingSchedule>> (self=0xf5b05c48, f=...) at src/loom/std/unsafe_cell.rs:16
#48 tokio::runtime::task::core::Core<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::{impl#0}::launch::{closure_env#0}>, tokio::runtime::blocking::schedule::BlockingSchedule>::poll<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::{impl#0}::launch::{closure_env#0}>, tokio::runtime::blocking::schedule::BlockingSchedule> (self=0xf5b05c38, cx=...) at src/runtime/task/core.rs:320
#49 0xf631acd0 in tokio::runtime::task::harness::poll_future::{closure#0}<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::{impl#0}::launch::{closure_env#0}>, tokio::runtime::blocking::schedule::BlockingSchedule> () at src/runtime/task/harness.rs:500
#50 core::panic::unwind_safe::{impl#23}::call_once<core::task::poll::Poll<()>, tokio::runtime::task::harness::poll_future::{closure_env#0}<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::{impl#0}::launch::{closure_env#0}>, tokio::runtime::blocking::schedule::BlockingSchedule>> (self=...)
    at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/core/src/panic/unwind_safe.rs:272
#51 std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::{closure_env#0}<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::{impl#0}::launch::{closure_env#0}>, tokio::runtime::blocking::schedule::BlockingSchedule>>, core::task::poll::Poll<()>> (data=<optimized out>)
    at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/panicking.rs:557
#52 std::panicking::try<core::task::poll::Poll<()>, core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::{closure_env#0}<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::{impl#0}::launch::{closure_env#0}>, tokio::runtime::blocking::schedule::BlockingSchedule>>> (
    f=<error reading variable: Cannot access memory at address 0xc>) at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/panicking.rs:521
--Type <RET> for more, q to quit, c to continue without paging--
#53 std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::{closure_env#0}<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::{impl#0}::launch::{closure_env#0}>, tokio::runtime::blocking::schedule::BlockingSchedule>>, core::task::poll::Poll<()>> (f=...)
    at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/panic.rs:350
#54 0xf630b1bc in tokio::runtime::task::harness::poll_future<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::{impl#0}::launch::{closure_env#0}>, tokio::runtime::blocking::schedule::BlockingSchedule> (core=0xf5b05c38, cx=...) at src/runtime/task/harness.rs:488
#55 0xf630aa04 in tokio::runtime::task::harness::Harness<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::{impl#0}::launch::{closure_env#0}>, tokio::runtime::blocking::schedule::BlockingSchedule>::poll_inner<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::{impl#0}::launch::{closure_env#0}>, tokio::runtime::blocking::schedule::BlockingSchedule> (self=<optimized out>) at src/runtime/task/harness.rs:209
#56 0xf630a928 in tokio::runtime::task::harness::Harness<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::{impl#0}::launch::{closure_env#0}>, tokio::runtime::blocking::schedule::BlockingSchedule>::poll<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::{impl#0}::launch::{closure_env#0}>, tokio::runtime::blocking::schedule::BlockingSchedule> (self=...) at src/runtime/task/harness.rs:154
#57 0xf630f18c in tokio::runtime::task::raw::RawTask::poll (self=...) at src/runtime/task/raw.rs:201
#58 tokio::runtime::task::UnownedTask<tokio::runtime::blocking::schedule::BlockingSchedule>::run<tokio::runtime::blocking::schedule::BlockingSchedule> (self=...) at src/runtime/task/mod.rs:473
#59 tokio::runtime::blocking::pool::Task::run (self=...) at src/runtime/blocking/pool.rs:160
#60 tokio::runtime::blocking::pool::Inner::run (self=<optimized out>, worker_thread_id=0) at src/runtime/blocking/pool.rs:518
#61 0xf6314cf4 in tokio::runtime::blocking::pool::{impl#6}::spawn_thread::{closure#0} () at src/runtime/blocking/pool.rs:476
#62 std::sys::backtrace::__rust_begin_short_backtrace<tokio::runtime::blocking::pool::{impl#6}::spawn_thread::{closure_env#0}, ()> (f=...)
    at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/sys/backtrace.rs:152
#63 0xf63120b8 in std::thread::{impl#0}::spawn_unchecked_::{closure#1}::{closure#0}<tokio::runtime::blocking::pool::{impl#6}::spawn_thread::{closure_env#0}, ()> ()
    at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/thread/mod.rs:538
#64 core::panic::unwind_safe::{impl#23}::call_once<(), std::thread::{impl#0}::spawn_unchecked_::{closure#1}::{closure_env#0}<tokio::runtime::blocking::pool::{impl#6}::spawn_thread::{closure_env#0}, ()>>
    (self=<error reading variable: Cannot access memory at address 0x0>) at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/core/src/panic/unwind_safe.rs:272
#65 std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<std::thread::{impl#0}::spawn_unchecked_::{closure#1}::{closure_env#0}<tokio::runtime::blocking::pool::{impl#6}::spawn_thread::{closure_env#0}, ()>>, ()> (data=<optimized out>) at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/panicking.rs:557
#66 std::panicking::try<(), core::panic::unwind_safe::AssertUnwindSafe<std::thread::{impl#0}::spawn_unchecked_::{closure#1}::{closure_env#0}<tokio::runtime::blocking::pool::{impl#6}::spawn_thread::{closure_env#0}, ()>>> (f=<error reading variable: Cannot access memory at address 0x0>) at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/panicking.rs:521
#67 std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<std::thread::{impl#0}::spawn_unchecked_::{closure#1}::{closure_env#0}<tokio::runtime::blocking::pool::{impl#6}::spawn_thread::{closure_env#0}, ()>>, ()> (f=<error reading variable: Cannot access memory at address 0x0>) at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/panic.rs:350
#68 std::thread::{impl#0}::spawn_unchecked_::{closure#1}<tokio::runtime::blocking::pool::{impl#6}::spawn_thread::{closure_env#0}, ()> ()
    at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/thread/mod.rs:537
#69 core::ops::function::FnOnce::call_once<std::thread::{impl#0}::spawn_unchecked_::{closure_env#1}<tokio::runtime::blocking::pool::{impl#6}::spawn_thread::{closure_env#0}, ()>, ()> ()
    at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/core/src/ops/function.rs:250
#70 0xf64da77c in alloc::boxed::{impl#48}::call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:2070
#71 alloc::boxed::{impl#48}::call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:2070
#72 std::sys::pal::unix::thread::{impl#2}::new::thread_start () at library/std/src/sys/pal/unix/thread.rs:108
#73 0xf757b99e in start_thread (arg=0xe2bfcba) at pthread_create.c:477
#74 0xf751402c in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:73 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb)

@kingosticks
Copy link
Member

Do you have the same versions of Gstreamer in Docker and non-Docker armhf setups? Can you try the latest release at https://github.com/kingosticks/gst-plugins-rs-build/releases/tag/gst-plugin-spotify_0.14.0-alpha.1-1 ? I don't think there's anything we can do about this here other than try and work out which upstream to take this to.

@mczerski
Copy link
Author

mczerski commented Oct 7, 2024

Yes, versions are the same. also tried v0.14.0 - same result. Actually on the other host spotify works also in docker so I suspect that this issue is somehow related to the machine rather than docker itself. Unfortunetly I cannot test without docker on this machine (CoreElec)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kingosticks @mczerski