Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installing fails on Debian Stable (buster) with "Cannot locate symbol 'sk_num' in native library 'libssl.so'" #82

Open
zerodogg opened this issue Dec 23, 2020 · 13 comments

Comments

@zerodogg
Copy link

I've installed libssl-dev which provides libssl.so. Without libssl-dev installed I get Cannot locate native library 'libssl.so': libssl.so: cannot open shared object file: No such file or directory instead.

Here's trying to install the module on Debian Stable with rakudo v2020.12.

$ zef install OpenSSL
===> Searching for: OpenSSL
===> Testing: OpenSSL:ver<0.1.23>:auth<github:sergot>
[OpenSSL] # NETWORK_TESTING was not set
[OpenSSL] Cannot locate symbol 'sk_num' in native library 'libssl.so'
[OpenSSL]   in method setup at /opt/rakudo-pkg/share/perl6/core/sources/947BDAB9F96E0E5FCCB383124F923A6BF6F8D76B (NativeCall) line 298
[OpenSSL]   in sub sk_num at /opt/rakudo-pkg/share/perl6/core/sources/947BDAB9F96E0E5FCCB383124F923A6BF6F8D76B (NativeCall) line 587
[OpenSSL]   in block <unit> at t/10-client-ca-file.t line 10
[OpenSSL] # You planned 7 tests, but ran 1
[OpenSSL] # NETWORK_TESTING was not set
===> Testing [FAIL]: OpenSSL:ver<0.1.23>:auth<github:sergot>
Aborting due to test failure: OpenSSL:ver<0.1.23>:auth<github:sergot> (use --force-test to override
@lizmat
Copy link

lizmat commented Mar 25, 2021

I just ran into this as well :-(

@lizmat
Copy link

lizmat commented Mar 25, 2021

Oddly enough, cloning the OpenSSL repo into a local dir, all tests pass. And thus zef install . works like a charm.

@raydiak
Copy link

raydiak commented Apr 15, 2021

I am also encountering this problem on Kubuntu 20.04, where libssl.so is a symlink to libssl.so.1.1

@usev6
Copy link
Contributor

usev6 commented Jun 20, 2021

I've hit the same problem on Debian 10 (buster) that comes with openssl (and libssl) 1.1.1d.

I've used rakudo-pkg for this installation.

$ raku --version
Welcome to Rakudo(tm) v2021.05.
Implementing the Raku(tm) programming language v6.d.
Built on MoarVM version 2021.05.

If I'm not mistaken things work smoothly if libssl-dev is installed before trying to run zef install OpenSSL the first time:

$ sudo apt install libssl-dev
[...]
libssl-dev is already the newest version (1.1.1d-0+deb10u6).
[...]
$ /opt/rakudo-pkg/bin/install-zef
[...]
$ PATH=~/.raku/bin:$PATH
$ zef --version
v0.11.7
$ zef install OpenSSL
===> Searching for: OpenSSL
===> Updating fez mirror: http://360.zef.pm/
===> Updating cpan mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/cpan1.json
===> Updating p6c mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/p6c1.json
===> Updated p6c mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/p6c1.json
===> Updated fez mirror: http://360.zef.pm/
===> Updated cpan mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/cpan1.json
===> Building: OpenSSL:ver<0.1.24>
===> Building [OK] for OpenSSL:ver<0.1.24>
===> Testing: OpenSSL:ver<0.1.24>
[OpenSSL] # NETWORK_TESTING was not set
[OpenSSL] # NETWORK_TESTING was not set
===> Testing [OK] for OpenSSL:ver<0.1.24>
===> Installing: OpenSSL:ver<0.1.24>
$ raku -e 'use OpenSSL; say OpenSSL.new'
OpenSSL.new(ctx => OpenSSL::Ctx::SSL_CTX.new(method => OpenSSL::Method::SSL_METHOD.new(version => 65536)), ssl => OpenSSL::SSL::SSL.new(version => 772, type => 0, method => OpenSSL::Method::SSL_METHOD.new(version => 65536), rbio => OpenSSL::Bio::BIO, wbio => OpenSSL::Bio::BIO, bbio => OpenSSL::Bio::BIO, rwstate => 1, in_handshake => 0, handshake_func => NativeCall::Types::Pointer, server => 1, new_session => 0, quiet_shutdown => 0, shutdown => 0, state => 0, rstate => 0), client => Bool::False, using-bio => Bool::False, bio-read-buf => Buf[uint8].new(), net-write => Any, net-read => Any, net-bio => Any, internal-bio => Any)

Now I nuke the newly installed stuff, remove libssl-dev and start anew. As expected the installation fails with error messages pointing to the missing libssl.so:

$ rm -rf ~/.raku/ ~/.zef/
$ sudo apt -y remove libssl-dev
[...]
Removing libssl-dev:amd64 (1.1.1d-0+deb10u6) ...
$ /opt/rakudo-pkg/bin/install-zef
[...]
$ zef --version
v0.11.7
$ zef install OpenSSL
[...]
===> Testing: OpenSSL:ver<0.1.24>
[OpenSSL] Cannot locate native library 'libssl.so': libssl.so: cannot open shared object file: No such file or directory
[OpenSSL]   in method setup at /opt/rakudo-pkg/share/perl6/core/sources/947BDAB9F96E0E5FCCB383124F923A6BF6F8D76B (NativeCall) line 299
[OpenSSL]   in sub EVP_aes_128_cbc at /opt/rakudo-pkg/share/perl6/core/sources/947BDAB9F96E0E5FCCB383124F923A6BF6F8D76B (NativeCall) line 589
[OpenSSL]   in method new at /home/christian/.zef/store/openssl.git/06ffd7c48970e52d6e261ea3e13601166c7b1364/lib/OpenSSL.pm6 (OpenSSL) line 36
[OpenSSL]   in block <unit> at t/01-basic.t line 6
[...]
===> Testing [FAIL]: OpenSSL:ver<0.1.24>
Aborting due to test failure: OpenSSL:ver<0.1.24> (use --force-test to override)

I'd expect that (re-)installing libssl-dev would fix the problem, but then the complaint about not finding sk_num pops up:

$ sudo apt install libssl-dev
[...]
$ zef install OpenSSL
===> Searching for: OpenSSL
===> Building: OpenSSL:ver<0.1.24>
===> Building [OK] for OpenSSL:ver<0.1.24>
===> Testing: OpenSSL:ver<0.1.24>
[OpenSSL] # NETWORK_TESTING was not set
[OpenSSL] Cannot locate symbol 'sk_num' in native library 'libssl.so'
[OpenSSL]   in method setup at /opt/rakudo-pkg/share/perl6/core/sources/947BDAB9F96E0E5FCCB383124F923A6BF6F8D76B (NativeCall) line 299
[OpenSSL]   in sub sk_num at /opt/rakudo-pkg/share/perl6/core/sources/947BDAB9F96E0E5FCCB383124F923A6BF6F8D76B (NativeCall) line 589
[OpenSSL]   in block <unit> at t/10-client-ca-file.t line 10
[OpenSSL] # You planned 7 tests, but ran 1
[OpenSSL] # NETWORK_TESTING was not set
===> Testing [FAIL]: OpenSSL:ver<0.1.24>
Aborting due to test failure: OpenSSL:ver<0.1.24> (use --force-test to override)

After nuking ~/.raku and ~/.zef the installation works again.

$ rm -rf ~/.raku/ ~/.zef/
$ /opt/rakudo-pkg/bin/install-zef
[...]
$ zef install OpenSSL
===> Searching for: OpenSSL
===> Updating fez mirror: http://360.zef.pm/
===> Updating cpan mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/cpan1.json
===> Updating p6c mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/p6c1.json
===> Updated p6c mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/p6c1.json
===> Updated cpan mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/cpan1.json
===> Updated fez mirror: http://360.zef.pm/
===> Building: OpenSSL:ver<0.1.24>
===> Building [OK] for OpenSSL:ver<0.1.24>
===> Testing: OpenSSL:ver<0.1.24>
[OpenSSL] # NETWORK_TESTING was not set
[OpenSSL] # NETWORK_TESTING was not set
===> Testing [OK] for OpenSSL:ver<0.1.24>
===> Installing: OpenSSL:ver<0.1.24>
$ raku -e 'use OpenSSL; say OpenSSL.new'
OpenSSL.new(ctx => OpenSSL::Ctx::SSL_CTX.new(method => OpenSSL::Method::SSL_METHOD.new(version => 65536)), ssl => OpenSSL::SSL::SSL.new(version => 772, type => 0, method => OpenSSL::Method::SSL_METHOD.new(version => 65536), rbio => OpenSSL::Bio::BIO, wbio => OpenSSL::Bio::BIO, bbio => OpenSSL::Bio::BIO, rwstate => 1, in_handshake => 0, handshake_func => NativeCall::Types::Pointer, server => 1, new_session => 0, quiet_shutdown => 0, shutdown => 0, state => 0, rstate => 0), client => Bool::False, using-bio => Bool::False, bio-read-buf => Buf[uint8].new(), net-write => Any, net-read => Any, net-bio => Any, internal-bio => Any)

So I suspect something is left over from the first (unsuccessful) try. Does that ring a bell for someone?

It would be nice if someone else could verify that things work as expected if libssl-dev is available when trying to run zef install OpenSSL for the first time.

@ugexe
Copy link
Collaborator

ugexe commented Jun 20, 2021

@usev6 Based on your observations I suspect you could reproduce the issue with a local repo (i.e. using zef install . instead of zef install OpenSSL) and then "fix" it by deleting .precomp/ (instead of deleting ~/.zef)

@usev6
Copy link
Contributor

usev6 commented Jun 20, 2021

That was spot on:

$ git clone https://github.com/sergot/openssl.git
$ cd openssl
$ sudo apt -y remove libssl-dev
$ zef install .
[...]
[OpenSSL] Cannot locate native library 'libssl.so': libssl.so: cannot open shared object file: No such file or directory
[...]
===> Testing [FAIL]: OpenSSL:ver<0.1.24>
Aborting due to test failure: OpenSSL:ver<0.1.24> (use --force-test to override)
$ sudo apt install libssl-dev
$ zef install .
[...]
[OpenSSL] Cannot locate symbol 'sk_num' in native library 'libssl.so'
[...]
===> Testing [FAIL]: OpenSSL:ver<0.1.24>
Aborting due to test failure: OpenSSL:ver<0.1.24> (use --force-test to override)
$ rm -r .precomp
$ zef install .
===> Building: OpenSSL:ver<0.1.24>
===> Building [OK] for OpenSSL:ver<0.1.24>
===> Testing: OpenSSL:ver<0.1.24>
[OpenSSL] # NETWORK_TESTING was not set
[OpenSSL] # NETWORK_TESTING was not set
===> Testing [OK] for OpenSSL:ver<0.1.24>
===> Installing: OpenSSL:ver<0.1.24>

@usev6
Copy link
Contributor

usev6 commented Jun 26, 2021

Unfortunately the new version 0.1.25 (with the patch from 72ce2495b6) fails to install for me even with libssl-dev available:

$ sudo apt -y install libssl-dev
[...]
$ git clone https://github.com/sergot/openssl.git
[...]
$ cd openssl
$ zef install .
===> Searching for missing dependencies: PathTools, JSON::Fast
===> Updating fez mirror: http://360.zef.pm/
===> Updating cpan mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/cpan1.json
===> Updating p6c mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/p6c1.json
===> Updated p6c mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/p6c1.json
===> Updated fez mirror: http://360.zef.pm/
===> Updated cpan mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/cpan1.json
===> Building: OpenSSL:ver<0.1.25>
===> Building [OK] for OpenSSL:ver<0.1.25>
===> Testing: PathTools:ver<0.1.1>:auth<github:ugexe>
===> Testing [OK] for PathTools:ver<0.1.1>:auth<github:ugexe>
===> Testing: JSON::Fast:ver<0.16>
===> Testing [OK] for JSON::Fast:ver<0.16>
===> Testing: OpenSSL:ver<0.1.25>
[OpenSSL] # NETWORK_TESTING was not set
[OpenSSL] Unable to read key
[OpenSSL]   in method new at /home/christian/openssl/lib/OpenSSL/RSATools.pm6 (OpenSSL::RSATools) line 23
[OpenSSL]   in block <unit> at t/03-rsa.t line 10
[OpenSSL] # NETWORK_TESTING was not set
===> Testing [FAIL]: OpenSSL:ver<0.1.25>
Aborting due to test failure: OpenSSL:ver<0.1.25> (use --force-test to override)

@usev6
Copy link
Contributor

usev6 commented Jun 26, 2021

I don't know if this is helpful, but adding a no precompilation at the top of lib/OpenSSL/Stack.pm6 seems to make the original problem go away.

My next thought was: That's not a general solution.
But then I started to wonder, how an update of a native library would (and should) be handled. I don't understand what precompilation does for subs that are defined with is native.
(Maybe that's not even relevant here and the problem is that the precompiled version of sub real_symbol returns the wrong result in https://github.com/sergot/openssl/blob/67269bc82e/lib/OpenSSL/Stack.pm6#L15-L19.)

@ugexe
Copy link
Collaborator

ugexe commented Jun 27, 2021

If installation was working before and after manually deleting the .precomp directory then I wonder if 72ce2495b6 is actually deleting all the precomp files

@usev6
Copy link
Contributor

usev6 commented Jun 27, 2021

This is weird. It looks like the new problem (Unable to read key in OpenSSL::RSATools) goes away randomly.

I just tried to run zef install . a couple of times and it succeeded at the fifth attempt:

$ sudo apt -y install libssl-dev
[...]
$ rm -rf ~/.raku ~/.zef ~/openssl
$ /opt/rakudo-pkg/bin/install-zef
[...]
$ export PATH=~/.raku/bin:$PATH
$ git clone https://github.com/sergot/openssl.git
$ cd ~/openssl
$ zef install .
[...]
[OpenSSL] Unable to read key
[OpenSSL]   in method new at /home/christian/openssl/lib/OpenSSL/RSATools.pm6 (OpenSSL::RSATools) line 23
[OpenSSL]   in block <unit> at t/03-rsa.t line 10
[...]
Aborting due to test failure: OpenSSL:ver<0.1.25> (use --force-test to override)
$ zef install .
[...]
[OpenSSL] Unable to read key
[OpenSSL]   in method new at /home/christian/openssl/lib/OpenSSL/RSATools.pm6 (OpenSSL::RSATools) line 23
[OpenSSL]   in block <unit> at t/03-rsa.t line 10
[...]
Aborting due to test failure: OpenSSL:ver<0.1.25> (use --force-test to override)
$ zef install .
[...]
[OpenSSL] Unable to read key
[OpenSSL]   in method new at /home/christian/openssl/lib/OpenSSL/RSATools.pm6 (OpenSSL::RSATools) line 23
[OpenSSL]   in block <unit> at t/03-rsa.t line 10
[...]
Aborting due to test failure: OpenSSL:ver<0.1.25> (use --force-test to override)
$ zef install .
[...]
[OpenSSL] Unable to read key
[OpenSSL]   in method new at /home/christian/openssl/lib/OpenSSL/RSATools.pm6 (OpenSSL::RSATools) line 23
[OpenSSL]   in block <unit> at t/03-rsa.t line 10
[...]
Aborting due to test failure: OpenSSL:ver<0.1.25> (use --force-test to override)
$ zef install .
[...]
===> Testing [OK] for OpenSSL:ver<0.1.25>
[...]
===> Installing: OpenSSL:ver<0.1.25>

@usev6
Copy link
Contributor

usev6 commented Jul 14, 2021

I've opened a separate issue for the new problem: #87

@usev6
Copy link
Contributor

usev6 commented Jul 18, 2021

With #87 out of the way, the installation of OpenSSL (0.1.26) behaves as expected:

  1. fails with missing package libssl-dev
  2. but succeeds after installing that package
$ rm -rf ~/.zef/ ~/.raku/  ## cleanup old installation
$ sudo apt -y remove libssl-dev  ## remove needed package to see expected error
[...]
$ /opt/rakudo-pkg/bin/install-zef
[...]
$ export PATH=~/.raku/bin:$PATH
$ zef install OpenSSL
[...]
[OpenSSL] Cannot locate native library 'libssl.so': libssl.so: cannot open shared object file: No such file or directory
[...]
===> Testing [FAIL]: OpenSSL:ver<0.1.26>
Aborting due to test failure: OpenSSL:ver<0.1.26> (use --force-test to override)
$ sudo apt -y install libssl-dev  ## provide needed package
$ zef install OpenSSL
[...]
===> Installing: OpenSSL:ver<0.1.26>
$ raku -e 'use OpenSSL; say "alive"'
alive

To be sure, I've tried the above commands a couple of times, but the results were consistent.

Maybe someone else could double check and then this issue could be closed?

@raydiak
Copy link

raydiak commented Jul 18, 2021

I've switched OSes since I last tried this, but I can confirm that this module now installs successfully without errors on Manjaro which currently provides OpenSSL 1.1.1k

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants