Error of ~/anaconda3/envs/moose/libmesh/contrib/bin/libtool: 1: /usr/bin/sed: not found

[yw5aj]

Dear all,

Sorry about this naive question...

Everything was working fine until earlier today that I updated MOOSE based on this page. I made sure that moose-libmesh is up-to-date, re-pulled the github repo of moose, and then tried to run make cloberrall.

But then I got the message of /anaconda3/envs/moose/libmesh/contrib/bin/libtool: 1: /usr/bin/sed: not found. Could anyone please help me out a bit?

Best,
Shawn

[jessecarterMOOSE]

Glad to hear I'm not the only one with this problem! @lindsayad @milljm did you ever figure out what was going on? I ended up just making a soft link.

[lindsayad]

No I don't have much of a clue on this one

[milljm]

I am not sure why libtool would hard code a path to sed like that. If it's Anaconda, that would be even stranger (implying that it would go about mucking with packages like this). I don't think it's libMesh's fault as well, as this is a 'contrib'. The other thing is, this wasn't an issue until recently. The Conda recipes creating moose-libmesh have not changed since it's inception.

[milljm]

@cticenhour reported this error happening on a few of our build machines (but then stopped). I suspect it occurred when that build machine found an issue with conda, and attempted to build the environment from scratch (such as what would happen with a new-user install).

I'll take some time today and try and reproduce the issue.

Edit: I can replicate this error... I'll look into it.

[cticenhour]

It's definitely still happening....I just haven't been able to get back to this (too many meetings!). Let me know how I can help (or provide another box to test something on)

[cticenhour]

Last data point I have was 10 hours ago on mobuild31 for Freya.

[milljm]

It looks like libmesh/contrib/bin/libtool is generated during ./configure. And so is recording where it found sed at the time. Unfortunately, not all machines have sed in the same place. Not exactly sure why we didn't see this error immediately after the first version of moose-libmesh rolled out the door.

What I think we need to do, is include sed as a requirement when building/running libMesh. That way when moose-libmesh is installed, the dependency sed will also be installed, and hopefully be available in the same location as it was during build. Conda does some magical things when it prepares a package... hopefully it corrects more than just link lines (libtool is a script, and I am not sure Conda sniffs out paths inside scripts when it corrects/prepares them for redistribution)

[laumiulun]

softlink kind of bandage solve the problem, but it keep showing the message:

'libtool: error: ignoring unknown tag CXX'

It still finishes compiling and the test run fine.

[milljm]

When I build moose-libmesh and set a dependency on sed, it works. So I will commit a change to moose/conda/libmesh/meta.yaml, unless @lindsayad has a better idea,

[milljm]

scratch that. It works. But I am spammed by a new message:

libtool:   error: ignoring unknown tag CXX

Still investigating.

[lindsayad]

I would like to try using a conda/system provided libtool per the slack discussion, but that would require some changes in the MOOSE makefiles

[laumiulun]

I didn't encounter this issue if I started from scratch in a VM, only when I am updating moose & conda. Nuking the conda environment didn't help either.

[milljm]

A couple of work-arounds:

Downgrade to the previous version of moose-libmesh:

conda install moose-libmesh=2020.10.05

Or build your own libMesh:

conda deactivate
conda create -n mylibmesh moose-petsc moose-tools
conda activate mylibmesh # use this instead of 'moose' for now...
cd moose
export MOOSE_JOBS=8   # what ever core count you use
scripts/update_and_rebuild_libmesh.sh

[lindsayad]

Hard to say. Our MOOSE-libMesh integration is typically pretty tight. My only way of knowing would be to go try it. I'd put it at about 50% odds of being successful. Sometimes I'm aware of backwards incompatible changes in libMesh...and I think we had one of those not this libMesh update but the one prior. So you might be good to go...

[mcacace]

Hi,
FYI ...
Yesterday I found that manual installation (petsc and libmesh) worked in the end --> this was my "easy" work-around after a full half day of trying.
I'm currently testing the second work-around as per @milljm post (my bad not having read it before). One thing I noticed is that in order to compile libmesh (despite activating petsc and the moose utils from the env in conda), I must set all CPPFLAGS, CXXFLAGS, CFLAGS, FFLAGS. Before I was having errors related to the metaphysicl package.
So far configuring libmesh seems to run (not yet finished though) and will see if moose also compiles fine.
Thanks again for the support,
Mauro

[mcacace]

quick update, it did not work. Will investigate it further

[mcacace]

solved the issue (conflicting boost libraries). Moose compiled fine by configuring libmesh only as well.
Sorry for the multiple posts.

[milljm]

That was interesting that you had to set all those CFLAGS... Apologies about the delayed response. I think my GitHub account and email notifications are now set correctly.

Can you post the error which occurred during your attempt to build libMesh?

[lindsayad]

Ok I've opened an issue for this at #16232

[milljm]

I built moose-libmesh manually and have made it available on Anaconda. Please remove moose-libmesh (if you're using it) and re-install the environment (after clearing your download cache to make sure you get the latest version from anaconda).

conda deactivate   # make sure you are not using moose
conda env remove -n moose --yes
conda clean --all --yes
conda create -n moose moose-libmesh moose-tools

Edit: When I originally posted this, it didn't work. I have since corrected that error (verified on Ubuntu 18)

[tfc1992]

Ok, thanks a lot.

[milljm]

@tfc1992 What make and version of Linux are you using by chance?

[tfc1992]

@milljm make v 4.1 and ubuntu 18.01

[milljm]

Thank you!

[milljm]

I am building another version of moose-libmesh at the moment. I am going to just hack my way through it as suggested by @dschwen. I'll post again when it's complete.

[tfc1992]

Many thanks. I am a new moose user.

[milljm]

okay, finished... if you would perform the following and try again:

conda update --all

This is assuming you have not already build your own version of libMesh as suggested by @GiudGiud.

[tfc1992]

Wow, great, the compilation was successful, thank you very much! @milljm

[milljm]

Welcome!

[milljm]

For those still receiving this error, please perform an update to get the latest moose-libmesh package:

conda update --all

[jessecarterMOOSE]

@milljm Does the new moose-libmesh package only contain a fix to the sed/grep issue or are there other changes I need?

[loganharbour]

To hop in - it also contains the changes in #16375 if you haven't updated in a while (beginning of Nov)

We've gotten bad about sending out emails for libmesh updates.

[jessecarterMOOSE]

Ah, I'll update it then.

Just curious, do you still plan on notifying people about libmesh updates and if so how if you're not using the google group?

[jessecarterMOOSE]

I just wonder how many people are "subscribed" to these github discussions.

[milljm]

That is a good question. The short answer is that right at this moment, it contains actual changes to libMesh (as Logan just mentioned). Though had you run conda update --all at 3AM on Tuesday when I created the fix manually... no it would not have. That package at 3AM only contained the fix for sed|grep|dd....

[jessecarterMOOSE]

Though had you run conda update --all at 3AM on Tuesday when I created the fix manually...

Nope definitely didn't do that.

I had added some softlinks for grep/sed to fix the problem on my system and was just wondering if there was more I needed to do.