Linux strip command breaks single-file, native assemblies in 3.1

[MichaelStromberg]

Description

When including single-file, native binaries in an rpm package, the strip command is also typically run. For example, on a CentOS 7 system, rpmbuild runs the following script: /usr/lib/rpm/redhat/brp-strip-comment-note

brp-strip-comment-note executes "strip -R .comment -R .note " on each binary and library that's being added to an RPM package. This strips the .comment and the .note sections from the binary. When this happens on a .NET Core 3.1.300 single-file, native binary; the file gets truncated and corrupted.

Running the original binary works well:

$ ./Nirvana
---------------------------------------------------------------------------
Nirvana                                             (c) 2020 Illumina, Inc.
Stromberg, Roy, Lajugie, Jiang, Li, and Kang               3.9.1-0-gc823805
---------------------------------------------------------------------------

USAGE: dotnet Nirvana.dll -i <vcf path> -c <cache prefix> --sd <sa dir> -r <ref path> -o <base output filename>
Annotates a set of variants

OPTIONS:
      --cache, -c <prefix>   input cache prefix
      --in, -i <path>        input VCF path
      --out, -o <file path>  output file path
      --ref, -r <path>       input compressed reference sequence path
      --sd <directory>       input supplementary annotation directory
      --force-mt             forces to annotate mitochondrial variants
      --disable-recomposition
                             don't recompose function relevant variants
      --legacy-vids          enables support for legacy VIDs
      --str <VALUE>          user provided STR annotation TSV file
      --help, -h             displays the help menu
      --version, -v          displays the version

Cache version: 26, Supplementary annotation version: 57, Reference version: 7

Comparing this file with the original yields:

$ ls -la Nirvana
-rwxr-xr-x 1 user 504 74861535 Jun  2 18:47 Nirvana
$ strip -R .comment -R .note Nirvana
$ ls -la Nirvana
-rwxr-xr-x 1 user 504 90552 Jun  2 18:48 Nirvana
$ ./Nirvana
Failure processing application bundle; possible file corruption.
I/O failure reading contents of the bundle.
A fatal error was encountered. Could not extract contents of the bundle

• Include minimal steps to reproduce the problem if possible.

# grab the code from GitHub
git clone https://github.com/Illumina/Nirvana.git

# grab the proper branch
cd Nirvana
git checkout stripping_is_not_fun

# build the Nirvana project
cd Nirvana
dotnet publish -r linux-x64 -c Release /p:DebugType=None

# check that the compiled binary produces the help menu
cd ../bin/Release/netcoreapp3.1/linux-x64/publish
./Nirvana

# verify that the following strip command (typically run in rpmbuild) destroys the binary
strip -R .comment -R .note Nirvana
./Nirvana

• What behavior would you expect? Ideally, the normal rpmbuild process should not cannibalize the .NET native binaries.

Configuration

• Which version of .NET is the code running on? .NET Core 3.1.300
• What OS and version, and what distro if applicable? CentOS Linux release 7.7.1908
• What is the architecture (x64, x86, ARM, ARM64)? x64
• Do you know whether it is specific to that configuration? This behavior can probably be generalized for any RedHat-like OS that uses RPM packages.

Other information

[ghost]

Tagging subscribers to this area: @swaroop-sridhar
Notify danmosemsft if you want to be subscribed.

[MichaelStromberg]

Here's an even more minimal way of reproducing the error:

# create a new HelloWorld project
mkdir HelloWorld
cd HelloWorld
dotnet new console

Update the HelloWorld.csproj file:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp3.1</TargetFramework>
    <PublishTrimmed>true</PublishTrimmed>
    <PublishReadyToRunShowWarnings>true</PublishReadyToRunShowWarnings>
    <PublishReadyToRun>true</PublishReadyToRun>
    <PublishSingleFile>true</PublishSingleFile>
  </PropertyGroup>
</Project>

And then the last bit:

# natively compile our binary
dotnet publish -r linux-x64 -c Release /p:DebugType=None

# check that it works
bin/Release/netcoreapp3.1/linux-x64/publish/HelloWorld

# verify that the following strip command destroys the binary
strip bin/Release/netcoreapp3.1/linux-x64/publish/HelloWorld
bin/Release/netcoreapp3.1/linux-x64/publish/HelloWorld

[swaroop-sridhar]

@MichaelStromberg thanks for reporting the issue.

The single-file bundler writes meta-data with absolute offsets, which is consumed by the host components at run-time. The post-processing with strip ... invalidated the offsets written in the bundle meta-data. This is why the app fails to run.

Solving this problem may require generating single-file bundles in coopeartion with the native linker, so that the bundle header (and embedded files) can be located without relying on absolute offsets. (CC: @janvorli)

In a way, both single-file apps, and rpm are packaging solutions. So, I'm curious why you'd use both technologies in conjunction. Thanks.

[dotnet-policy-service]

Due to lack of recent activity, this issue has been marked as a candidate for backlog cleanup. It will be closed if no further activity occurs within 14 more days. Any new comment (by anyone, not necessarily the author) will undo this process.

This process is part of our issue cleanup automation.