Problem: reingest failing on 1.9.1 #594
Comments
sromkey
added
the
Severity: critical
|
I could reproduce the issue with the transfer I have compared both environments when running a reingest. a) Pipeline and SS in Separate VMs: SS uses unar to extract the 7z AIP file. b) Standard AM: SS uses 7z to extract the 7z AIP file: I run the same "unar" command for each AIP and I got the same error in both enviroments: |
|
The reason for the switch is in And I am just trying to bottom out this issue here: #599 but if there is no pointer file, which is the behavior we are seeing, the default is to unar. There are a number of ways of solving this, short- to longer- term. Once 599 is understood those options will be a little clearer. |
|
As shown in #599 a pointer file isn't created in this type of deployment. And noted above, as we see in this line here: Note: the discussion below is contingent on #599. If we can ensure pointer files are created through configuration then this problem disappears for now. There are three tools used for unpacking an archive in the function below: def _get_decompr_cmd(compression, extract_path, full_path):
"""Returns a decompression command (as a list), given ``compression``
(one of ``COMPRESSION_ALGORITHMS``), the destination path
``extract_path`` and the path of the archive ``full_path``.
"""
if compression in (utils.COMPRESSION_7Z_BZIP, utils.COMPRESSION_7Z_LZMA):
return ['7z', 'x', '-bd', '-y', '-o{0}'.format(extract_path),
full_path]
elif compression == utils.COMPRESSION_TAR_BZIP2:
return ['/bin/tar', 'xvjf', full_path, '-C', extract_path]
return ['unar', '-force-overwrite', '-o', extract_path, full_path]
So the answer isn't straight forward. I would also like to understand the implications of using tar, two outputs shown below using both Actual solutions
$ file --mime-type -F : *
11-f7fba8b8-dfad-4ec9-b6ea-3f5059b21a06.tar: application/x-tar
12-81b44c2c-e65f-4871-818e-056f79382e18.tar.bz2: application/x-bzip2
13-ebd86be1-7153-4ef2-a2dc-b49010a11e3d.7z: application/x-7z-compressed
demo-f56695de-01dd-46b6-86cf-3b84f8ed20a8.7z: application/x-7z-compressed
demo-normalize-12c8aeaf-4d03-44d3-969d-510613776e77.7z: application/x-7z-compressed
NB If there were time to work on the upstream issue, i.e. identify why |
|
NB. The different package types are currently listed like this by file -F : --mime-type *
type_3-46db9581-544f-4810-a5a4-f3c988dc0ae5.7z: application/x-7z-compressed
type_4-00368858-7e61-4f39-9938-2ee86c5c600e.7z: application/x-7z-compressed
type_5-3ae92cc8-6042-4f6b-b44d-e9de951e5172.tar: application/x-tar
type_6-84f6e507-ccb5-4b86-92d0-ed0458808286.7z: application/x-7z-compressed
type_7-f78ed85f-1972-4b0f-b43e-cbc43b7ecb08.tar.bz2: application/x-bzip2 |
|
I found the same error decompressing with the unar command this AIP 7z file in bionic, xenial and CentOS. I built from sources the unar command and I got the same error. |
ross-spencer
added
the
Status: review
|
Full AIP reingest is now working on am19bionic.qa. |
sromkey
added
Status: verified
and removed
Status: review
|
Added to release notes |
Expected behaviour
Reingest should succeed.
Current behaviour
It is not. The error in the UI just says "Error re-ingesting package: An unknown error occurred."
Error in the SS logs:
The text was updated successfully, but these errors were encountered: