Docker permissions #25
Comments
|
Hi, I have updated PrepareAA to handle issues related to permissions of the output directory in 580f923 and also consolidate a file from AmpliconClassifier that may be trying to write to a location not in nessarily in that same spot. Can you please pull the latest version of the docker image and try again? You may already have done so, but also please double check that the location you are hoping to save data to exists and has write permissions for root. Thanks, |
|
I made a change to the run file so when you execute it, it looks like this docker run -u So everything should be read and written as the enduser running the app. I will pull down the update(s) and give it a shot. Thank you. |
|
Pulled and running, it will take over 20hours but i will let you know. Thank you for your time. I do find adding the following to the run script avoids a lot of issues, so the data is written as the caregiver and not root:
|
|
Thank you, this is a good suggestion, I will incorporate it.
…On Thu, Jul 28, 2022, 4:17 PM MrDotOne ***@***.***> wrote:
Pulled and running, it will take over 20hours but i will let you know.
Thank you for your time.
I do find adding the following to the run script avoids a lot of issues,
so the data is written as the caregiver and not root:
" -u id -u $USER:id -g $USER "
—
Reply to this email directly, view it on GitHub
<#25 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADM3Q43O4XBY5NPJ2CEIIU3VWMIHXANCNFSM546XOEXA>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
Someone on another repo suggested it, when i was having issues with the results being written as root and the person running it didnt have escalation privileges. I thought i would pass on that nugget. |
|
I am still having issues [root:INFO] #TIME 79252.045 Plotting SV View for amplicon7 Traceback (most recent call last): Matched /home/bam_dir/cofinal.bam to reference genome GRCh38 Running CNVKit batch Running CNVKit segment Cleaning up temporary files Running amplified_intervals Running AC 2022-07-29 21:26:25.262009 |
|
I will run as root and that should fix it but ... |
|
OK, i reran the run as root using the run script as provided in the repo. It seems to have completed successfully. This is good progress. However, the two times i have run it with the run -u |
|
Unfortunately that is not working. The run file works fine, for root, but not for a non-escalated account. I keep getting this error when i run as a user with the id stuff in the run command [root:INFO] #TIME 79384.895 Plotting SV View for amplicon7 Traceback (most recent call last): Matched /home/bam_dir/cofinal.bam to reference genome GRCh38 Running CNVKit batch Running CNVKit segment Cleaning up temporary files Running amplified_intervals Running AC 2022-08-01 00:05:22.206630 |
|
Hi, Thank you for sharing. I have also now done some testing on my end and it appears that assigning a custom user for the image is non-trivial and that the above proposed solution (adding -u id $UID:id $GID) does not quite work as expected. I recommend that users run with the current default settings, generating the files as root and then users can chmod or copy the relevant files later if they need non-root ownership. I do not plan to address this issue of non-root ownership in the PrepareAA generated files at this particular time, but perhaps in the future if there is a compelling reason. Jens |
|
Non-root users cannon chown/chgrp files., that is a serious cybersecurity concern. |
|
Is there a way to implement a python script within the run file to do something similar to this? (base) [root@lri-uapps-2 data]# cat chown.py Michael |
|
Hi Michael, Without re-assigning user IDs inside the container itself or alternatively sharing the /etc/passwd file from the host machine with the docker image, there is no way to provide the docker image with exact same user ids account/group information of the host machine. The previously proposed solution runs the image as a specific user inside the image, but that user is not mapped to the same user on the host machine. Perhaps one option is instead to have the docker script recursively chmod to add global read/write permissions on all the files written by the image into the mounted directory when it is finished. Would this solution be satisfactory for you? I can test this out in the next couple of days. Jens |
|
That is a solution i am trying to implement. I tried to use /home/output however the result was no such file or directory. |
|
I just pulled [fc3b5e8] and will give a try with the --run_as_user option which looks promising already: docker run --rm -e HOST_UID=$(id -u) -e HOST_GID=$(id -g) -u I will let you know what i find. Thank you for looking into this |
|
This is perfect: (base) [root@lri-uapps-2 data]# cd output |
|
Glad to hear it is working for you. Reopening issue for others who may run in to issues despite this fix. I will note that this solution works as long as the docker daemon is configured to not offset UIDs and GIDs, which is sometimes done to improve security of the host machine. More info about the docker namespace remapping is available here: https://docs.oracle.com/cd/E37670_01/E75728/html/ol-docker-userns-remap.html. Jens |
|
Thank you for the fixes and the link, i will check it out. There are a couple other repos like this that could use this technique. Unfortunately, we may be in research here, but this is not academia, and we lock stuff down pretty tightly. Sometimes to the point where things are unusable. This was of great benefit. Thank you. |
Just pulled PAA down the other day and have running it, my run command is:
/data/PrepareAA/docker/run_paa_docker.py -o /data/output -s Colo -t 16 --bam /data/Data/Colo/cofinal.bam --run_AA --run_AC
however after 22+ hours i get to this point and if fails miserably:
/home/programs/AmpliconClassifier-main/amplicon_classifier.py -i /home/output//Colo_classification/Colo.input --ref GRCh38 -o /home/output//Colo_classification/Colo --annotate_cycles_file --report_complexity
reading /home/data_repo/GRCh38/Genes_hg38.gff
read 22998 genes
Traceback (most recent call last):
File "/home/programs/AmpliconClassifier-main/amplicon_classifier.py", line 667, in
f2gf = open("feature_to_graph.txt", 'w')
PermissionError: [Errno 13] Permission denied: 'feature_to_graph.txt'
Traceback (most recent call last):
File "/home/programs/AmpliconClassifier-main/make_results_table.py", line 65, in
with open(args.input) as input_file, open(args.classification_file) as classification_file:
FileNotFoundError: [Errno 2] No such file or directory: '/home/output//Colo_classification/Colo_amplicon_classification_profiles.tsv'
2022-07-27 22:49:31.494158
I am unsure where the feature_to_graph.txt should be found and the Colo_amplicon_classification_profiles.tsv doesnt seem to be getting generated.
Any assistance would be appreciated
The text was updated successfully, but these errors were encountered: