Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make submit --dry work with S3 #6545

Closed
belforte opened this issue Apr 16, 2021 · 2 comments
Closed

make submit --dry work with S3 #6545

belforte opened this issue Apr 16, 2021 · 2 comments
Assignees

Comments

@belforte
Copy link
Member

looks like current dry run is based on TaskWorker/Actions/DryRunUploader preparing a tarball called dry-run-sandbox.tar.gz and uploading it to crabcache "by name" and then crab submit --dry finds it there (so if a user were to run submit --dry from two tasks at same time, there would be a mess).

Best way to do this in S3 seems to add another objectype and hanlde this like clientlog or twlog, placing it in the task directory of the user, since the taskname is known at the time that all those actions are done.

This requires another change to RESTCache, but then changes to client and DryRunUploader should be very small.

To be tested.

@belforte belforte self-assigned this Apr 16, 2021
@belforte
Copy link
Member Author

belforte commented Apr 16, 2021

so I have three actions

and possibly a fourth one

@belforte
Copy link
Member Author

all done. closing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant