project templates aren't actually copied

[johrstrom]

The feature for project templates is still a bit incomplete. Although we can create a project from a template, I don't believe it actually does anything more than creating a project.

When creating a project from a template it should copy over several files, among them

• any *.{sh,bash,slurm} files (etc here, there are a few other extentions)
• the .ondemand/ directory

It should also exclude things like

• log files

[abujeda]

I have been playing with the templates and copying files. I started by looking at how batch connect sessions stage a nw interactive sessions based on the app that was selected. It executes an rsync command to copy the files and this was the base I was using.

Things I noticed:
1.- If there are scripts within the .ondemand folder in the template, these will be copied. The scripts configuration contain reference to files and folders with a full path. These will be problematic when copied from a template.

2.- Scripts also store cache values and execution history, these are problematic too.

3.- All of this can be fixed only copying files with a pattern as mentioned in the issue. I am just wondering if it will not be as useful as projects change quite a lot and trying to guess what files are useful will be difficult.

[johrstrom]

Yea my guess is we need some sort of interface to specify what to copy. We can start with an initial phase of copying everything we have or can find, then add the capability to filter/specify what to copy. IDK what that looks like, but it's likely yaml, probably an extension of manifest.yml.

[abujeda]

Created an implementation that copies all the files in the template: #3031

[johrstrom]

Now that I think about it we may just be good with this. No need for a follow up to make it more complicated. These template directories are created & controlled by admins. So they should be able to remove log files or errant things they don't want copied.

Here's where we store OSC's templates for the current job composer. I.e., the project templates aren't say copied from an existing project that has log files and so on. It's basically a project that has no history/past executions.

https://github.com/OSC/osc-ood-config/tree/master/ondemand.osc.edu/apps/myjobs/templates