Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When using temp as destination, have option to not start a new plot until archiver completes #828

Open
michaelc95 opened this issue Jul 5, 2021 · 1 comment
Labels
enhancement New feature or request

Comments

@michaelc95
Copy link

When configured without a destination the software leaves it in the temp directory it created it in. In my case this is 1 of 24x 10 SAS hard drives. Then the archiver comes along and transfers it to the remote system. While the archiver is trying to do its thing plotman says hey I got a free temp directory and starts up a new plot on that drive slamming the transfer rate of the archiver.
This slows things WAY down. Maybe an option could be added that just checks if a .plot file exists in the temp directory
consider it not ready.

@michaelc95 michaelc95 added the enhancement New feature or request label Jul 5, 2021
@realdevnullius
Copy link

realdevnullius commented Aug 21, 2021

You could limit the amount of plots with the phase options, no?

    # Optional: Allows the overriding of some scheduling characteristics of the
    # tmp directories specified here.
    # This contains a map of tmp directory names to attributes. If a tmp directory
    # and attribute is not listed here, the default attribute setting from the main
    # configuration will be used
    #
    # Currently support override parameters:
    #     - tmpdir_stagger_phase_major (requires tmpdir_stagger_phase_minor)
    #     - tmpdir_stagger_phase_minor (requires tmpdir_stagger_phase_major)
    #     - tmpdir_stagger_phase_limit
    #     - tmpdir_max_jobs
    tmp_overrides:
            # In this example, /mnt/tmp/00 is larger and faster than the
            # other tmp dirs and it can hold more plots than the default,
            # allowing more simultaneous plots, so they are being started
            # earlier than the global setting above.
            #"/mnt/tmp/00":
            #        tmpdir_stagger_phase_major: 1
            #        tmpdir_stagger_phase_minor: 5
            #        tmpdir_max_jobs: 5
            # Here, /mnt/tmp/03 is smaller, so a different config might be
            # to space the phase stagger further apart and only allow 2 jobs
            # to run concurrently in it
            # QUESTION HOW TO PLAY WITH THESE PHASES?? :(
            #"/mnt/tmp/03":
            #        tmpdir_stagger_phase_major: 3
            #        tmpdir_stagger_phase_minor: 1
            #        tmpdir_max_jobs: 2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants