MultiProgress falls apart if the number of ProgressBars excedes the number of lines in a terminal

[alerque]

I successfully converted a project of mine from hand rolled terminal messaging that was very verbose (potentially hundreds of jobs with messages like "starting X, update about X, finished X") to utilizing indicatif with a single progress bar for each job with a start messages, ticks, then finally updating to a finished message.

This worked great in testing with 1-50 jobs and I thought I got everything working the way I wanted. Then I moved over to a production project that uses my tool and ran into a case where the number of jobs running was larger than the terminal was tall. It took me a while to figure out if my spinner update messages were getting cross-wired, but I was able to put together an MWE that seems to have the same problem.

Here is a stand alone script that can be saved to a file like status.rs, make executable with chmod 755 status.rs, then run with ./status.rs 10.

#!/usr/bin/env -S cargo +nightly -Z script
## [package ]
## edition = "2021"
## [dependencies]
## indicatif = "0.17"
## lazy_static = "1.4"
## rayon = "1.9"

use indicatif::*;
use lazy_static::lazy_static;
use std::env::args;
use std::sync::{Arc, RwLock};
use std::{thread, time::Duration};

lazy_static! {
    pub static ref MP: RwLock<MultiProgress> = RwLock::new(MultiProgress::new());
}

fn main() {
    // $1 is number of jobs
    let jobs: u64 = args().last().unwrap().parse().unwrap();

    // A section header that will stay above the jobs
    let pb = ProgressBar::new_spinner()
        .with_style(ProgressStyle::with_template("{spinner} {msg}").unwrap())
        .with_message("Starting jobs");
    let pb = MP.write().unwrap().add(pb);
    pb.enable_steady_tick(Duration::from_millis(100));

    let results = Arc::new(RwLock::new(Vec::new()));

    // A bunch of jobs with their own spinners
    rayon::scope(|s| {
        for n in 1..=jobs {
            let results = &results;
            s.spawn(move |_| {
                let pb = ProgressBar::new_spinner()
                    .with_style(ProgressStyle::with_template("{spinner} {msg}").unwrap())
                    .with_message(format!("foo {n}"));
                let pb = MP.write().unwrap().add(pb);
                pb.enable_steady_tick(Duration::from_millis(100));
                thread::sleep(Duration::from_millis(n * 200));
                pb.finish_with_message(format!("bar {n}"));
                results.write().unwrap().push(true);
            });
        }
    });

    let ret = results.read().unwrap().iter().all(|&v| v);

    // Update the header at the top
    pb.finish_with_message(format!("Completed jobs = {ret}"));

    // A trailing status message
    let pb = ProgressBar::new_spinner()
        .with_style(ProgressStyle::with_template("{spinner} {msg}").unwrap());
    let pb = MP.write().unwrap().add(pb);
    pb.finish_with_message("Wrapped up");
}

Here is what the finished output looks like when the terminal is big enough to show everything at once:

$ ./status.rs 10
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.01s
     Running `/home/caleb/.local/share/cargo/debug/status 10`
  Completed jobs = true
  bar 2
  bar 1
  bar 7
  bar 5
  bar 10
  bar 9
  bar 6
  bar 3
  bar 8
  bar 4
  Wrapped up

Here is what I get if the terminal is only about 8 lines (or you can test with a number bigger than 10 to see the effect on a bigger terminal):

$ ./status.rs 10
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.01s
     Running `/home/caleb/.local/share/cargo/debug/status 10`
  Completed jobs = true
  bar 5
  bar 1
  bar 3
  bar 6
  bar 2
  bar 4
  bar 10
  bar 7
  bar 9
  bar 8

Note all the job bars do end up getting drawn and updated, but only after they are all complete. After the screen is full the remaining active spinners are hidden until the app finishes. And even then the final progress bar isn't drawn at all.

Obviously the thread model here isn't necessary for this example, but it's close enough to the overall model used in my app to replicate the problem.

[djc]

So we've had #582 and a bunch of follow-up work to try and do this better. I don't have all the context paged in, but would be happy to review a PR that addresses this.