Get the pid of a started command

[snowe2010]

Is there any way to retrieve the pid of a running command?

[oconnor663]

There isn't, mainly because an expression can be a group of pids and not just one. In fact if you're using then expressions, there might be processes in the group that will start in the future, and which don't have pids yet. (I'm considering getting rid of then because the complexity this introduces is pretty high.)

We could expose pids if the handle of a running expression had a structure that mirrored the expression itself, where you could go piece by piece and ask what each one's pid is. That's doable, but it's a pretty big increase from an api perspective.

Out of curiosity, what are you thinking about doing with the pid? If you want to send signals, there's already support for that.

[snowe2010]

I need to write the pid to a file. My app dies and leaves zombies running (on purpose, the app is meant to start a bunch of spring boot services and node servers). Currently I can accomplish this using

    let mut command = tools::create_command("rake", tools::append_to_app_dir(repo).as_str(), None);
    let output = command.stdout(rake_out).spawn().expect("rake command failed to start {}");
    let repo_pid: u32 = output.id();

but I would really like to use your library because it's able to combine stdout and stderr, which from what I've seen is quite difficult (for me at least).

[oconnor663]

Hmm, yeah. If all you want to do is the stdout+stderr pipe, it might be simpler to just call into os_pipe directly, to do what duct would've done for you. There's some example code here: https://github.com/oconnor663/os_pipe.rs#example. Here are a couple more examples of the ways you might write a helper function to do it:

use os_pipe::{pipe, IntoStdio, PipeReader};
use std::io;
use std::process::{Command, Child, Stdio};

// A key detail when juggling pipes by hand is that reading the output
// of a pipe will block until all the write ends are closed. That means
// we have to be careful to close them immediately after spawning a
// child, including the ends that are *inside* the Command object. That
// gives us two options:
// 1) Take the Command by value, so that it automatically drops at the
//    end of the function. (Simple but maybe annoying for the caller.)
// 2) Take an &mut Command, but manually overwrite its stdout and stderr
//    fields after spawning, to force the pipes there to drop.

fn spawn_joined_by_value(mut command: Command) -> io::Result<(Child, PipeReader)> {
    let (reader, writer) = pipe()?;
    let writer_clone = writer.try_clone()?;
    command.stdout(writer.into_stdio());
    command.stderr(writer_clone.into_stdio());
    let child = command.spawn()?;
    Ok((child, reader)) // command and its pipe writers automatically drop here
}

fn spawn_joined_by_reference(command: &mut Command) -> io::Result<(Child, PipeReader)> {
    let (reader, writer) = pipe()?;
    let writer_clone = writer.try_clone()?;
    command.stdout(writer.into_stdio());
    command.stderr(writer_clone.into_stdio());
    let child = command.spawn()?;
    // Force the pipe writers to drop before returning.
    command.stdout(Stdio::inherit());
    command.stderr(Stdio::inherit());
    Ok((child, reader))
}

[oconnor663]

Thinking out loud: duct is intended mainly for programs that want to run shell commands to accomplish some specific task. Maybe run a build, or change some system config, or whatever. It's less suitable for programs whose entire purpose is to manage child processes, which are probably going to want really fine grained control over their children. The Handle API exists for callers who want to do something a little more complicated like "run this command, but kill it after 5 seconds if it's not done yet." I don't know if we'll ever support things like "run these commands under a different process group number" or "run some code in each child after forking but before execing" or "run children with a different niceness value".

[snowe2010]

Ok thank you for the help. I'll try out that code and see if I can get it working. I'm not truly trying to manage the child processes, I just want to start some children, have the app exit, and be able to find those children later, so I was using the pids for that. But if your library isn't aimed at that then I'll have to continue down the path I already was. Thank you very much!

[oconnor663]

Let me know how it goes. Also a couple thoughts that you might already be familiar with:

• I think when a parent exits before waiting on a child, that child gets re-parented to PID 1, usually systemd or whatever init daemon you're running. Other processes might be able to kill them, but can't wait on them.
• PIDs can recycle. So if you read a PID out of a file, you have to be careful to avoid a situation like "so-and-so wrote this PID file, but then they exited and we waited on them (freeing their PID for reuse), and then an hour later another random process spawned somewhere else that happens to use the same PID". That can matter if you're going to kill the PID listed. [Edit: You can work around that by trying to delete the PID file as soon as wait returns, but that's not atomic, and it's possible that your parent process might get killed somehow in between those two things. Maybe there's some fanciness to solve this with flock?]

[snowe2010]

Ok I'm back around to this after working on some other stuff. I knew your first point there, but not the second.
So ideally I would start my services, wait until I receive the text indicating they started, and then I'm sure that my pid is accurate.
I'm having trouble figuring out how to use your example to write to a file, I'm still pretty new to rust so I don't understand how to use the PipeWriter returned by pipe() to output to a file. https://docs.rs/os_pipe/0.6.0/os_pipe/struct.PipeWriter.html

Also, if I need to wait for some text to appear in the output, but I want to be logging everything to a file, how would I accomplish that? I have a few ideas, but I'm unsure if they're 'rusty' or if I'm even on the right track.

[oconnor663]

PipeWriter is for sending bytes into a pipe. That's what your child will do, but not what your parent cares about. (What's going to happen under the covers is that the OS will give a clone of the file descriptor represented by the PipeWriter::into_stdio() struct to your child process. The child will never see the PipeWriter or std::process::Stdio Rust types per se.)

Instead, you want to get your hands on the bytes that come out the other end, by reading from the PipeReader. These bytes will be the combined output of the child, if you spawned the child with that PipeWriter (or rather, it's descriptor) as both stdout and stderr. PipeReader implements std::io::Read, so you can read from it as you would from any file. If you also want to copy those bytes to a log, can you just put write calls immediately after your reads? This isn't very specific to Rust. I think it's similar to what you'd do if you wanted to, say, copy one file to another and into memory at the same time in any other language?

[snowe2010]

Ah, I guess the problem I wanted to solve is how do I get the child to continue writing to the file after rust exits. So say I read from the file (concurrently writing the lines to a log), when I reach the lines that look like so

2017-08-15 14:02:34.458  INFO 69701 --- [           main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat started on port(s): 8080 (http)
2017-08-15 14:02:34.509  INFO 69701 --- [           main] c.p.lp.gateway.GatewayApplication        : Started GatewayApplication in 40.893 seconds (JVM running for 63.131)

I want to stop reading from the file, and I want rust to be able to exit and keep writing to that log, like Command::spawn() does.. I'm guessing my problem is much more difficult than I understand.

[oconnor663]

Ah yes, that's interesting. I don't think Linux provides a way for a pipe write to automatically end up in two files, or in a pipe and a file. I think at the end of the day some process has to sit in the middle and copy those bytes. On the command line, people often do this with tee:

my_program.sh | tee -a my_logfile.txt | ...

So you could spawn children with an extra tee process reading their stdout? You could also consider having the child write to the file only, and then doing some kind of tail -f on the file? That would be pretty easy to set up in a shell script, but maybe hard to do in a platform-independent way.

[snowe2010]

Fantastic. This is the kind of lead I needed I think. I'll look into those things. Thank you very much! I thought about a tail -f but wasn't sure how to implement it, but I now know the general direction I need to go in.