Question: Is support for pipelines and redirection planned ?

[Byron]

This crate already looks like something that is making the invocation of external commands so much easier. Especially after reading how julia handles program invocation, I was wondering whether this crate plans to add features to make pipelines and redirection even easier.

Thank you

[krdln]

Supporting piping and redirection would be a nice addition, sure! But the problem is, that I wanted to create a macro that creates a std::process::Command struct – so the user can use all the regular methods from this struct and use this macro as a drop-in replacement.

Redirecting to files, with > filename syntax may be possible without chaning the API (still returning Command, I mean), by creating a File and calling into_raw_fd, but that poses a problem with when the descriptor should be freed and how to handle errors of file creation. And I think that it's quite a low priority, since most of commands have some kind of -o switch, which handles redirection. And quite a common usecase for writing a file is to read it afterwards. In that case, it's better to just .output() and have to file in a Vec (or use .piped() and just read from it).

More general redirection (| and >) could be done in two ways:

• By passing everything to sh and let it do the work. This is the approach that duct.rs takes. This approach is simple, but poses its own problems. First, it's spawning a shell each time. It has a non-neglegible cost and in my opinion, violates Rust's idea of zero-cost abstractions. Second, it requires to put strings in quotes and escape special characters – this feels error-prone. You can always simulate it with eg. command!( sh -c foo|bar>(file) ).
• By creating a few separate Commands with manually created pipes. This also seems to be supported by duct.rs. This is more elegant, but it may be harder to implement properly, especially the job management. Maybe I could join my efforts with that crate's author? The reason I haven't pursued that solution was that it's impossible to "just return Command" that way.

So generally, your question boils down to: "should this crate implement its own Command-like struct, or stay faithful to std::process::Command". The reason I wanted to reuse the std's one was to not reinvent the wheel and to have a bunch of methods to reuse. And to make it more composable.

So, is it worth to create my own Command? If yes, should there be two versions of this macro (two, because returing something different based on presence of | feels really ugly)? Or maybe we should go in different direction – to add a join(self, Command) method to the std::process::Command, and then let this macro just use that funcion?

[Byron]

Thank you for the very elaborate and incredibly well thought-out reply ! Even though my reply cannot possibly be as well formed as I clearly am on the user-side of such an implementation, I hope you are fine approaching it that way.

Generally, I agree that it should be favoured to keep returning actual Command instances as long as possible.

What I would hope for, maybe even as a first step, is to allow simple piping of arbitrary processes. It could look like this:

if let Ok(cc) = pipe!(command!(ls), command!(sort), command!(uniq)) {
    println!("{:?}", cc.wait_for_output());
}

This would do something like that under the hood:

use std::process::{Stdio, Command};
use std::os::unix::io::{IntoRawFd, FromRawFd};

fn main() {
    let res = {
        let mut a = Command::new("ls");
        let mut b = Command::new("sort");
        let mut c = Command::new("uniq");

        a.stdout(Stdio::piped())
            .spawn()
            .and_then(|ac| {
                b.stdin(unsafe { Stdio::from_raw_fd(ac.stdout.unwrap().into_raw_fd()) })
                    .stdout(Stdio::piped())
                    .spawn()
                    .and_then(|bc| {
                        c.stdin(unsafe { Stdio::from_raw_fd(bc.stdout.unwrap().into_raw_fd()) })
                            .stdout(Stdio::piped())
                            .spawn()
                    })
            })
    };

    if let Ok(cc) = res {
        println!("{:?}", cc.wait_with_output());
    }
}

So it would even have to deal with platform specifics, and hide some unsafe code, however, without sacrificing any error handling.

The program above outputs Ok(Output { status: ExitStatus(ExitStatus(0)), stdout: "Cargo.lock\nCargo.toml\nsrc\ntarget\n", stderr: "" }) by the way, so that appears to be one way to do it.

Using a custom trait to extend the existing Command and make the above easier to write would probably be something as well.

In any case, making this workflow more convenient seems very beneficial to me, and something worth investigating a little more :).

[Byron]

As a side-note: It would get even more interesting if this system would be composable, which is probably where the actual difficulties start:

pipe!(pipe!(command!(ls), command!(sort), command!(uniq)), command!(wc -l));

[ArtemGr]

if let Ok(cc) = pipe!(command!(ls), command!(sort), command!(uniq))

This doesn't have to be a macro though, does it? It might as well be a function, like pipe (&[command! (ls), command! (sort), sommand! (uniq)]). Less namespace pollution that way.

Maybe even a newtype-kind of struct implementing std::ops::BitOr, like Pipe (command! (ls)) | command! (sort) | command! (uniq) or use Pipe; command! (ls) | command! (sort) | command! (uniq).

[Byron]

@ArtemGr Certainly that would be fine as well. The benefit of Macros usually is the highly optimized/specialized syntax, but in the end it can surely be handled by functions and types.