How to grep standard error stream (stderr)?

Question

I am using ffmpeg to get the meta info of an audio clip. But I am unable to grep it.

    $ ffmpeg -i 01-Daemon.mp3  |grep -i Duration
    FFmpeg version SVN-r15261, Copyright (c) 2000-2008 Fabrice Bellard, et al.
      configuration: --prefix=/usr --bindir=/usr/bin 
      --datadir=/usr/share/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib
      --mandir=/usr/share/man --arch=i386 --extra-cflags=-O2 
      ...

I checked, this ffmpeg output is directed to stderr.

$ ffmpeg -i 01-Daemon.mp3 2> /dev/null

So I think that grep is unable to read error stream to catch matching lines. How can we enable grep to read error stream?

Using nixCraft link, I redirected standard error stream to standard output stream, then grep worked.

$ ffmpeg -i 01-Daemon.mp3 2>&1 | grep -i Duration
  Duration: 01:15:12.33, start: 0.000000, bitrate: 64 kb/s

But what if we do not want to redirect stderr to stdout?

Answer 1

If you're using bash why not employ anonymous pipes, in essence shorthand for what phunehehe said:

ffmpeg -i 01-Daemon.mp3 2> >(grep -i Duration)

Answer 2

None of the usual shells (even zsh) permit pipes with the | operator other than from stdout to stdin. But all Bourne-style shells support file descriptor reassignment (as in 1>&2). So you can temporarily divert stdout to fd 3 and stderr to stdout, and later put fd 3 back onto stdout. If stuff produces some output on stdout and some output on stderr, and you want to apply filter on the error output leaving the standard output untouched, you can use { stuff 2>&1 1>&3 | filter 1>&2; } 3>&1.

$ stuff () {
  echo standard output
  echo more output
  echo standard error 1>&2
  echo more error 1>&2
}
$ filter () {
  grep a
}
$ { stuff 2>&1 1>&3 | filter 1>&2; } 3>&1
standard output
more output
standard error

---

Ksh, bash and zsh support pipes on arbitrary file descriptors, but they work differently: one command is the main command, and you can pipe its input or output into another command. Process substitution runs a command in the background with either its standard input or its standard output connected to a pipe which you can use in a redirection on the main command. Here you want to filter the standard error, so redirect it to an output process substitution.

$ stuff () {
  echo standard output
  echo more output
  echo standard error 1>&2
  echo more error 1>&2
}
$ filter () {
  grep a
}
$ stuff 2> >(filter)
standard output
more output
standard error

Note that there are two > characters with a space in between: that's 2> to redirect standard output and >(…) to make a process substitution. The space is necessary because 2>>… would be parsed as an append redirection.

---

To make this approach function like the first part of the answer, redirect the output of the filter command back to stderr:

    ︙
$ stuff 2> >(filter >&2)

Answer 3

This is similar to phunehehe's "temp file trick", but uses a named pipe instead, allowing you to get results slightly closer to when they are output, which can be handy for long-running commands:

$ mkfifo mypipe
$ command 2> mypipe | grep "pattern" mypipe

In this construction, stderr will be directed to the pipe named "mypipe". Since grep has been called with a file argument, it won't look to STDIN for its input. Unfortunately, you will still have to clean up that named pipe once you are done.

If you are using Bash 4, there is a shortcut syntax for command1 2>&1 | command2, which is command1 |& command2. However, I believe that this is purely a syntax shortcut, you are still redirecting STDERR to STDOUT.

Answer 4

Gilles and Stefan Lasiewski's answers are both good, but this way is simpler:

ffmpeg -i 01-Daemon.mp3 2>&1 >/dev/null | grep "pattern"

I am assuming you don't want ffmpeg's stdout printed.

How it works:

• pipes first
  • ffmpeg and grep are started, with ffmpeg's stdout going to grep's stdin
• redirections next, left to right
  • ffmpeg's stderr is set to whatever its stdout is (currently the pipe)
  • ffmpeg's stdout is set to /dev/null

Answer 5

See below for the script used in these tests.

Grep can only operate on stdin, so therefore you must convert the stderr stream in a form that Grep can parse.

Normally, stdout and stderr are both printed to your screen:

$ ./stdout-stderr.sh
./stdout-stderr.sh: Printing to stdout
./stdout-stderr.sh: Printing to stderr

To hide stdout, but still print stderr do this:

$ ./stdout-stderr.sh >/dev/null
./stdout-stderr.sh: Printing to stderr

But grep won't operate on stderr! You would expect the following command to suppress lines which contain 'err', but it does not.

$ ./stdout-stderr.sh >/dev/null |grep --invert-match err
./stdout-stderr.sh: Printing to stderr

Here's the solution.

The following Bash syntax will hide output to stdout, but will still show stderr. First we pipe stdout to /dev/null, then we convert stderr to stdout, because Unix pipes will only operate on stdout. You can still grep the text.

$ ./stdout-stderr.sh 2>&1 >/dev/null | grep err
./stdout-stderr.sh: Printing to stderr

(Note that the above command is different then ./command >/dev/null 2>&1, which is a very common command).

Here's the script used for testing. This prints one line to stdout and one line to stderr:

#!/bin/sh

# Print a message to stdout
echo "$0: Printing to stdout"
# Print a message to stderr
echo "$0: Printing to stderr" >&2

exit 0

Answer 6

bash can redirect stdout to stdin stream via regular pipe - | It also can redirect both stdout and stderr to stdin by |&

Answer 7

When you pipe the output of one command to another (using |), you are only redirecting standard output. So that should explain why

ffmpeg -i 01-Daemon.mp3 | grep -i Duration

doesn't output what you wanted (it does work, though).

If you don't want to redirect error output to standard output you can redirect error output to a file, then grep it later

ffmpeg -i 01-Daemon.mp3 2> /tmp/ffmpeg-error
grep -i Duration /tmp/ffmpeg-error

Answer 8

You may swap the streams. This would enable you to grep the original standard error stream while still getting the output that originally went to standard output in the terminal:

somecommand 3>&2 2>&1 1>&3- | grep 'pattern'

This works by first creating a new file descriptor (3) open for output and setting it to the standard error stream (3>&2). Then we redirect standard error to standard output (2>&1). Finally standard output is redirected to the original standard error, and the new file descriptor is closed (1>&3-).

In your case:

ffmpeg -i 01-Daemon.mp3 3>&2 2>&1 1>&3- | grep -i Duration

Testing it:

$ ( echo "error" >&2; echo "output" ) 3>&2 2>&1 1>&3- | grep "error"
output
error

$ ( echo "error" >&2; echo "output" ) 3>&2 2>&1 1>&3- | grep -v "error"
output

Answer 9

I like to use the rc shell in this case.

First install the package (it's less than 1MB).

This is an example of how you would discard stdout and pipe stderr to grep in rc:

find /proc/ >[1] /dev/null |[2] grep task

You can do it without leaving Bash:

rc -c 'find /proc/ >[1] /dev/null |[2] grep task'

As you may have noticed, the syntax is straightforward, which makes this my preferred solution.

You can specify which file descriptor you want piped inside brackets, just after the pipe character.

Standard file descriptors are numbered as such:

• 0 : Standard input
• 1 : Standard ouptut
• 2 : Standard error

Answer 10

A variation on the bash subprocess example:

echo two lines to stderr and tee stderr to a file, grep the tee and pipe back out to stdout

(>&2 echo -e 'asdf\nfff\n') 2> >(tee some.load.errors | grep 'fff' >&1)

stdout:

fff

some.load.errors (eg stderr):

asdf
fff

Answer 11

ls FileNotExist  |& grep -v acc

ls FileNotExist 2<&1 | grep -v acc

In this example, command1 is executed, and both its output and any error messages are piped directly into command2. This can be particularly useful when you want to filter or process the output and errors of a command in the same way.

The |& operator is equivalent to 2>&1 |, which explicitly redirects stderr (file descriptor 2) to stdout (file descriptor 1), and then pipes the combined output to another command.

Remember, this operator is specific to the Bash shell, and might not be available in other shells. If you’re using a different shell that doesn’t support |&, you would need to use the longer form 2>&1 | to achieve the same result.

Answer 12

try this command

• create file with random name
• send output to the file
• send content of the file to pipe
• grep

ceva --> just a variable name (english = something)

ceva=$RANDOM$RANDOM$RANDOM; ffmpeg -i qwerty_112_0_0_record.flv 2>$ceva; cat $ceva | grep Duration; rm $ceva;