🔧 Linux: Search, Filter and Output

On some occasions, one may need to filter through the contents of large files. For this, Linux has several commands to read the files and filter their contents simultaneously. Let's go through some examples.

Reading through a file in the command line

When reading through a file using the cat command, you usually see the last line of the file displayed before the next command prompt, requiring you to scroll backward to view the previous content.

With the more and less commands, the content fits on the screen, and you can scroll down to see the rest of it.

Try the following to see the difference.

cat /etc/passwd
cat /etc/passwd | more
cat /etc/passwd | less

Sometimes you may only want to read the beginning or end of a large file. For that, you can use the head or tail command, which will display the first 10 lines and the last 10 lines, respectively. Of course, you can specify a different number of lines with their options.

Use the man command to see the rest of the options.

Searching and filtering contents in a file

Using the grep command, one can search for matching strings or patterns in the contents of a file.

# displays matching lines containing the string "bin" in the file
cat /etc/passwd | grep "bin"
# displays matching lines that don't contain the string "false"
cat /etc/passwd | grep -v "false"
# Show all lines that match the regex pattern
cat /etc/ssh/sshd_config | grep -E "^[^#]"

Outputting the results of a file

Linux manages input and output operations using references or identifiers called file descriptors. The file descriptors for the input data stream are 0, for output, 1, and for error, 2.

Suppose you want to store the output in a file. (By default, the standard output (STDOUT - FD1) is redirected to the file, even if you don't specify the file descriptor 1.)

find /etc/ -name *.conf 1>results.txt
find /etc/ -name *.conf > results.txt

The file will be created if it doesn't exist.

Suppose you want to discard all the errors into a null device before storing or displaying the results.

find /etc/ -name shadow 2>/dev/null > results.txt

Here you specify the file descriptor 2 for the standard error (STDERR - FD2)

You can also log the error to a file.

find /etc/ -name shadow 2> stderr.txt

If you want to append to the already existing file, use >> instead of >

find /etc/ -name shadow 2>> stderr.txt