Search, Filtering, and Text Processing

The command line in Linux provides powerful utilities for searching, filtering, and processing text, enabling users to manipulate data efficiently and extract meaningful information.

Whether handling logs, code, or large datasets, mastering these tools is essential for developers and system administrators aiming to automate tasks and analyze data quickly. 

Search Tools: Finding Text Patterns

grep (Global Regular Expression Print)

The most commonly used tool for searching plain-text data for lines matching a regular expression.

Basic usage: grep "pattern" filename

Recursive search: grep -r "pattern" directory/

Case-insensitive: grep -i

Show line numbers: grep -n

Invert match: grep -v (select non-matching lines)

Example:

Searches recursively, case-insensitive, showing line numbers.

ack, ag (the silver searcher), rg (ripgrep)

Tools optimized for speed and developer workflows supporting Gitignore files, recursive search, and colorful output.

Filtering and Selecting Text: cut, sort, uniq

cut: Extracts sections from each line of input, useful for column-based data.

Parameters:

-d defines delimiter

-f selects fields

Example:

sort: Sorts lines alphabetically or numerically for ordering data.

Example:

uniq: Filters repeated adjacent lines to unique ones, often used after sorting.

Example:

Text Processing Utilities: awk, sed, tr, paste

awk: A programming language for pattern scanning and processing.

It can filter records, perform calculations, and generate reports.

Example:

Extracts and prints second and third fields.

sed (stream editor): Performs basic text transformations like substitution and deletion on input streams.

Example:

Replaces all instances of ‘old’ with ‘new’.

tr (translate): Translates or deletes characters. Useful for simple character-level transformations.

Example:

Converts lower case to upper case.

paste: Merges lines from multiple files horizontally, aligning columns.

Example:

Combining Commands with Pipes

Piping (|) connects the stdout of one command to the stdin of another, enabling complex text processing pipelines.

Example:

Finds unique error messages and sorts by frequency.