Linux · text processing

xargs

turn a list into arguments

xargs bridges two kinds of command: ones that print a list, and ones that want their input as arguments, not on a pipe — rm, cp, chmod. It reads the list and builds the command line.

find produces names; xargs hands them to the command that acts on them.

Reads

A list

separated by spaces or new lines

Runs

One command

with many arguments at once

One at a time

-n 1 or -I {}

when the command needs it

Safe with spaces

-0

paired with find -print0

At the terminal

Read the session first, then the notes: each numbered line in the output is answered by the note with the same number.

user@netflow — bashfindutils

$ find . -name "*.tmp" | xargs rm -v

1removed './cache/a.tmp'

removed './cache/b.tmp'

2$ echo "one two three" | xargs -n 1 echo item:

item: one

item: two

item: three

3$ find . -name "*.log" -print0 | xargs -0 gzip

4$ cat hosts.txt | xargs -I {} ping -c 1 {}

Illustrative session · names, sizes and dates are examples, and every address is from a documentation or private range

Reading the output

What the numbered lines are telling you

One rm, many files

find printed the names, and xargs ran a single "rm -v ./cache/a.tmp ./cache/b.tmp".

-n 1 runs the command per item

Three separate echo commands, each given one word.

Handling awkward file names

-print0 and -0 separate names with a null byte instead of a new line, so names with spaces are passed correctly.

-I {} places the item

Each line of hosts.txt is put where {} appears, and ping runs once per host.

Switches

A switch changes how the command behaves. These are the ones used day to day; man xargs lists the rest. Single-letter switches can usually be joined, so -l -a may be written -la.

-n NUM: Use at most NUM arguments per command.
-I {}: Replace {} in the command with each input item.
-0: Expect items separated by null bytes, as from find -print0.
-P NUM: Run up to NUM commands in parallel.
-p: Prompt: show each command and ask before running it.
-r: Do not run the command at all if there is no input.

Variations worth knowing

The same tool, pointed at a narrower question. These four cover most of what it is reached for day to day.

grep -rl "old-domain" . | xargs sed -i "s/old-domain/new-domain/g"

Replaces text in every file that contains it.

cat urls.txt | xargs -n 1 -P 4 curl -sO

Downloads four at a time, in parallel.

find . -type f | xargs -p rm

Shows the command and asks before running it.

ls *.jpg | xargs -I {} mv {} photos/

Moves each file into a directory.

In cybersecurity

xargs runs commands built from data it was handed, so the data matters.

File names with spaces

Without -0, a file called "my notes.txt" becomes two arguments, and rm may delete a different file called "my". Use -print0 with -0.

Names chosen by someone else

A file named "-rf" is read as a switch by the command. Put -- before the arguments so nothing after it is treated as an option.

Rehearse destructive pipelines

Replace rm with echo, or use -p, to see the command that would run before letting it loose.

Filed under

Text processing

Read alongside

findWalks a directory tree and prints every entry that passes the tests you give it — by name, age, size, owner or permissions.grepSearches files for lines that match a pattern, and prints the lines it finds.