...
Table of Contents |
---|
Introduction
Is a very useful program to take a list and run commands against that list. xargs will take a list of arguments, loop through them and run a command against 1 or more arguments, one at a time.
Tip |
---|
If you like xargs you might want to check out GNU Parallel. |
Basic Example
Here is a really straightforward non-destructive example of using xargs to calculate a MD5 hash on every file in the current directory,
Code Block | ||
---|---|---|
| ||
ls | xargs -t -n1 md5md5sum |
This is how it works,
- -t will show you what xargs is about to execute before it executes it.
- -n1 specifies that xarg work with he arguments passed by the directory to x arguments at a time, in this case one argument at a time.xargs will by default take the output of ls one line at a time and append it to the end of the command
Thanks to the -t the output will be shown on screen,
Code Block | ||
---|---|---|
| ||
md5md5sum planetary.doc MD5ab5970d50d67bcafe5c554387f76534e (bash) = ab5970d50d67bcafe5c554387f76534eplanetary.doc md5md5sum Superman.jpg MD5cdefa50d737dfcf8dc57886ea1a758c4 (cat) = cdefa50d737dfcf8dc57886ea1a758c4Superman.jpg |
Substitution to Rename Files
Now let's get more advanced and use -I to allow substitution and explicitly set the location of what xargs receives. First we'll create a some temporary files,
...
Code Block | ||
---|---|---|
| ||
ls | xargs -t -I {} mv {} {}.txt
mv file1 file1.txt
mv file2 file2.txt
mv file3 file3.txt |
The -I{} specifies that the arguments by results from ls will be placed in the location of the {} called the replacement string. In fact you can use whatever variable name you want instead of {}. For example, the following will also workchanging {} to varX also works,
Code Block | ||
---|---|---|
| ||
ls | xargs -t -IIvarX varX md5 varX md5 file1.txt MD5 (file1.txt) = d41d8cd98f00b204e9800998ecf8427e md5 file2.txt MD5 (file2.txt) = d41d8cd98f00b204e9800998ecf8427e md5 file3.txt MD5 (file3.txt) = d41d8cd98f00b204e9800998ecf8427e |
Note |
---|
One item I don't understand yet is why {} forces arguments to be iterated through one at a time. Also, how would we allow more than one argument? -n2 will not work. |
Debugging xargs with echo
The echo command is useful to test and see what xargs will be looping through,
Code Block | ||
---|---|---|
| ||
ls | xargs -I {} echo "mv {} {}.txt"
mv file1 file1.txt
mv file2 file2.txt
mv file3 file3.txt |
Notice that using echo I omit the -t but you will want to put the -t back when you are actually executing your command.
Dealing with Special Characters
When using xargs it will not work with special characters like apostrophe in file names.
To get around this limitation use the find command's -print0 option in combination with -0 which handles special characters white space, quote marks, backslashes, blanks and/or newlines,
Code Block | ||
---|---|---|
| ||
find . -print0 | xargs -0 -I{} echo {} |
This is because "ls" produces slightly different output from "find .". Here is an example,
Code Block | ||
---|---|---|
| ||
# Data I am working with
ls
11 My Baby's Got To Pay the Rent 1.m4a 6 Habits (Stay High) [Hippie Sabotage Remix] 1.txt
11 My Baby's Got To Pay the Rent 1.txt Tin's file.txt
11 Summertime Sadness 1.m4a hello
11 The Troubles 1.m4a pwd
12 Canoeing (Katie and Alex's Theme) 1.m4a test123
# Apostrophe kills xargs here
xargs: unterminated quote
# -0 by itself does not solve the problem
ls | xargs -0 -I{} echo {}
{}
# Now it works.
find . -print0 | xargs -0 -I{} echo {}
.
./11 My Baby's Got To Pay the Rent 1.m4a
./11 My Baby's Got To Pay the Rent 1.txt
./11 Summertime Sadness 1.m4a
./11 The Troubles 1.m4a
./12 Canoeing (Katie and Alex's Theme) 1.m4a
./6 Habits (Stay High) [Hippie Sabotage Remix] 1.txt
./hello
./pwd
./test123
./Tin's file.txt
# Remove a file to show -0 works with smaller data set,
rm test123
# -0 by itself now works, but making any file names longer or adding back test123 breaks it
ls | xargs -0 -I{} echo {}
{}
Kitchen-iMac:tmp tin.pham$ ls | xargs -0 -I{} echo {}
11 My Baby's Got To Pay the Rent 1.m4a
11 My Baby's Got To Pay the Rent 1.txt
11 Summertime Sadness 1.m4a
11 The Troubles 1.m4a
12 Canoeing (Katie and Alex's Theme) 1.m4a
6 Habits (Stay High) [Hippie Sabotage Remix] 1.txt
Tin's file.txt
hello
pwd
# Show's that find looks different than ls and you want to keep that in mind,
find . -print0 | xargs -0 -I{} echo {}
.
./11 My Baby's Got To Pay the Rent 1.m4a
./11 My Baby's Got To Pay the Rent 1.txt
./11 Summertime Sadness 1.m4a
./11 The Troubles 1.m4a
./12 Canoeing (Katie and Alex's Theme) 1.m4a
./6 Habits (Stay High) [Hippie Sabotage Remix] 1.txt
./hello
./pwd
./Tin's file.txt |
Useful Applications of xargs
Search - ...
find . -print0 | xargs -0 -I{} echo {}