xargs
Is a very useful program to take a list and run commands against that list. xargs will take a list of arguments, loop through them and run a command against 1 or more arguments at a time.
Basic Example
Here is a really straightforward example of using xargs to calculate a MD5 hash on every file in the current directory,
Code Block |
---|
|
ls | xargs -t -n1 md5 |
This is how it works,
- -t will show you what xargs is about to execute before it executes it.
- -n1 specifies that xarg work with he arguments passed by the directory to x arguments at a time, in this case one argument at a time.
Thanks to the -t the output will be shown on screen,
Code Block |
---|
md5 planetary.doc
MD5 (bash) = ab5970d50d67bcafe5c554387f76534e
md5 Superman.jpg
MD5 (cat) = cdefa50d737dfcf8dc57886ea1a758c4
|
Substitution to Rename Files
Now let's get more advanced and use -I to allow substitution. First we'll create a some temporary files,
Code Block |
---|
|
mkdir temp
cd temp
touch files1 file2 file3 # Creates 3 empty files |
Now using xargs we will add the txt extension to each file,
Code Block |
---|
ls | xargs -t -I {} mv {} {}.txt
mv file1 file1.txt
mv file2 file2.txt
mv file3 file3.txt |
The -I {} specifies that the arguments by ls will be placed in the location of the {} called the replacement string. In fact you use whatever variable name you want instead of {}. For example, the following will also work,
Code Block |
---|
|
ls | xargs -t -I varX md5 varX
md5 file1.txt
MD5 (file1.txt) = d41d8cd98f00b204e9800998ecf8427e
md5 file2.txt
MD5 (file2.txt) = d41d8cd98f00b204e9800998ecf8427e
md5 file3.txt
MD5 (file3.txt) = d41d8cd98f00b204e9800998ecf8427e |
Also, one item I don't understand yet is that it seems to default to 1 argument at a time.
Debugging Xargs with echo
The echo command is useful to test and see what xargs will be looping through,
Code Block |
---|
|
ls | xargs -I {} echo "mv {} {}.txt"
mv file1 file1.txt
mv file2 file2.txt
mv file3 file3.txt |
Search Inside of Files
Try to memorize this command,
Code Block |
---|
|
find [folder] -type f | xargs -I {} grep -li "text" {}
find [folder] -type f {search the specified folder for all files, returns full path of each file}
| xargs -I {} grep -li "[text]" {} {piped into xargs to grep for all files containing specified text ignoring case} |
Warning |
---|
More details should be added for this xargs command or if possible rewrite it in a way that is more clear. |
Search & Replace Inside of Files
Try to memorize this command,
Code Block |
---|
|
find [folder] -type f | xargs -I {} grep -li "text" {} | xargs perl -pi -e 's/[text_to_search_for]/[text_to_search_for]/g'
find [folder] -type f # search the specified folder for all files, returns full path of each file
| xargs -I {} grep -li "[text]" {} # piped into xargs with to grep for all files containing specified text ignoring case}
| xargs perl -pi -e 's/[text_to_search_for]/[text_to_replace_with]/g' # pipe list of files and search/replace with specified text} |
Warning |
---|
More details should be added for this xargs command or if possible rewrite it in a way that is more clear. |
Disk Management
List directories from largest to smallest at the top level only.
Code Block |
---|
du -sk * | sort |
Long Running Processes
Other Useful Commands
...
Detailed Articles
Useful One Off Recipes
Disk Management
List directories from largest to smallest at the top level only. On older system h will not work and you must use k.
Code Block |
---|
|
du -sh * | sort |
Long Running Processes
Look for long running processes,
Information
Last
Last searches back through the file /var/log/wtmp (or the file designated by the -f flag) and displays a list of all users logged in (and out) since that file was created.
Lastb
Lastb is the same as last, except that by default it shows a log of the file /var/log/btmp, which contains all the bad login attempts.
Quck String Manipulation
Quick reference of manipulating standout,
Code Block |
---|
|
echo "some_file_name.txt" | tr "_" " " # result will be "some file name.txt"
paste oldName.txt newName.txt > runMe.txt # connects line by line to contents of both files together |
Integrity Checking
MD5
Check if a file is corrupt,
Code Block |
---|
|
# On Ubuntu
md5sum /path/to/file
# On Solaris
digest -a md5 -v /path/to/file |