[xargs command] again, I might not remember it by tomorrow, still i noted it #linux

xargs.md

Parallel download and rename at the same time

https://stackoverflow.com/a/51584237/9122512

Intro

xargs reads lines of text from the STDIN or from the pipelined outputs of another command and turns them into multiple commands and executes them.

xargs usually go with find command.

Syntax

xargs [options] [command [initial-arguments]]

Check the file size of all the files in 1 folder

This won't work: ls -l | awk 'NR > 1 { print $NF }' | du -h

Why? First, the 1st pipeline outputs a list of words seperated by newline character, semantically they're not file names. Second, du don't accept STDIN.

Xargs method

Take the outputs and write it into new commands

ls -l | awk 'NR > 1 { print $NF }' | xargs du -h

Is equilvalent to

du -h lily.txt one_lotus.txt rose.txt three_lotus.txt two_lotus.txt

Xargs and Find

The find command searches for files and directories and return their names. Thanks to xargs, you can use the result of find command for specific purposes: renaming, moving, deleting, ...

Using find and xargs together: find . -type f -name "*.txt" | xargs du -h: if names don't have whitespace find . -type f -name "*.txt" -print0 | xargs -0 du -h: if names have whitespace, if so, the outputs from find commands will be seperated by ASCII null character instead of newline character, and xargs is instucted to accept ASCII null.

NOTE: there is an alternative called find -exec

Options

-t: see what command is being executed

E.g. find . -type f -name "*.txt" | xargs -t touch

-p: force manual confirmation

E.g. find . -type f -name "*.txt" | xargs -p rm

-I: using placeholder

This is used when the order of arguments from output in new command is not right. So, you use -I to defined a placeholder for those arguments, by convention it's {}

E.g. find . -type f -name "*.txt" | xargs -p -I {} mv {} new_dir

NOTE #1: each line from output is seperated into multiple arguments for multiple new commands, not 1 new command with multiple arguments like before.

By using placeholder, you can Run multiple commands

find . -type f -name "*.txt" -print0 | xargs -I {} sh -c 'ls -l {}; du -h {}'

NOTE #2*: If you use placeholder, there won't be pipelining here.

-n: maximum number of arguments passing to each new command

-P: maximum number of processes that can be run concurrently.