Some links on xargs simulation in PowerShell
Posted by jpluimers on 2021/10/13
On nx, I’m used to xargs which allows to convert from a pipe of output into arguments passed to a command. This is useful, as many commands only accept arguments as parameters.
In PowerShell, you can usually avoid an xargs equivalent because commandlet output is a stream of objects that you can post-process using . I for instance used that in PowerShell: recovering from corrupt empty *.nupkg files after a disk was accidentally full during update.
Here are some xargs equivalency examples:
- [WayBack] PowerShell tips for bash users, part 1 • Five
…
xargs
Xargs is one of the most powerfull UNIX commands. It is used to build and execute command lines from standard input. For example:
$ cat dirs | xargs mkdir
will use cat to take the strings (be it newline or blank character separated) from file ‘dirs’ and pass them through pipe to xargs which will then send one by one line as argument to mkdir which will then create those dirs or complain if those are existent.PowerShell equivalent:
PS> cat dirs | %{mkdir $_}There is no ‘xargs’ command in PS, but you can use ‘foreach ‘ loop and pass the piped variable ‘$_’ to the mkdir. Shorthand for ‘foreach’ is ‘%’. This time also only newlines will separate the strings apart. If multiple strings separated by blanks are found in same line, mkdir will create a directory with blanks in the name, while we must quote to have the same in bash:
$ cat dirs | sed 's|^|"|g' | sed 's|$|"|g' |xargs mkdir…
- [WayBack] What’s the equivalent of xargs in PowerShell? – Stack Overflow
Q
The POSIX-defined
xargscommand takes all of the items it receives from standard input and passes them as command-line arguments to the command it receives on it’s own command line. E.g:grep -rn "String" | xargs rm.What’s the equivalent in PowerShell?
The following questions all ask this:
but there is no correct answer because all the answers either use
ForEach-Object, which will process items one-at-a-time (likexargs -n1) which gets the desired result for the examples given, or store the intermediate result in a variable, which offends my functional commandline-fu.A
There are two ways that I’ve found. The first is probably more idiomatic PowerShell, and the second is more true to the pipe-based spirit of
xargs.As an example, let’s say we want to pass all our cat pics to
myapp.exe.Method #1: Command substitution
You can do something similar to using $(command substitution) in
shby embedding your pipeline in the command string:&"myapp.exe" @(Get-ChildItem -Recurse -Filter *.jpg | Another-Step)The
@(...)creates an array from the command inside it, and PowerShell automatically expands arrays passed to&into seperate command-line parameters.However, this does not really answer the question, because it will only work if you have control over the command you want to pass to, which may not be the case.
Method #2: True piping
You can also construct a “double pipeline” by having a sub-expression to pipe your objects, collecting them to an array, and then piping the array to your final command.
,@(Get-ChildItem -Recurse -Filter *.jpg | Another-Step) | %{&"myapp.exe" $_}The
@(...)as before collects the items into an array, and the array is then piped to the final command which is invoked using%(ForEach-Object). Ordinarily, this would then loop over each item individually, because PowerShell will automatically flatten the array when it’s piped, but this can be avoided by prepending the,operator. The$_special variable is then used as normal to embed the passed array.So the key is to wrap the pipeline you want to collect in
,@(...), and then pipe that to something in%{...}.
References
- [WayBack] ForEach-Object
- [WayBack] PowerShell ForEach-Object Parallel Feature | PowerShell
- [WayBack] syntax – What does “%” (percent) do in PowerShell? – Stack Overflow
- [WayBack] PowerShell – Special Characters And Tokens – Welcome to Neolisk’s Tech Blog
% (percentage)
1. Shortcut to foreach.
Task: Print all items in a collection.
Solution.
... | % { Write-Host $_ }2. Remainder of division, same as Mod in VB.
Example:
5 % 2
- [WayBack] PowerShell – Special Characters And Tokens – Welcome to Neolisk’s Tech Blog
- [WayBack] powershell – Why is a leading comma required when creating an array? – Stack Overflow
- [WayBack] about_Operators:
@( )array-subexpression-operator– – PowerShell | Microsoft Docs
Array subexpression operator
@( )Returns the result of one or more statements as an array. If there is only one item, the array has only one member.
@(Get-CimInstance win32_logicalDisk) - [WayBack] about_Operators:
,unary comma operator – PowerShell | Microsoft Docs
Comma operator
,As a binary operator, the comma creates an array. As a unary operator, the comma creates an array with one member. Place the comma before the member.
$myArray = 1,2,3 $SingleArray = ,1
- [WayBack] about_Operators:
- [WayBack] xargs – Wikipedia:
xargs is a command on Unix and most Unix-like operating systems used to build and execute commands from standard input. It converts input from standard input into arguments to a command.
Some commands such as
grepandawkcan take input either as command-line arguments or from the standard input. However, others such ascpandechocan only take input as arguments, which is why xargs is necessary.…
find /path -type f -print | xargs rmIn the above example, the
findutility feeds the input ofxargswith a long list of file names.xargsthen splits this list into sublists and callsrmonce for every sublist.…
xargs often covers the same functionality as the backquote (`) feature of many shells, but is more flexible and often also safer, especially if there are blanks or special characters in the input. It is a good companion for commands that output long lists of files such as
find,locateandgrep, but only if you use-0, sincexargswithout-0deals badly with file names containing ', " and space. GNU Parallel is a similar tool that offers better compatibility with find, locate and grep when file names may contain ', ", and space (newline still requires-0).
–jeroen
#/media/File:Sgpstemmen1.png){kind=link}
Leave a Reply