The problem with calling legacy/native apps from PowerShell

This post is an explanation of the major problems with invoking native apps from PowerShell 2.0, and the simple work-around. There is going to be a little bit of code and then quite a bit of sample output (along with some pointers so you don’t have to play spot the differences). In order to avoid keeping you waiting while I get to the workaround, I’ll let out the secret right here at the top, because it is really very simple: use Start-Process and a here-string to make sure PowerShell does not mess with your arguments (more on this later).

So, first, to demonstrate the differences, lets create an executable which will use various methods to access its command-line parameters and print them out. We will do this in C# although you could do it in C++ and see some variations because the .Net runtime does some parsing of it’s own, which is a little weird — but this simple example should be enough to demonstrate the differences.

The code here is in PowerShell format, so you can simply paste it into PowerShell and get an executable as output, which you can then move to C:\Windows\System32 for testing purposes.


Add-Type -Type @"
using System;
internal class ArgsTest
{
   private static void Main(string[] args)
   {
      Console.WriteLine();
      /*
         I've commented this out because (at least in C#)
         it is the same as Environment.GetCommandLineArgs()
         Except that GetCommandLineArgs shows parameter 0 as the executable path
      */
      // Console.WriteLine("
Using args:");
      // Console.WriteLine();
      // for (int i = 0; i < args.Length; i++)
      // {
      //    Console.WriteLine("
  Arg {0} is: {1}", i, args[i]);
      // }
      // Console.WriteLine();
      // Console.WriteLine();
     
      Console.WriteLine("
CommandLine:");
      Console.WriteLine();
      Console.WriteLine("
  " + Environment.CommandLine);
      Console.WriteLine();
      Console.WriteLine();
     
      Console.WriteLine("
CommandLineArgs:");
      Console.WriteLine();
      string[] arguments = Environment.GetCommandLineArgs();
      for (int i = 0; i < arguments.Length; i++)
      {
         Console.WriteLine("
  Arg {0} is: {1}", i, arguments[i]);
      }
      Console.WriteLine();
      Console.WriteLine();
     
      System.Threading.Thread.Sleep(10000);
   }
}
"
@ -OutputAssembly ArgsTest.exe -OutputType ConsoleApplication
 

To test things, we are going to invoke it in PowerShell, in CMD.exe (DOS), and using the Run dialog (and finally, using Start-Process, to demonstrate that it has the same output as the Run dialog).

A note about path problems

In order to see the differences in how paths are handled, I put ArgsTest.exe in C:\Windows\System32\ — this allows me to call it with no path at all, with a full path, or with a partial path. That turned out to be important because the first difference of PowerShell is that it always executes the app with a FULL path, even if you’re calling it with a partial path.

The reason for this is simple: PowerShell doesn’t ever set the “CurrentDirectory” in it’s operating evironment when you Set-Location (presumably because they figured that often that location would be in a non-FileSystem provider). Because it doesn’t set the current directory, it always needs to invoke applications and scripts with their full path name.

The test case

We will execute the following (long) command-line:


ArgsTest.exe -this is, a, $Home, %UserProfile%   /test  -extra spaces /slashes:"including spaces" -dashes:"including 'quotes'" -and:'some colons' /with:'single $Home quotes'
 

And to get us started, lets look at what that looks like when you execute it from cmd:


CommandLine:

   ArgsTest.exe  -this is, a, $Home, C:\Users\jbennett   /test  -extra spaces /slashes:"including spaces" -dashes:"including 'quotes'" -and:'some colons' /with:'single $Home quotes'


CommandLineArgs:

   Arg 0 is: ArgsTest.exe
   Arg 1 is: -this
   Arg 2 is: is,
   Arg 3 is: a,
   Arg 4 is: $Home,
   Arg 5 is: C:\Users\jbennett
   Arg 6 is: /test
   Arg 7 is: -extra
   Arg 8 is: spaces
   Arg 9 is: /slashes:including spaces
   Arg 10 is: -dashes:including 'quotes'
   Arg 11 is: -and:'some
   Arg 12 is: colons'
   Arg 13 is: /with:'single
   Arg 14 is: $Home
   Arg 15 is: quotes'
 

Notice that in this output, Environment.CommandLine is exactly what was typed!
Now, let’s see the same thing from the run dialog:


CommandLine:

   "C:\Windows\system32\ArgsTest.exe" -this is, a, $Home, C:\Users\jbennett   /test  -extra spaces /slashes:"including spaces" -dashes:"including 'quotes'" -and:'some colons' /with:'single $Home quotes'


CommandLineArgs:

   Arg 0 is: C:\Windows\system32\ArgsTest.exe
   Arg 1 is: -this
   Arg 2 is: is,
   Arg 3 is: a,
   Arg 4 is: $Home,
   Arg 5 is: C:\Users\jbennett
   Arg 6 is: /test
   Arg 7 is: -extra
   Arg 8 is: spaces
   Arg 9 is: /slashes:including spaces
   Arg 10 is: -dashes:including 'quotes'
   Arg 11 is: -and:'some
   Arg 12 is: colons'
   Arg 13 is: /with:'single
   Arg 14 is: $Home
   Arg 15 is: quotes'
 

The thing I want you to notice is that the results are almost exactly the same. The only difference is that cmd shows you just “ArgsTest.exe” which is what was actually typed, whereas the Run dialog inserts the full path to the executable.

But now, take a look at that same command in PowerShell:


CommandLine:

   "C:\Windows\system32\ArgsTest.exe"  -this is a C:\Users\jbennett %UserProfile% /test -extra spaces "/slashes:including spaces" "-dashes:"including 'quotes'"" "-and:"some colons"" "/with:single $Home quotes"


CommandLineArgs:

   Arg 0 is: C:\Windows\system32\ArgsTest.exe
   Arg 1 is: -this
   Arg 2 is: is
   Arg 3 is: a
   Arg 4 is: C:\Users\jbennett
   Arg 5 is: %UserProfile%
   Arg 6 is: /test
   Arg 7 is: -extra
   Arg 8 is: spaces
   Arg 9 is: /slashes:including spaces
   Arg 10 is: -dashes:including
   Arg 11 is: 'quotes'
   Arg 12 is: -and:some
   Arg 13 is: colons
   Arg 14 is: /with:single $Home quotes
 

A Diagnosis

The one thing we expected, of course, is that Powershell interpreted the $Home variable instead of the UserProfile variable, but that’s hardly the most significant difference …

In an attempt to translate what you wrote into what they think you meant, PowerShell has rewritten all of the quoting in the command line!

  • First, since PowerShell understands that commas are simply array separators, they simply remove them completely when calling a “native” app. PowerShell is actually taking the array, and converting the array into a string with space separators. They don’t seem to respect $OFS when doing this, so I don’t really know what the justification is for doing it.
  • Second, PowerShell changes single quoted strings into double quoted strings (without expanding the variables) to make sure that what PowerShell thinks of as strings are also thought of as strings by the target application (of course, they wouldn’t be strings in DOS, but this is PowerShell). If your executable depends on getting single quotes in the arguments, you’re going to have some pain because of that, otherwise, you won’t even notice.
  • Third, PowerShell actually MOVES some quotes, like the one after /slashes … Obviously PowerShell doesn’t treat / as a switch, so it treats them as any other string character. Since in PowerShell, if you don’t quote the WHOLE string, you can still start quoting halfway through it when you realize you need to type spaces … they move the quote to the start of the string so that your app will see the same thing a cmdlet would have.
  • Finally, PowerShell INSERTS extra quotes around -dashes. This is the one thing I can’t really explain right now. PowerShell inserts extra quotes which result in what looks like a real problem in the CommandLine string due to double-double quotes … but somehow those escape parsing as an escaped double quote character when you look at the output of GetCommandLineArgs.

Work arounds

You can get the same results from GetCommandLineArgs when invoking from a PowerShell command by quoting your single quotes and commas, knowing that you need to use $Env: instead of %%, and escaping your dollar signs:


ArgsTest.exe -this "is," "a," "`$Home," $Env:UserProfile   /test  -extra spaces /slashes:"including spaces" "-dashes:including 'quotes'" "-and:'some" "colons'" "/with:'single" `$Home "quotes'"
 

However, if the app you’re calling relies on parsing CommandLine, instead of using args[] or GetCommandLineArgs(), then you have to do it differently, and you can’t ever know that until you try it…

The best workaround, therefore is to use Start-Process. Of course, if you have UserProfile or other environment variables, then you still have to replace those with $Env: and escape your other dollar signs, but if you don’t … you can just use a single-quote here-string. Note the minor difference:


Start-Process ArgsTest @'
-this is, a, $Home, %UserProfile%   /test  -extra spaces /slashes:"including spaces" -dashes:"including '
quotes'" -and:'some colons' /with:'single $Home quotes'
'
@
 

CommandLine:

   "C:\Windows\system32\ArgsTest.exe" -this is, a, $Home, %UserProfile%   /test  -extra spaces /slashes:"including spaces" -dashes:"including 'quotes'" -and:'some colons' /with:'single $Home quotes'


CommandLineArgs:

   Arg 0 is: C:\Windows\system32\ArgsTest.exe
   Arg 1 is: -this
   Arg 2 is: is,
   Arg 3 is: a,
   Arg 4 is: $Home,
   Arg 5 is: %UserProfile%
   Arg 6 is: /test
   Arg 7 is: -extra
   Arg 8 is: spaces
   Arg 9 is: /slashes:including spaces
   Arg 10 is: -dashes:including 'quotes'
   Arg 11 is: -and:'some
   Arg 12 is: colons'
   Arg 13 is: /with:'single
   Arg 14 is: $Home
   Arg 15 is: quotes'
 

And finally, we have it!


Start-Process ArgsTest @"
-this is, a, `$Home, $Env:UserProfile   /test  -extra spaces /slashes:"
including spaces" -dashes:"including 'quotes'" -and:'some colons' /with:'single `$Home quotes'
"
@
 

CommandLine:

   "C:\Windows\system32\ArgsTest.exe" -this is, a, $Home, C:\Users\jbennett   /test  -extra spaces /slashes:"including spaces" -dashes:"including 'quotes'" -and:'some colons' /with:'single $Home quotes'


CommandLineArgs:

   Arg 0 is: C:\Windows\system32\ArgsTest.exe
   Arg 1 is: -this
   Arg 2 is: is,
   Arg 3 is: a,
   Arg 4 is: $Home,
   Arg 5 is: C:\Users\jbennett
   Arg 6 is: /test
   Arg 7 is: -extra
   Arg 8 is: spaces
   Arg 9 is: /slashes:including spaces
   Arg 10 is: -dashes:including 'quotes'
   Arg 11 is: -and:'some
   Arg 12 is: colons'
   Arg 13 is: /with:'single
   Arg 14 is: $Home
   Arg 15 is: quotes'

 

As you can see, the only difference is the value in Arg 5 … but this last attempt actually results in a completely compatible CommandLine and CommandLineArgs. Of course, if you need to capture the output, then you need to append: -Wait -RedirectStandardOutput StdOut.log; gc StdOut.log … but that won’t work if you need to use the application interactively (although, if you just want to script an interactive app, there is also a -RedirectStandardInput parameter).

In any case, if your app is non-interactive, you can use something like this:


&{
Start-Process ArgsTest @"
-this is, a, `$Home, $Env:UserProfile   /test  -extra spaces /slashes:"
including spaces" -dashes:"including 'quotes'" -and:'some colons' /with:'single `$Home quotes'
"
@ -NoNewWindow -Wait -RedirectStandardOutput stdout.log; gc stdout.log; rm stdout.log
}
 

But still

If you need to run an app in the PowerShell window, interact with it (e.g.: answer a prompt), and then capture, process, or redirect the output … you need to invoke the app “natively” ... not using Start-Process. The bottom line is that PowerShell still needs a way to invoke apps that works as simply as Start-Process, but that invokes them inline in the shell. In the meantime, you should generate that ArgsTest executable (from poshcode) to help you figure out the right combination of quotation magic to convince PowerShell 2.0 to pass the right values!

Similar Posts:

3 thoughts on “The problem with calling legacy/native apps from PowerShell”

  1. Note: if you are using the PowerShell Community Extensions module, we’ve had “EchoArgs.exe”, which does the same as ArgsTest, for years. :-)

    1. Yep, the only problem with EchoArgs is that it doesn’t show Environment.CommandLine … which is what makes analysis (specifically, my discovery that PowerShell replaces all quotes with double quotes) possible…

Comments are closed.