Tag Archives: CTP

PowerShell 3 – Finally on the DLR!

For those of you living in a cave: PowerShell 3 will be released in Windows 8, and we got a CTP at roughly the same time as the Windows 8 Developer Preview was released (at Microsoft’s new //Build/ conference in September 2011). A second CTP was released just in time for Christmas.

I’ve been playing with PowerShell 3 for a few months now, and I guess it’s long past time I started blogging about it.

There are a lot of new things coming in this release, but for me, the biggest change is the fact that PowerShell is now based on the Dynamic Language Runtime, a runtime environment that adds a set of services for dynamic languages to the Common Language Runtime (CLR), which is the core of the .NET Framework. The DLR makes it easier to develop dynamic languages to run on the .NET Framework. Of course, PowerShell is a dynamic language that runs on the .NET framework, but it was originally begun before the DLR had been released, so it’s only now that it’s finally been adapted to the DLR.

However, although PowerShell 3 is implemented using the DLR, it’s not a DLR language in every way that IronPython or IronRuby are. Let me borrow a couple of graphics from the DLR overview documentation.

The DLR Overview

DLR Overview

You can see there’s three major sections to the DLR as it’s available on CodePlex: hosting, runtime, and language. However, not all of the DLR actually shipped in the .NET Framework 4.0 CLR.

The DLR shipped in CLR 4

DLR Shipped in CLR4

PowerShell 3 takes advantage of all (or most) of what shipped in the CLR, but since the PowerShell team wasn’t willing to be responsible for shipping the rest of the DLR in the operating system, they didn’t implement the rest of it. Which is to say, PowerShell 3 is using the DLR Language Implementation code, with Shared AST and Expression trees, as well as the DynamicObject and Call Site Caching portions of the runtime, but none of the Common Hosting pieces like ScriptRuntime, ScriptScope, ScriptSource, or CompiledCode …

This means that you cannot use the same hosting APIs for PowerShell that you use for IronPython or IronRuby. However, even though you’re stuck using the same hosting APIs that you used with PowerShell 2 … you do get to use dynamic instead of PSObject when you’re working with the output in C#.

This really is a big deal

I wouldn’t care to speculate how many of the changes you’ll see in PowerShell 3 are directly due to the conversion to the DLR, but there are a few changes that you ought to be aware of. The first thing that you’ll probably notice is the difference in execution and performance. Anything you’ve learned about the relative performance of Scripts vs. Functions vs. Cmdlets and the load time of binary vs. script modules is going to go right out the window with PowerShell 3, as scripts and functions are now no longer (re)interpreted each time they’re run, but are compiled, executed, and (sometimes) cached. The result is that initial runs of scripts and imports of script modules are sometimes slower than they used to be, but subsequent runs of the same script or execution of functions from script modules run much faster, and this applies in particular to actual scripts in files and pre-defined functions in modules. Running a function repeatedly is now much faster than pasting the same code repeatedly into the console.

A more subtle, but significant difference is the change to PSObject.

In PowerShell 3, PSObject is a true dynamic object, and thus the output of cmdlets or scripts called in C# can be used with the dynamic keyword in C# instead of with the pseduo-reflection methods which are required for working with PSObject. However, this is just the tip of the iceberg, so to speak.

In PowerShell 2, all of PowerShell’s Extended Type System (ETS) was based on PSObject. New members were always added to a PSObject which is wrapped around the actual “BaseObject” — regardless of whether they came from a types.ps1xml file, or from calling Add-Member on an object. If you use Add-Member on strongly objects that are not already wrapped in a PSObject, you have to specify the -Passthru parameter and capture the output in order to have your object wrapped into a PSObject that the new member can be added to. In addition, when you cast an object to a specific type, those ETS members are mostly lost. Take this script for example:


$psObject = Get-ChildItem
$psObject.Count
$Count1 = ($psObject | where { $_.PSIsContainer }).Count

[IO.FileSystemInfo[]]$ioObject = Get-ChildItem
$ioObject.Count
$Count2 = ($ioObject | where { $_.PSIsContainer }).Count

$Count3 = ($ioObject | where { $_ -is [IO.DirectoryInfo] }).Count
 

In PowerShell 2, $Count1 and $Count3 will be the number of folders in the current directory, but $Count2 will always be ZERO, because the PSIsContainer property is actually an ETS property that’s lost when you cast the object to FileSystemInfo (and therefore it always evaluates as null and

However, in PowerShell 3 that’s no longer true. PowerShell now works with everything as dynamic objects, and Add-Member no longer needs the PSObject to keep track of these ETS members. This script will now get $Count1, $Count2, and $Count3 equal, as expected. Obviously the -Passthru switch on Add-Member is only needed when you’re trying to pipeline things, and not for simple assignments. However, there may also be other implications on when things get wrapped into a PSObject, and when it matters.

I think you’ll agree that having PowerShell on the DLR is awesome! But be aware that there are a few inconsequential breaking changes hiding in this kind of stuff. For example, after running that script above, try these three lines on PowerShell 2 and PowerShell 3 CTP2:


$Count1 -eq $Count2
$e = $ioObject[0] | Add-Member NoteProperty Note "This is a note" -Passthru
$f = $ioObject[0] | Add-Member NoteProperty Note "This is a note" -Passthru
 

In PowerShell 2, you’ll get False, and then the next two lines will work fine. In PowerShell 3 the first line will return True, and since Add-Member actually affects the underlying object even when it’s not wrapped in a PSObject, the third line will actually cause an error, because “Add-Member : Cannot add a member with the name “Note” because a member with that name already exists.”

Anyway, I’m sure I’ll have more to write about the DLR and the changes it’s bringing to PowerShell, but for now, I hope that’s enough to get you thinking ;-)

WPF From PowerShell – The Missing Walkthrough

If you’ve been reading my series of articles about using WPF from PowerShell and wishing I had started with the basics about what WPF is … you’re in luck. Following my series of posts, the PowerShell team has written their own series — with a much more methodical approach. Starting with an introduction to WPF and PowerShell followed by a great overview of how to use some of the WPF controls from PowerShell and how to handle WPF events , they have now moved beyond the basics of XAML and PowerShell to show you how you can make GUI a part of your PowerShell pipeline like I did with my Select-Grid post. With their most recent post, they explain how to run WPF in a background runspace and they promise tomorrow they’ll expand on that to “make controls in the background stream data and talk to the main runspace” — these two topics are what I was going get to next, so this frees me up to play with something more interesting :-D ….

PowerShell 2 CTP2 — Wow!

The PowerShell team has released the second CTP of PowerShell 2 and it’s got so much new stuff in it, that it’s honestly hard to know where to start, so this first post is just going to be a list of the things I find the most exciting.

STA Threading

You can run PowerShell in Single Threaded Apartment mode, which will let us create WPF/XAML and some neat UI tricks that we couldn’t do before.

Modules

Modules are … like classes. They export public functions, and can have private functions and variables… you basically import the functions into your runspace, so you can call them, and they can share data via variables and even other functions in the same module. This is really awesome, and will almost certainly improve a dozen or more of my scripts which I’ve been using as containers for sets of functions.

Events

PowerShell has support for events, including a whole set of cmdlets: Register-ObjectEvent, Register-PSEvent, Wait-PSEvent, Remove-PSEvent, Unregister-PSEvent, Get-PSEvent, New-PSEvent, Get-PSEventSubscriber which will, in many cases, seem extremely familiar to users of x0n’s PSEventing snapin.

Constrained Runspaces

I haven’t had a chance to test this out yet, but the word is that we can create constrained runspaces that limit access to commands, scripts, and language elements. I’m not sure if this lets you prevent access to part of the .Net framework, but it looks interesting in terms of running other peoples scripts.

Splatting Operator

When you’ve got an array, you can split it up and pass it to a function so each element in the array shows up as a separate argument to the function, so if you have a function foo can call a function bar and pass it all the arguments that get passed to foo, without needing to know what they are or how many there are. (in v1, you have to do things like this: iex "&'bar' $args" which just gets ugly).

Verbose, Warning and Debug streams

I haven’t figured out how to really do anything with this yet, but the release notes say that these streams now contain string messages, invocation information and even pipeline metadata like counts of how many times each command in a pipeline has been invoked — someone was just asking for this on the usenet newsgroup the other day …

[new] Add-Type

There’s no help for this cmdlet, but it’s primarily a wrapper around the .net compilers (CSharp, CSharpVersion3, VisualBasic, and JScript, by default). However, it can also replace all the [Reflection.Assembly]::Load* methods so you can save some typing. I’ve figured out a couple of uses so far, and both are really useful:


# This is equivalent to [Reflection.Assembly]::LoadWithPartialName("System.Web")
Add-Type -Assembly System.Web
# And this is equivalent to [Reflection.Assembly]::LoadFrom( (ls PsXmppClient.dll) )
Add-Type -Path PsXmppClient.dll

# And this syntax lets you compile your own C# code on the fly to create custom types
# So next time you needed to complete Scripting Games 2008 event 8, you could use this:
Add-Type @"
using System;
public struct Song {
   public String Artist;
   public String Name;
   public TimeSpan Length;

   public Song(string artist, string name, TimeSpan length)
   {
      Artist = artist;
      Name = name;
      Length = length;
   }
}
"
@
 

Incidentally, there’s no help for Add-Type for some reason, but you can get basic information about how to use a cmdlet by running a command like this: (Get-Command Add-Type).Definition -replace "`n","`n`n"

PowerShell 2.0 CTP – Script Cmdlets

Well, the first alpha CTP release of PowerShell 2.0 is out, and there’s a lot of new stuff in it … but I won’t repeat the list from the PowerShell blog, because I’m sure you’ve seen it five or six times already. Instead, lets just skip straight to talking about one of the features we’ve been hearing about the longest: in PowerShell 2, you can create Cmdlets in script … bringing nearly full parity between whats possible in a C# cmdlet and what’s possible in script.

There are a few caveats still (Parameter Sets aren’t working yet, and neither is help, really), and a few surprises … there’s a few downsides to PowerShell script vs C# ... but in this particular context one thing that stands out is that in C# the BeginProcessing, ProcessRecord, and EndProcessing blocks are actually methods which can call each other, and as demonstrated in my tutorial for writing cmdlets that work in the pipeline, they can be recursive — without getting duplicate variables.

A sample Script Cmdlet

In the interests of being the first to publish an interesting script cmdlet ;) and to continue my recent trend of talking about writing for the PowerShell pipeline, I’ve merged the logic of my script function and my pipeline cmdlet into a single sample script cmdlet for PowerShell 2.0 and it works great!

A few observations from the process, in no particular order:

  • If you recursively call your cmdlet from within itself, you have to test for parameters using the new $CommandLineParameters.ContainsKey because parameter variables keep their values through recursion if you don’t explicitly pass a value.
  • $CommandLineParameters.ContainsKey works differently in the Begin block where it will return $false for arguments which will get their values from the pipeline, than in the Process block where it will treat values which were passed as CommandLineParameters the same as those which were passed via the pipeline.
  • If you want to see how your function behaves in a pipeline, you should make sure to test it at different points in the pipeline: at the front, in the middle, and at the end.
  • Cmdlets are functions: they show up in the Function provider.
  • Cmdlets are functions: they have to be dot-sourced before you can call them.
  • Cmdlets are not functions: they are a single command Cmdlet which takes a name (which must have a – in it) and a couple of other parameters followed by a function script block.
  • When you recurse by executing &($MyInvocation.InvocationName), that second invocation has an InvocationName of “&” ... so you can’t go any further (this might be a good thing, if you want to stop recursion at one level no matter what. If you want to go further, you need to put your commands into a string, and use Invoke-Expression.

#requires -version 2.0
###################################################################################################
## A Template for Script Cmdlets which can _also_ be executed in the pipeline ....
##   by Joel Bennett, in hopes it will help...
## Version History
## v1.0 - First public release (after over 9 different versions in my various other functions)
## v1.2 - Show the use of Write-Output, and change "return" in the BEGIN to "Write-Output" to avoid
##      the pooling of the output from the process block when it's invoked as a function.
## v1.3 - Switched back to "break" instead of "return" so that if you pass via the pipeline AND via
##      the inputObject, only the inputObject gets process (this is how cmdlets behave).
##      - Cleaned up the comments, and removed the confusing alternate method and $args handling
##
## v2.0 - First Version as a Script Cmdlet.
##      This is much easier with support for [ValueFromPipeline] and [ValueFromPipelineByName]
##
###################################################################################################
Cmdlet Test-PipelineV2 -ConfirmImpact low -snapin Huddled.Tests
{
   Param (
      [Position(0)] [ConsoleColor] $Color,
      [Position(1)] [Mandatory] [ValueFromPipeline] [String[]] $InputObject
   )
   BEGIN {
      if ($CommandLineParameters.ContainsKey("InputObject")) {
         # Don't do anything here, because we're about to get re-invoked...
         $FromArgs = $true
      } else {
         # Normal "run-once" BEGIN processing
         $FromArgs = $false
         Write-Verbose "Begin $Color"
      }
   }
   PROCESS {
      # We no longer have to test for $_ or even to see if the [ValueFromPipeline] param is set
      # It *HAS* to be set, because it's a [Mandatory] parameter :)
      if ($FromArgs) {
         # Don't do anything here except re-invoke ourselves.
         Write-Output $InputObject | &($MyInvocation.InvocationName) $Color
      } else {
         # Normal Pipeline-friendly per-item processing
         Write-Host "Process: $InputObject" -Fore $Color
         ## You should make a practice of explicitly calling Write-Output on things
         ## That's how you emit them into the pipeline instead of just printing them
         Write-Output $InputObject
      }
   }
   END {
      if ($FromArgs) {  
         # Don't do anything here ... it just confuses things
      } else {
         # Normal "run-once" END processing
         Write-Verbose "End $Color"
      }
   }
}
 

A test case


## Test Script:
##
## "a","b","c" | Test-PipelineV2 "Cyan" -verbose
## @("a","b","c") | Test-PipelineV2 "Cyan" -verbose
## Test-PipelineV2 "Cyan" @("a","b","c") -verbose
##
## "a","b","c" | Test-PipelineV2 "Cyan" -verbose | Test-PipelineV2 "Green" -verbose
## @("a","b","c") | Test-PipelineV2 "Cyan" -verbose | Test-PipelineV2 "Green" -verbose
## Test-PipelineV2 "Cyan" @("a","b","c") -verbose  | Test-PipelineV2 "Green" -verbose
###################################################################################################
## Expected Output (sorry, no color here...)

VERBOSE: Begin Cyan
Process: a
a
Process: b
b
Process: c
c
VERBOSE: End Cyan
VERBOSE: Begin Cyan
Process: a
a
Process: b
b
Process: c
c
VERBOSE: End Cyan
VERBOSE: Begin Cyan
Process: a
a
Process: b
b
Process: c
c
VERBOSE: End Cyan
VERBOSE: Begin Cyan
VERBOSE: Begin Green
Process: a
Process: a
a
Process: b
Process: b
b
Process: c
Process: c
c
VERBOSE: End Cyan
VERBOSE: End Green
VERBOSE: Begin Cyan
VERBOSE: Begin Green
Process: a
Process: a
a
Process: b
Process: b
b
Process: c
Process: c
c
VERBOSE: End Cyan
VERBOSE: End Green
VERBOSE: Begin Green
VERBOSE: Begin Cyan
Process: a
Process: a
a
Process: b
Process: b
b
Process: c
Process: c
c
VERBOSE: End Cyan
VERBOSE: End Green