Huddled Masses

Archive for May, 2008

So, people keep talking about a better PowerShell script repository, and there’s at least 3 or 4 different people working on improvements, but for now, the one we have is working ok, and already has hundreds of scripts in it — but there hasn’t been a way to add or retrieve scripts from within PowerShell.

So while I was taking a break from working on my own replacement script repository I whipped up some scripts to Send-Paste, Get-Paste and Find-Paste. These scripts still need some work, but I think you’ll find them useful, so I’ve posted a version that is compatible with PowerShell v1 (and works fine on the 2.0 CTP) and doesn’t need any additional scripts (it includes the latest Get-WebFile script for downloading the files).

The Find-Paste function takes a search string, and adds wildcards — but otherwise performs the same search as you can already do on the web page. It returns the results in a PowerShell friendly PSObject which by default is just displayed in the host.

In order to download a script using Get-Paste, you must know the ID of the script — which you can get from the output of Find-Paste. Note that because Find-Paste does a full-text search, you may get results that are not the script you’re looking for (if they call that script, for instance), and you may get many versions of the script (and newer versions are not necessarily at the top of the search results).

The Send-Paste script is most useful for sending things to the temporary PowerShell pastebin on my site but can be used successfully to send scripts to the permanent PowerShell Script Repository … in fact, that’s how I uploaded the PowerShell Pastebin Functions to the repository … you just have to be careful to specify the title and description when you do that. Edit: I attached the script here on my site also because I made a mistake and deleted the original copy from the repository. [new]

So, you now have something a little bit more like a proper script repository. Hopefully the Usage comments in the script will be enough to help you figure out how to use the various commands, (and I did blog about the Send-Paste script about a month ago) but just in case you need help, let me give you a simple example.

Suppose I need to find a script to encrypt text so I can store some SecureString objects (like the passwords in a PSCredential object) in a file…

It looks like I found two, but clearly the second one is not the right one (I mentioned this in one of my examples in the pastebin scripts, and amusingly enough, that screwed up my example). So to download the script I want, I just do:

Simple enough, right? I could have used the -Passthru option to see the script in the console, but since I was pretty sure it was the only one that might be useful (and since using -Passthrough just to display it on the console would mean downloading it a second time, if I actually wanted it). Another option would be to use the Tee-Object cmdlet, which would let us download it, show it in the host, and also save it in the file, all at once!

Incidentally, I have a version of this which uses my PoshHttp snapin and Get-Web cmdlet, but it’s also written as a module and therefore requires PowerShell 2.0 CTP2 — for various reasons, I’m choosing to release that one separately later. If you are using the CTP, you could take advantage of the WPF UI abilities and try piping Find-Script ... | "Select-Grid":http://huddledmasses.org/wpf-from-powershell-select-grid/ | Get-Script to pick the script you want visually.

I was just having some fun with some recent blog posts…

WPF & PowerShell – Part 5 has a script for “Get-Listbox” and for “Show-Control” and Halr9000 wrote a script he called Get-PSBlogroll

I had modified the example from the WPF post to create a listbox which will “start” whatever you double click …

And I tried something like this to let me launch links from hal9000’s blogroll:

Of course, what I’d like is to see the titles, instead of the URL, but because of how Get-ListBox was written, it only accepts strings, and therefore only returns strings. I tried adding the URL as a NoteProperty on the title string, but there’s a bug in the script cmdlet implementation that strips ETS properties. So if you specify your parameter type as a [string], you loose the NoteProperty.

Idea 1: Use [PsObject] for your pipeline parameter

Of course, if they fix the bug this tip will go away, but in the meantime, you’re better off not casting the input from the PowerShell native [PsObject] type, because you loose any extended type system attributes (and if you cast XML to a string, it comes out yucky).

Idea 2: Don’t output xml nodes

Even after fixing the Get-Listbox script so it outputs what it gets in without casting it, I still have to craft PsObjects with the output from Halr9000’s script, because otherwise Get-ListBox just outputs the .ToString() representation of the xml node, which is invariably useless — I could data-bind to the xml attributes, but it requires a separate syntax in the XAML — which means you’d have to have separate code for xml vs objects. So basically, you’re better off with a | Select * at the end of your Get-PsBlogRoll.

Idea 3: Display scripts should take a property (or a property list)...

What I eventually ended up doing is rewriting Get-Listbox to take a “Properties” parameter so that I could specify which of the attributes of an object I wanted to use … that way, I can pass in the almost unmodified output from Get-PSBlogRoll into the new Get-ListBox, and run with it. In the end, it looks something like this:

Of course, because of how I wrote the DataTemplate for the properties (using a StackPanel and a loop to output multiple TextBlock objects) you could choose to show the Url as well, and all you’d have to do is change the call to Get-Listbox like this: Get-Listbox title,htmlUrl … which is pretty cool. Does anyone else have any ideas for making these even easier to compose?

A really nice trick would be if the wpf cmdlet could use Write-Output in the click handler and actually yield to allow the rest of the pipeline to process that item, but so far, I haven’t been able to find a way to do that (short of closing the WPF window, of course).

If you’ve been reading my series of articles about using WPF from PowerShell and wishing I had started with the basics about what WPF is … you’re in luck. Following my series of posts, the PowerShell team has written their own series — with a much more methodical approach. Starting with an introduction to WPF and PowerShell followed by a great overview of how to use some of the WPF controls from PowerShell and how to handle WPF events , they have now moved beyond the basics of XAML and PowerShell to show you how you can make GUI a part of your PowerShell pipeline like I did with my Select-Grid post. With their most recent post, they explain how to run WPF in a background runspace and they promise tomorrow they’ll expand on that to “make controls in the background stream data and talk to the main runspace” — these two topics are what I was going get to next, so this frees me up to play with something more interesting ….

Modules are a new feature of PowerShell v2 which were added to CTP2 with very little fanfare and no documentation. After struggling with the demo from the release notes, and searching in vain for help files, I was finally able to figure some of this out with some help from x0n of Nivot Ink and since I still can’t find anything on the web about modules, I figured I’d share what little I know, even though I’m sure the moment I do the PowerShell team will write it up properly on their blog ([new] Edit I was off by three months, but the PowerShell team has finally started explaining these) ... let’s start with a direct quote from the release notes:

Modules allow script developers and administrators to partition and organize their Windows PowerShell code in self-contained, reusable units. Code from a module executes in its own self-contained context and does not affect the state outside of the module. Modules also enable you to define a restricted runspace environment by using a script.

The point of a script module is that it allows much better control over what portions of a script are exposed in your runspace. Code from a module executes in its own self-contained context and does not have to affect the state outside of the module. Modules have access to the $PsScriptRoot variable which holds the path to the folder the script is running in, which allows you to dot-source additional scripts from the module directory, so you can split up your code into multiple files however you like — perhaps a file per function, or whatever works for you. Modules can even define a restricted runspace environment. If you’re a programmer, the best analogy seems to be to think of a powershell script module as basically a “class” with public and private methods and members. Obviously it can be a bit more than that, since cmdlets are rather more than simple methods, but try to keep that model in mind.

One key feature of modules (for scripters) is that the self-contained state of modules is also persistent, and there can be private and public member functions (or cmdlets). This might be the key feature of script modules: Script-scope variables are visible to all functions loaded inside the module, and their state and values are persistent through multiple calls to cmdlets or functions from the same module. They are invisible to code outside the module, which means that multiple functions that work together don’t need to store settings or state in the global scope, and don’t have to worry about name clashes (other than the publically exported members, we’ll get to that later).

Modules are the new Snapins?

As I’ve been experimenting, it turns out there’s way more to this than simple script modules. Modules appear destined to replace snapins as the main way to extend PowerShell. In fact, in the current CTP2 you can load most existing snapins as a module instead, which means you don’t have to be an administrator to load a new snapin — there’s no need to registery them by running InstallUtil.exe — you can simply place them in a folder and tell PowerShell where to find them.

Package Paths?

You can load any script as a module by specifying the full path as it’s name — that’s not the best way to handle modules. The best way is to create a folder with the module’s name inside a package directory. The environment variable PsPackagePath (which I believe will be changed to PSModulePath or even ModulePath in the next release) contains a list of folders where modules can be placed. The defaults are two folders called “Packages,” one in your Documents\WindowsPowerShell and one in $PSHome … the folders aren’t created by the CTP2 installer, and they’ll probably be changed to “Modules” in a future release.

In one of those path folders (or the default locations), you simply create a folder with the package name, and then put your module in it with the same name as it’s base name and either a .ps1, .psm1, .dll or .psd1 extension PSD1 files don’t actually work in CTP2. Having done that, you can load them by name without specifying the path. The cool thing about this is that you can actually create a folder on a USB or network drive, and stick all your modules in it … and then just add that folder to your PsPackagePath environment variable. It’s just like any other path variable, so you can specify multiple directory paths separated by semi-colons.

The key thing to understand (excuse me for repeating myself) is what PowerShell expects: inside the PsPackagePath there should be a folder for each module, and the name of the folder becomes the name by which you call the module — the module file and the folder should have the same name, so for instance if you want to load a script TestModule.ps1 as a module, you need to put it in ~\Documents\WindowsPowerShell\Packages\TestModule\TestModule.ps1 and then you can just Add-Module TestModule. The name of the folder is the name of the module, and the main module file must have the same name even if it’s a dll module.

Here is my TestModule.psm1 script. This example is fun because it demonstrates quite vividly the difference between a psm1 module and a ps1 script:

If you comment out the last line of that script and name it TestModule.ps1 (after putting it in an appropriate folder), you can Add-Module TestModule and run fooize and barize … they work fine. The problem is that the $count variable leaks into your runspace, and since it isn’t initialized as part of the script, you can’t reset it by re-running Add-Module TestModule, even with the -Force parameter, and Remove-Module TestModule won’t work. You can also arbitrarily access the count variable: "The count is $count" will print the current value, and $count = 42 will change it without running the associated code.

However, if you rename it to TestModule.psm1 and restore the Export-ModuleMember call … then when you run it, it has it’s own internal $count variable. In fact, if you do that without resetting your runspace from the previous example, you’ll see that even though your $count is still at 42, the fooize and barize functions are using a separate counter which you can’t modify externally. If you run Add-Module TestModule -Force now, the $count variable will reset!

Cmdlets for Modules

There are so far four cmdlets which are available for dealing with modules, but there isn’t any help or documentation for any of them, short of the release notes, so apart from what I’ve said so far, let me give you their parameter set definitions, since that’s the best we’ve got, and I’ll add my explanations, as I understand them.

Get-Module (returns PSModuleInfo objects)

• Get-Module [[-Name] <String[]>]

Get-Module lets you get all, or some, of the loaded Modules. It returns PSModuleInfo objects for them which can be used in the Add and Remove cmdlets.

Remove-Module

• Remove-Module [-Name] <String[]>

• Remove-Module -ModuleInfo <PSModuleInfo[]>

Oddly enough, in the current CTP Remove-Module doesn’t really do what you expect. As far as I can tell, all that it does, is removes the PSmoduleInfo from the list of loaded modules. This means you can run add-module against it again (otherwise, running Add-Module a second time on a module that’s already loaded does nothing unless you use the -Force parameter) but it doesn’t really do anything else. In fact, you can continue to use cmdlets and functions from these “removed” modules without any problems that I can see so far. I’m not sure if this is considered a bug, or if the team expects to leave it like this. I dearly hope that in the next CTP we will at least see the cmdlets and functions removed so they can’t be executed, especially since I can already remove the functions by hand by simply deleting them from the provider.

Add-Module

• Add-Module [-Name] <String[]> [[-ImportList] <String[]>] [-Force] [-PassThru] [-ScriptBlock <ScriptBlock>]
• Add-Module [-Name] <String[]> [-ScriptBlock] <ScriptBlock> [[-ImportList] <String[]>] [-Force] [-PassThru]

• Add-Module [-Force] [-PassThru] [-ScriptBlock <ScriptBlock>] [-ModuleInfo <PSModuleInfo[]>]

The Add-Module cmdlet is the main cmdlet that provides the functionality of the modules. It loads modules into your runspace, and it can load various types of modules:

1. The first, and most obvious from the release notes, is the script module — a PowerShell script with the extension PSM1 — I’ll come back to these in a bit.
2. The second type is what we now call “Snapins” in v1 — as far as I can tell, you can load nearly any snapin dll as a module if you configure it carefully — although I’m not sure that Providers can be loaded this way, and some of the snapin initialization code isn’t run if you do this. The current PSCX snapin, for instance, intializes many things during its “install” and doesn’t work right if you don’t use InstalUtil and load it as a Snapin.
3. You can also load PSD1 files, which are like metadata files about modules (which as far as I can tell, is how you would load a multi-assembly snapin-type module, or modules which needed type files, etc. like the PSCX snapin).

1. And finally, you can load plain old ps1 script files, which are loaded just as though you had dot-sourced them (which incidentally means they can’t be “Remove“d at all).

I’ll fill in the details of how the ImportList works when I figure it out, I think it allows you to (further) restrict what is exported into your runspace from the module. You can use the -Force parameter to reload scripts you’ve already loaded, and you can specify -PassThru to have the PSModuleInfo object output from Add-Module (the same as running Get-Module)...

When you run Add-Module -Force, it reinitializes the script scope for the module. Take for example the previous script module named TestModule.psm1. If you Add-Module TestModule and then run “fooize” ... each time you run it, the number at the end of the string will increment. When you re-run Add-Module TestModule with the -Force parameter, the “count” variable is reset, but otherwise it remains, and even after you Remove-Module Test you can still run fooize and barize!

Forwarned is Forarmed

If you’re working on a compiled PowerShell snapin, you need to start working right now on what it would take to get it to load as a module, because unlike Snapins, Modules can be loaded by path, and more importantly, do not need installation by an administrator. This is going to mean that when PowerShell 2 becomes final, end users are going to have a clear preference for modules which are easier to trust (since they can be run entirely non-elevated), and easier to work with (since they can be carried on a memory stick or disk and loaded without installing on any computer).

Export-ModuleMember

• Export-ModuleMember [-ExportList] <String[]> [-Update]

The final module cmdlet is actually the second most important one — without it, nothing in your PowerShell script modules would be visible after you load them. Generally speaking, you run Export-ModuleMember and pass it a list of function or cmdlet names which you wish to export — to make them visible from outside the script.

Exporting a module member is analogous to marking a C# member method public, in that it marks it for use by external scripts, etc. You shouldn’t use this cmdlet in a plain .ps1 script (or on the command-line) because it won’t work, and it actually does weird things (like killing my profile) that I can’t explain .

And for my final trick …

In terms of “protecting” your scripts from malicious users you should know that it’s possible to access the “private” variables and methods in a cmdlet, but it’s not possible to “change” them from outside. So for instance, if we take the example TestModule again — you’ll notice there’s a function at the bottom that we haven’t exported and that isn’t used at all — but which would allow us to set the count value. Let me demonstrate why:

After looking over the scripts I’ve pasted in the last few days it struck me that all of them load the UI from XAML — and for the most part, you can do pretty much whatever you need to do in WPF in pure PowerShell if you want to.

To demonstrate this, I wrote a simple Out-Grid script. But then I got carried away, and wrote the script I’ve shown in this screenshot. The resulting script not only illustrates how to code WPF in PowerShell without XAML, it also illustrates one of the few areas where you’re much better off loading from XAML than writing the plain code — and that’s a DataTemplate. It is possible to write them in code, but believe it or not, some of their features are accessible only from XAML.

[new] I’ve updated both the script and the screenshot to correct a bug I noticed last night after I posted this … so if you tried this and got a lot more columns than you had bargained for … here is the Select-Grid script again. [new] I updated it again just now to switch my Set-PsDebug statement for a Set-StrictMode statement, after reading Jeffrey Snover’s post about strict mode best practices.

I’m not going to paste the whole script in-line, but I thought I’d paste some of it here so that I could give a little explanation of what’s happening, in case you need a little help figuring it out…

Using modules …

Some of you may know that normally in a script like this we would put the “utility” functions inside the BEGIN block so-as to encapsulate them from your main runspace, and may be wondering why I didn’t in this case. It’s because the new CTP2 of PowerShell has this concept called a “module” ... which is somewhat like a script snapin — basically it allows you to have functions in the script which don’t end up in the main runspace, but which are still available to the other functions (or cmdlets, in this case) in the script.

I will write a separate post about how PowerShell Modules work, and what you can do with them … it’s too complicated to get into here. Suffice it to say that all I had to do to the script is add a line at the bottom: Export-ModuleMember Select-Grid and rename it to [new] Select-Grid.psm1 and put it in a specific location: Documents\WindowsPowerShell\Packages\Select-Grid\Select-Grid.psm1 — but of course, I couldn’t resist cleaning it up a bit as I was writing about it (if you grabbed it before v3.4, you should grab it again — look in the comment at the top).

Having done that, all you have to do is run Add-Module Select-Grid instead of dot-sourcing it, and then run it as before (see the screenshot).

Let’s talk about code bay-bee …

There’s a few things I wanted to point out in the module’s code, so lets get to that. Read the rest of this entry »

After posting my last post, I started thinking that perhaps I shouldn’t really have started with something so splashy [rolleyes]. So I started thinking about what I could use as a proper example — not of what WPF can do, but of what you might want to use WPF for in PowerShell.

I came up with two really obvious ideas which I think are both good demonstrations of useful things to do from PowerShell, and are good examples of something that really just doesn’t work with WinForms.

User Input prompts

I’ll write more about this in future posts, but for now, take as an ultimate example the latest post from mow, The PowerShell Guy in which he rewrites his PowerShell WMI Explorer to be WPF based — it seems to not only run faster, but be based on less code (of course, that might just be because he’s gained experience since the first version).

Data-bound Graphs

There are dozens of ways you can do data-bound graphs in PowerShell — including 3d — but the simplest to demonstrate (and coincidentally the easiest for me to come up with a use-case for) is a plain bar graph. I’ve written a simple graph window in XAML, and created a script Out-BarGraph which lets you send data to it … but there are, as the Genie says, some caveats, provisos, addendums, and quid pro quos. Read the rest of this entry »

In my last post I wrote about how you could make a WPF Splash screen window in PowerShell, but I stated that: “if the images are remote, the WPF window has to download them, and therefore won’t work” correctly. I had played with downloading images directly by setting the Source attribute of the image to the image URL, and hadn’t been able to find a way to get the threading to work well enough to actually download the image.

Well, I figured that out, so I thought I’d go ahead and share … basically, all you have to do is call Invoke() on the window’s dispatcher. The reason that works is that WPF handles events and “work” in general in the order it needs to be done, so since the most important thing is drawing the UI it will do that first, and since your UI is based on downloading the image, it will take care of that first.

This is PowerShell v2 CTP2 code…

All of this code is written to target the CTP2 release of PowerShell v2 — it will not work in PowerShell v1 or even in v2 CTP1 … and it may not work in later releases, although I expect it will. Also, as with any code which uses WPF from PowerShell, these code examples require you to launch PowerShell with the -STA option.

Having said that, lets just straight to a WPF example Read the rest of this entry »

I mentioned in my last post that the new PowerShell 2 CTP2 release includes a STA switch which lets PowerShell run in Single Threaded Apartment mode. I also mentioned that this means we can do WPF and build UIs in XAML ... but I left it at that.

I thought I should go ahead and give you a better idea what sort of things that makes possible, but I have to admit that I haven’t had a whole lot of time to play with it — and there’s still going to be threading issues unless someone can help me figure out a way to spin out a new thread for the UI. So far everything I’ve tried to get a thread has just managed to crash PowerShell. [oops] Luckily, the first couple of WPF tricks I could think of don’t really need threads at all, so they’ll work fine — lets just dive right in.

A PowerShell Splash Screen

Don’t forget to run PowerShell with the -STA switch in order to try these demos, and before you run any of these scripts, you need to have .Net 3.0 or later installed. There are lots of ways to build WPF UIs, but perhaps the simplest thing is to build it in XAML since PowerShell supports XML natively. The first thing we have to do is add a reference to the WPF library, and then we can have some fun.

As you can see, I’ve also loaded my PoshHttp snapin so I can use Get-Web to retrieve a web page as XML and then load the image — this is mostly about convenience (to save myself the trouble of parsing html as text) but also because if the images are remote, the WPF window has to download them, and therefore won’t work unless you ShowDialog() and give the WPF window the thread. You can just use the path to any image on your hard-drive.

The PowerShell team has released the second CTP of PowerShell 2 and it’s got so much new stuff in it, that it’s honestly hard to know where to start, so this first post is just going to be a list of the things I find the most exciting.

STA Threading

You can run PowerShell in Single Threaded Apartment mode, which will let us create WPF/XAML and some neat UI tricks that we couldn’t do before.

Modules

Modules are … like classes. They export public functions, and can have private functions and variables… you basically import the functions into your runspace, so you can call them, and they can share data via variables and even other functions in the same module. This is really awesome, and will almost certainly improve a dozen or more of my scripts which I’ve been using as containers for sets of functions.

Events

PowerShell has support for events, including a whole set of cmdlets: Register-ObjectEvent, Register-PSEvent, Wait-PSEvent, Remove-PSEvent, Unregister-PSEvent, Get-PSEvent, New-PSEvent, Get-PSEventSubscriber which will, in many cases, seem extremely familiar to users of x0n’s PSEventing snapin.

Constrained Runspaces

I haven’t had a chance to test this out yet, but the word is that we can create constrained runspaces that limit access to commands, scripts, and language elements. I’m not sure if this lets you prevent access to part of the .Net framework, but it looks interesting in terms of running other peoples scripts.

Splatting Operator

When you’ve got an array, you can split it up and pass it to a function so each element in the array shows up as a separate argument to the function, so if you have a function foo can call a function bar and pass it all the arguments that get passed to foo, without needing to know what they are or how many there are. (in v1, you have to do things like this: iex "&'bar' $args" which just gets ugly).

Verbose, Warning and Debug streams

I haven’t figured out how to really do anything with this yet, but the release notes say that these streams now contain string messages, invocation information and even pipeline metadata like counts of how many times each command in a pipeline has been invoked — someone was just asking for this on the usenet newsgroup the other day …

[new] Add-Type

There’s no help for this cmdlet, but it’s primarily a wrapper around the .net compilers (CSharp, CSharpVersion3, VisualBasic, and JScript, by default). However, it can also replace all the [Reflection.Assembly]::Load* methods so you can save some typing. I’ve figured out a couple of uses so far, and both are really useful:

Incidentally, there’s no help for Add-Type for some reason, but you can get basic information about how to use a cmdlet by running a command like this: (Get-Command Add-Type).Definition -replace "`n","`n`n" …

I keep hearing from new users who are used to bash-style aliases, how frustrating it is not to be able to create aliases with parameters, the way you can in bash …

I’m going to show you how to make the “alias” command work roughly the way it does in bash, but first, let me clear this up:

In Bash, the recommendation is that “for almost every purpose, shell functions are preferred over aliases” ... and that recommendation is even stronger for PowerShell. The PowerShell “alias” is a true alias — it’s just another name for a command, script, function, etc. and it doesn’t support passing parameters or making mini-scripts at all.

Usually, an alias serves to give you a name you can remember, but sometimes it’s just to shorten the name, . Another use for them is to let you specify the default cmdlet — if you have two cmdlets (or script functions) with the same name, you can use an alias to override which one is executed by default.

The PowerShell function can do everything a bash alias can do (and everything a function or script can do, but this isn’t a tip about that, it’s a tip about making an alias [new] and unalias command that works the way you expect it to). So …

You can save that as alias.ps1 and create your aliases the way you used to in bash alias ls='ls -recurse' and still be able to invoke them and, you can pass them extra parameters somewhat like in csh when you invoke them. I’ve added some extra text in the function name and content and in the alias description, so it’s actually pretty easy to find all the aliases and functions this script creates and dump them to a file so you can reload them later if you want … but I haven’t actually written a function to do that yet myself.

One important note: You must not use recursive aliases in the bindings on v1 — that is, alias ls='ls -recurse' will loop until it hits PowerShell’s recursive limit (only 100) and exit if you try to use it on v1 — because the script won’t be able to resolve the alias “ls” to Get-ChildItem … you’re probably better off not relying on that feature anyway.