In order to use these new classes, you have to use Add-Type to import the System.IO.Compression.FileSystem assembly. If you don’t understand that, don’t worry about it, just copy the first line of the examples below. You only have to do it once, so it might be worth sticking it in your profile (or in a module file with a couple of scripts) if you think you’ll do a lot of this sort of thing.

That’s basically a one-liner to zip up a folder. Of course, you should have a one liner to unzip it too:

In order for the second example to work the way you want it to, you need a zip file where all the files are in a single folder — which is why in the first example I used the overload for CreateFromDirectory which takes a boolean for whether or not to include the root directory in the zip as opposed to just it’s contents. Otherwise, you would need to create the “WASP” folder, first, and then extract to that directory. Of course, if you do that when there is a directory in the zip (as there was, in this case), you’ll end up with a Modules\WASP\WASP ... which in PS3 will work (although it shouldn’t), but is rather frustrating.

So, to avoid ending up with folders inside folders, we can use the ZipArchive class.

The easiest way to get an actual ZipArchive is to use the “Open” method on the ZipFile class. Once you’ve done that you can easily check all the files in it: $archive.Entries | Format-Table and you can extract a single entry using ExtractToFile or the whole archive using ExtractToDirectory …

So for a final example, I’ll use ZipArchive to create a script that will always unzip to a new folder (unless there’s just one single file in the zip), and will create an “archive name” folder if there isn’t already a single folder root inside the archive. In fact, as you’ll see below, I went further and forced the resulting folder to always end up named after the archive.

Update and continuation

I just posted a new version of this to PoshCode including a New-ZipFile function, just because I can’t stop myself, and I’ve always wanted a zip function that behaved exactly the way I wanted it to. If anyone want’s to add to that ZipFile module, that would be the place to do it, for now. Here’s the current version:

However, sometimes you need to prompt the user for input and then do some work, and you want to present the users with output as the work progresses. With any graphical user interface, when you want to do any significant amount of work, you need to do it on a background thread. The ShowUI way to do that is to use the Invoke-Background command, which gives you an event-driven way to run a script block (or command) and capture all of it’s various outputs.

I’ll show you an example here which writes output and progress events, and show you how to handle updating the user interface with both (hopefully you’ll be able to figure out the other output options, including the errors).

The first key, of course, is knowing that Invoke-Background sets a DataContext on the object (either the object specified to the -Control parameter, or the parent who’s event handler calls it). That DataContext is a ShowUI.PowerShellDataSource (it’s code is in the C# folder in the ShowUI module), and it has events and properties for each of the streams: Output, Error, Warning, Verbose, Debug, and Progress. It also has a property for the “Last” item from each of those streams, and a TimeStampedOutput property which collects all of the output, in order, with a NoteProperty for the Stream and the TimeStamp. More importantly, it has events which fire on every output.

The second key is knowing that the properties like “Output” contain all the output, and properties like “LastOutput” contain only the very last thing output. Since in the case of the progress we only care about the very last value, we’re able to use the “LastProgress” property. Since we want to show all of the output, rather than try to collect the “last” one each time, we can just overwrite the textbox’s text with the output array.

It’s also very important to remember that the properties are arrays, and in particular to remember that the Output is an array of PSObject, and the others are arrays of ProgressRecords, DebugRecords, ErrorRecords, etc… so we use Out-String to make sure that we convert them to a string before we set the Text of our output pane.

Hopefully this function will help you out — I’ll try to improve it’s help and examples in the next release of ShowUI.

With the new Update-Help command, the PowerShell team has made it possible to Save-Help to disk, and then move that help and update servers that are offline or behind strict firewalls. However, there’s no built-in way to download help for a module that’s not installed on your local client … and you can’t use the output of Save-Help to read the help on your development box if you don’t have the module installed there.

My new module HelpModules aims to solve both those problems with two commands:

New-HelpModule

New-HelpModule will let you generate a simple module stub that contains just enough information to convince Update-Help and Save-Help to do their jobs. What’s more, it works on the pipeline so you can use Invoke-Command to get the module information from remote servers and pipe it straight into New-HelpModule:

This example would actually list all the modules from a server named “Server1” that have updatable help, generate stubs for them in your local $PSModuleHelpRoot (more about that later), and update the help files (locally).

You can also generate a stub by hand, given the information about the module. In other words, call your mate up on the phone, have them run Get-Module Hyper-V -List | Format-List * and then read you the GUID, Version, and HelpInfoUri … then, you just run:

StubFunctions

The second problem we have is that we can’t run Get-Help on commands that don’t exist on our system. There are two ways around that, using this module. The simplest is to just pass the -StubFunctions switch when you’re calling New-HelpModule. This will generate empty function stubs for each command that’s in the original module — they have no parameters, no code, nothing.

StubFunctions are be enough to let Get-Help work on those commands, but you’ll have to add the $PSModuleHelpRoot to your $Env:PSModulePath in order to take advantage of it. The problem with that is that you’ll pollute your session with modules and commands that don’t really exist (or at least, don’t do anything). Incidentally, I promised more information about PSModuleHelpRoot:

PSModuleHelpRoot

This variable is exported by the HelpModules module, and it’s the path to where New-HelpModule will generate modules (and where Get-ModuleHelp will read from: more on that next). The path defaults to a “WindowsPowerShellHelpModules” folder in your Documents directory, but you can set it to anything you like after importing the module.

Get-ModuleHelp

Get-ModuleHelp is basically a simplified version of Get-Help that works straight on the XML files in your $PSModuleHelpRoot modules. Instead of searching for commands, it searches for help.

It basically works the same as Get-Help, so I’m not going to bother with documentation here — the point is, unlike Get-Help, this doesn’t require you to add $PSModuleHelpRoot to your $Env:PSModulePath, and thus doesn’t add empty modules and commands to your session. It’s a little harder to work with, since you have to know what help you have available, and you have to type the full command name (no wildcard support) but that seemed worth it to me.

Get HelpModules from PoshCode.org (you’ll want to save it as “HelpModules.psm1” to a path in your PSModulePath like: ~\Documents\WindowsPowerShell\Modules\HelpModules\HelpModules.psm1

The Windows 8 App Certification requirements has one particular requirement that makes me (as a life-long scripter) very unhappy:

3.9 All app logic must originate from, and reside in, your app package

Your app must not attempt to change or extend the packaged content through any form of dynamic inclusion of code or data that changes how the application interacts with the Windows Runtime, or behaves with regard to Store policy. It is not permissible, for example, to download a remote script and subsequently execute that script in the local context of your app package

Bottom line: you cannot write extensible apps for the Windows Store. In fact, although Windows PowerShell is shipped even on Windows RT, you can’t use it from a certified Windows 8 Store app.

I don’t know about you, but the apps that I use on a regular basis are almost all extensible, and most of them have both plugins and scripting:

• Visual Studio (Thank goodness for NuGet, ReSharper, StyleCop, GhostDoc, NCrunch etc)
• Notepad++ and Sublime Text 2 and PyCharm
• PowerShell and ConEmu
• Microsoft Office: Word, Excel
• KeePass
• Firefox, and even Chrome and IE

• XChat and even Trillian

I’ve been using Windows 8 for months now, but every app pinned on my taskbar is extensible, and leaving aside video games, I can only see three apps I’ve used in the last month which aren’t readily extensible: PeaZip (which does have some scripting capabilities, but I don’t use them since I script from PowerShell), Zune, and Trillian (which is technically extensible, but all the plugins I use ship in the box).

Even Windows File Manager has shell extensions.

Now, I’m not saying I won’t use an app that’s not extensible … but without even thinking about it, most of the apps I use are scriptable and/or extensible, and I bet that’s true of most of the apps you use too. As a side note, one of the coolest new phone apps from Microsoft is on{x}, an automation app which is only available on Android (and can’t ever pass validation on the Windows Store because of this policy).

So yeah. Most of the stuff I do with computers is about automation, scripting, robotics… or gaming. I can’t see myself getting really fired up about that App Store stuff.

Let me know when 3.9 is revoked.

Now, I have faith in Microsoft. I’m sure they’re not trying to kill off running multiple windows on a desktop, but I don’t understand why they would write terms in their certification requirements that would prevent an app like Sublime Text 2, KeePass, or Firefox from being written. I certainly hope that they can be convinced to rewrite that constraint to allow for users who choose to install modules and scripts.

As a side note, there’s another point in there that I’m not too happy with either:

4.4 Your app must not be designed or marketed to perform, instruct, or encourage tasks that could cause physical harm to a customer or any other person

We would consider an app that allows for control of a device without human manipulation, or that is marketed for use to resolve emergency or lifesaving situations to violate this requirement.

At first, that one seemed fine. But when you read the detail, it’s clear that any app that is for robotics/AI and wants to interface with external devices is basically going to be refused. Your Lego Mindstorms apps are only allowed if they’re remote controls which require human manipulation, because they … might cause harm?

As long as we’ve got desktop mode and sideloading of non-certified apps, we’re ok (I guess), but Microsoft needs to stop limiting certified apps before they alienate the hackers and tinkerers. I’m a big fan (and author) of Open Source software, but I don’t want a world where all the commercial software companies lock out the geeks and our only option is Open Source.

In order to test this tip, you’re going to need a command console that you’ve used and typed several commands into, so that Get-History will return more than a few different commands. Now, actually run the Get-History command so you can see the list of your last few commands.

The basic tip is pretty simple: if you type “#” and then hit the Tab key, PowerShell completes the full previous command-line.

You can also hit tab again to complete the next oldest command-line, and so on, right back to the beginning (it actually wraps around). You can even hit Shift-Tab to reverse direction if you go past the command-line you wanted. Additionally, this works on your history, so it even completes multi-line items. The one weird thing is that if you tab-complete past an item with multiple lines, the TabExpansion function doesn’t realize the cursor’s not on the prompt line anymore, so it doesn’t quite redraw right, but it’s mostly ok: the commands still work.

Of course, if that’s all there was to this, I’d just have tweeted and gone back to preparing for my presentation at the Windows Server 2012 Launch Event in Rochester …

The really cool thing is that you can filter the feature. That is: if you type # and then the first few characters of some command-line in your history, when you hit tab you will get the most recent command-line that starts with those characters, and as before, you can hit tab repeatedly to cycle through all the commands in your history that match.

There’s one more part to the hash-tab feature: numbers. If you know the history id of the command you want to type, you can type, for instance, #20{Tab} to complete the 20th command from your PowerShell session. It’s basically the same as using “r” shortcut for invoke-history, except you hit tab after the number instead of space before, and you get to see the command (and edit it) before you press Enter.

So to sum up:

• hash-tab – completes command-lines from your history
• hash-txt-tab – filters your history like get-history | where { $_.Commandline -Like txt* }

• hash-id-tab – completes the command from history with the matching id

Get the SSDT projects to compile with parameters from MSBuild

The problem is that with the new .sqlproj and the SSDT\Microsoft.Data.Tools.Schema.SqlTasks.targets there’s no built-in way to pass the database name or even a connection string when building the project via MSBuild — which is a critical part of our continuous integration builds. The project I’m working on these days has 7 or 8 teams working in up to twice that many branches, and all those branches need CI builds, every one of which deploys the database project and validates it. Since the branches change at least once a week, it’s way too much work to run around modifying publish.xml files to change database names every time we create a new branch (which we need to do to avoid the builds deploying over the top of each other).

With the old .dbproj format, there was a SQLDeploy task called in the TeamData\Microsoft.Data.Schema.SqlTasks.targets build target file which included a whole bunch of variables that could be overridden on the command line, so we could pass TargetDatabase and TargetConnectionString as MSBuild arguments, and then, to be able to compile the whole solution and still call the Deploy target, we added this to the project file:

In our workflow, we redefine MSBuildArguments in the workflow, and now we can msbuild the whole solution and the database will be deployed:

But that doesn’t work with the new SSDT project type.

First of all, they don’t deploy with all their dependencies, instead, they have to be published. It’s basically the same thing, with a different name. But the SqlPublish task requires all the parameters to be in an xml file, and there’s no build properties we can override, because the properties are hiding in that publish.xml file that doesn’t get tokenized

I’ve spent the last couple of days figuring out a work around, so I figured I should blog it up here and help the next guy. The process is not simple. The bottom line is that I haven’t found a way to get the SQLPublish task to take it’s values from anywhere except a publish xml file, so the solution I came up with was to rewrite the publish file using the XDT transform tasks defined for Web.config transforms.

Web.Config Transforms, on random XML files

The cool thing is, there’s actually a ParameterizeTransformXml task which allows you to define your transform as a string in the build file.

In web projects, that task is used to replace connection strings (to hide them in web packages), but we can use it to replace the database name in our publish.xml. In fact, I can actually add the same properties to the build that we used to have with the old project format (and which we’ll define on the command-line, exactly the way we did before). We put some default properties for TargetDatabaseName and TargetConnectionString in our Debug.publish.xml and our CI.publish.xml and then we just replace them during the build.

It’s a lot more complicated than what we had to do previously, partly because we need to define the SqlPublishProfilePath for the Publish task, but we have to use CallTarget to call the Publish target (not Deploy this time), which doesn’t support passing properties, nor does the target you call inherit properties that are defined in your scope. This means we need to define the SqlPublishProfilePath property in the BeforePublish target which the publish target depends on (the “dependson” relationship inherits defined properties).

But the real work is actually setting up the TransformPublishXml property with the right XML to replace the nodes with the properties from the command-line arguments, and then actually calling the task. Since we imported the task before, we just need a property group to define our variables with default values, and then a BeforeBuild target to actually call the ParameterizeTransformXml:

So all you have to do is put those three blocks of XML at the bottom of your .sqlproj file, and then call msbuild with /p:TargetDatabase=DBName;TargetConnectionString="Data Source=DBServer;User ID=sa;Password=password";DBDeployOnbuild=True to get the database project to build and deploy to the database you want.

If you’ve got questions, post ‘em — I’m writing this at 1:30 in the morning so I’m not at my most lucid

In PowerShell, Get-Command is a command that serves two purposes: first it lets you search for commands using verb, noun, wildcards, module names etc. and then it also returns metadata about commands. In PowerShell 2, it could only search commands that were in modules (or snapins) you had already imported, or executables & scripts that were in your PATH.

So here’s the deal: Get-Command has always behaved differently when it thinks you’re searching. The only way it can tell that you’re searching is that you don’t provide a full command name. So, if you use a wildcard (e.g.: Get-Command Get-Acl* or even Get-Command Get-Ac[l]), or search using a Noun or Verb (e.g.: Get-Command -Verb Get or Get-Command -Noun Acl or even Get-Command -Verb Get -Noun Acl), then PowerShell assumes you’re searching (and won’t throw an error when no command is found).

In PowerShell 3, because modules can be loaded automatically when you try to run a command from them, Get-Command had to be modified to be able to return commands that aren’t already loaded. The problem the PowerShell team faced is that in order to get the metadata about a command, they needed to actually import the module. What they came up with is that if you’re searching … then Get-Command will not load modules which aren’t already loaded. If you specify a full command name with no wildcards, then PowerShell will load any module(s) where it finds a matching command in order to get the metadata (parameter sets, assembly info, help, etc). And of course, if you specify a full command that doesn’t exist, you’ll get an error!

Perhaps a few examples will help:

Launch PowerShell 3 using:

And then try this, noticing how much more information you get when you specify a specific full name:

But there are several problems:

Get-Command has another parameter: -Module, which allows you to specify which modules should be searched, and in PowerShell 3, it changes the behavior in weird (buggy) ways:

1. If you specify a single module, then that module is imported (to search it more thoroughly?), even if you specify a specific command that’s not in that module.
2. If you specify a single module that does not have a command that matches, then Microsoft.PowerShell.Management is loaded also. I don’t know why yet.
3. If you specify more than one module, and you’re searching, and none of them have a command that matches … it’s just as though you hadn’t specified modules, and nothing unexpected happens.
4. If you specify more than one module, and a specific command, then it gets really wierd:
   • If the command is in one (or more) of the specified modules, the first module (in PATH order, not the order you specified) which you listed that has the command is imported.
• If it’s a valid command in a different module, the first module with the command is loaded … and so is Microsoft.PowerShell.Management. I don’t know why! Oh, and you still get the error because it can’t find the command where you told it to look.

I filed a bug on Connect to cover that last scenario where the module containing the command is loaded even though you gave Get-Command a list of modules to look in, here’s another example, and notice that even though all I do here is run the same command over and over (I added some Get-Module to show you WHY you get these results, but it’s the same without them), but I get different results:

The markup generated is (I hope) reasonable and lightweight, with some simple css rules pre-applied. Feel free to customize the script to generate help however you like.

Generating many at once

I forgot to mention the other parameters on Get-HtmlHelp. They’re pretty cool, because if you want to upload your help you can do so with this. Say you created a module, and you wanted to generate all the help into files for uploading. You want to set the -BaseUrl to the location you will upload them to, and then use the -OutputFolder parameter to generate an html file for each command into the specified folder:

Now you can just take those files and upload them to the right spot on your website. I actually have some scripts which I can wrap around this to post the help to a wiki, but you’re just going to have to wait for that until the next time I get inspired to work on help …

Show-Help

I did include a little function in the comments and in the help for Get-HtmlHelp which uses ShowUI to display the rich formatted help in a popup window:

So anyway, enough of that. When you run it, it looks like this (click for the full screenshot):

One Catch

If you try this with IE and you get the error Unexpected error launching Internet Explorer. Protected Mode must be set to the same value (enabled or disabled) for all zones ... it means exactly what it says. You need to open “Internet Options” from your start menu (or from IE), and go through each “zone” and set the “Enabled Protected Mode” check box to the same value for each zone (either all checked, obviously the most secure, or all unchecked). I’m not going to debate whether setting them all unprotected is a good idea … I set mine to all protected, but I don’t generally use IE anyway.

If you want more help, Selenium’s documentation is great, and there’s a section on Getting Started with Selenium WebDriver which I found quite helpful (make sure your examples are in “csharp” and you can almost just copy and paste — someone should offer to do them in PowerShell).

If you want more information about the Internet Explorer driver and this problem in particular, the short answer is that “Protected Mode” is a security boundry, so if you cross over it the COM automation object doesn’t work — thus, you need to make sure you always stay on the same side. There’s a good discussion on the mailing list archive about how it works and why, as well a weird alternative documented on the Selenium JavaDocs

I’ve been playing with PowerShell 3 for a few months now, and I guess it’s long past time I started blogging about it.

There are a lot of new things coming in this release, but for me, the biggest change is the fact that PowerShell is now based on the Dynamic Language Runtime, a runtime environment that adds a set of services for dynamic languages to the Common Language Runtime (CLR), which is the core of the .NET Framework. The DLR makes it easier to develop dynamic languages to run on the .NET Framework. Of course, PowerShell is a dynamic language that runs on the .NET framework, but it was originally begun before the DLR had been released, so it’s only now that it’s finally been adapted to the DLR.

However, although PowerShell 3 is implemented using the DLR, it’s not a DLR language in every way that IronPython or IronRuby are. Let me borrow a couple of graphics from the DLR overview documentation.

The DLR Overview

You can see there’s three major sections to the DLR as it’s available on CodePlex: hosting, runtime, and language. However, not all of the DLR actually shipped in the .NET Framework 4.0 CLR.

The DLR shipped in CLR 4

PowerShell 3 takes advantage of all (or most) of what shipped in the CLR, but since the PowerShell team wasn’t willing to be responsible for shipping the rest of the DLR in the operating system, they didn’t implement the rest of it. Which is to say, PowerShell 3 is using the DLR Language Implementation code, with Shared AST and Expression trees, as well as the DynamicObject and Call Site Caching portions of the runtime, but none of the Common Hosting pieces like ScriptRuntime, ScriptScope, ScriptSource, or CompiledCode …

This means that you cannot use the same hosting APIs for PowerShell that you use for IronPython or IronRuby. However, even though you’re stuck using the same hosting APIs that you used with PowerShell 2 … you do get to use dynamic instead of PSObject when you’re working with the output in C#.

This really is a big deal

I wouldn’t care to speculate how many of the changes you’ll see in PowerShell 3 are directly due to the conversion to the DLR, but there are a few changes that you ought to be aware of. The first thing that you’ll probably notice is the difference in execution and performance. Anything you’ve learned about the relative performance of Scripts vs. Functions vs. Cmdlets and the load time of binary vs. script modules is going to go right out the window with PowerShell 3, as scripts and functions are now no longer (re)interpreted each time they’re run, but are compiled, executed, and (sometimes) cached. The result is that initial runs of scripts and imports of script modules are sometimes slower than they used to be, but subsequent runs of the same script or execution of functions from script modules run much faster, and this applies in particular to actual scripts in files and pre-defined functions in modules. Running a function repeatedly is now much faster than pasting the same code repeatedly into the console.

A more subtle, but significant difference is the change to PSObject.

In PowerShell 3, PSObject is a true dynamic object, and thus the output of cmdlets or scripts called in C# can be used with the dynamic keyword in C# instead of with the pseduo-reflection methods which are required for working with PSObject. However, this is just the tip of the iceberg, so to speak.

In PowerShell 2, all of PowerShell’s Extended Type System (ETS) was based on PSObject. New members were always added to a PSObject which is wrapped around the actual “BaseObject” — regardless of whether they came from a types.ps1xml file, or from calling Add-Member on an object. If you use Add-Member on strongly objects that are not already wrapped in a PSObject, you have to specify the -Passthru parameter and capture the output in order to have your object wrapped into a PSObject that the new member can be added to. In addition, when you cast an object to a specific type, those ETS members are mostly lost. Take this script for example:

In PowerShell 2, $Count1 and $Count3 will be the number of folders in the current directory, but $Count2 will always be ZERO, because the PSIsContainer property is actually an ETS property that’s lost when you cast the object to FileSystemInfo (and therefore it always evaluates as null and

However, in PowerShell 3 that’s no longer true. PowerShell now works with everything as dynamic objects, and Add-Member no longer needs the PSObject to keep track of these ETS members. This script will now get $Count1, $Count2, and $Count3 equal, as expected. Obviously the -Passthru switch on Add-Member is only needed when you’re trying to pipeline things, and not for simple assignments. However, there may also be other implications on when things get wrapped into a PSObject, and when it matters.

I think you’ll agree that having PowerShell on the DLR is awesome! But be aware that there are a few inconsequential breaking changes hiding in this kind of stuff. For example, after running that script above, try these three lines on PowerShell 2 and PowerShell 3 CTP2:

In PowerShell 2, you’ll get False, and then the next two lines will work fine. In PowerShell 3 the first line will return True, and since Add-Member actually affects the underlying object even when it’s not wrapped in a PSObject, the third line will actually cause an error, because “Add-Member : Cannot add a member with the name “Note” because a member with that name already exists.”

Anyway, I’m sure I’ll have more to write about the DLR and the changes it’s bringing to PowerShell, but for now, I hope that’s enough to get you thinking