Huddled Masses

Posts Tagged ‘PowerShell’

In July of last year I wrote a PowerShell script with the goal of allowing me to generate XML from PowerShell with a simple markup that would look a little like the resulting XML ... this week I was using that script again, and had a couple of issues that made me go back and look at the source.

While I was playing with the source and tweaking things a little bit to improve the way it handles namespaces, I started playing with the idea that I could improve the syntax. At the very least, I thought, I ought to be able to do away with all those “xe” aliases…

Well, I was able to. And what’s more, I managed to dramatically clean up the way namespaces work, and make it so that really, the only ugly part of the syntax is the initial declaration of namespaces! I’m going to start with two examples, and use them to walk you through the features

Example 1

The simplest example I could think of is to list all the files in a folder, with the file size and last modified stamp:

The output of that, when run on my formats folder, looks like this:

You can immediately see what the script does: New-XDocument (which is aliased as ‘xml’) actually generates the root xml node, so the first argument to it is the name of that node, and any other arguments become attributes … except for the script block. That script block turns into the contents of the node.

Inside the script block, PowerShell code is parsed as usual, but whenever a command that doesn’t exist is encountered, it is turned into an xml node! Pretty simple, right? Of course, if you wanted to create a node with a name that’s already taken by a PowerShell command, you can just replace file with New-XElement file, or (using aliases) xe 'file', which explicitly creates an xml node with the given name.

That’s pretty much it for our first example, so let’s look at a more complicated example, with multiple namespaces, and deeper nesting.

Example 2

This time, we’ll create an Atom document, and we’ll include some namespace extensions (including a made up one for listing my files as we did above):

There are four things you should notice, in particular:

First: the initial tag has a [XNamespace] added to it. You can specify a tag name that has a namespace by adding them together this way, or by embedding the namespace in the string like "{http://www.w3.org/2005/Atom}feed" instead. Either way works. This initial namespace becomes the default namespace for the document. If you don’t specify a namespace on tags later, they automatically belong to that one.

Second: when you want to add additional namespaces, you can do so with a custom prefix like: -dc ([XNamespace]"http://purl.org/dc/elements/1.1"), and that prefix (dc) takes on a special meaning. When you want to have a tag later on that is part of that namespace, you just prefix the tag, like dc:rights —the same way you would in XML.

Third: any number of attributes can be specified using the -name value syntax, but anything in a {scriptblock} becomes the content — and is subject to the same rules as the outer sections.

Fourth: This generates an XDocument. When you cast an XDocument to string, the xml declaration is left off, so if you want it, you need to manually add it via $XDocument.Declaration. Incidentally, XDocuments are not XMLDocuments, but they are trivially castable to them.

The output of that particular section of New-XDocument is this:

The New-XDocument script itself is on PoshCode in the Xml Module 4 along with a few interesting functions like Select-XML (which improves over the built-in by being able to ignore namespaces when you write XPath) and Remove-XmlNamespace (which was instrumental in removing namespaces for Select-Xml). There’s also a Format-Xml for pretty-printing, and a Convert-Xml for processing XSL transformations.

I’ll probably post some more examples of this in the next week or two, and I really should write some commentary about the function itself, which uses the tokenizer to discover which “commands” are really xml nodes … but for now, I’ll leave you to enjoy.

Related articles by Zemanta

• JSON from PowerShell (but why?) (huddledmasses.org)
• What’s in a Namespace (apache.sys-con.com)
• Ignoring Namespaces in XPath (huddledmasses.org)

I just put up a poll on the PowerShell Virtual Group to see if people are interested in a low-planning brown-bag event.

The initial question is: would you attend a weekly (or monthly) virtual brown-bag lunch if I put one together.

The idea is that we would start each session with a short collection of interesting links, tips and tricks, or connect issues, and then have a presentation or discussion or script club or open-mic session, depending on interest.

Basically, this is meant to flow, on a week-to-week basis, from script-club to formal user group presentations.

Some weeks we would have presenters from various local user groups share content they had prepared for their local groups. We would encourage you to take ownership of this time, request topics, and prepare presentations (no matter how short).

Some weeks we would have an open-format script club using our private pastebin, IRC channel, and LiveMeeting voice chat. This time could vary from real-world problem solving to scripting games and “project Euler”-style challenges.

Some weeks we would have open-mic time and solicit feedback for the PowerShell team from anyone who cared to give it. I would make an effort to make sure I wasn’t the only MVP there, so you could feel that even if there wasn’t a Microsoft employee present on a given day, your voice could be heard when you make suggestions or vent frustrations …

The point is: the format would vary a bit, and we would adjust it over time to fit whatever works the best within our virtual meeting and time constraints.

So, what do you think? Are you interested?

I’ve written this up on the new website for our Rochester PowerShell group and on the old blog, and on UGSS Facebook and Twitter … and now that there’s only two days left, it’s time to mention it again, on my blog

This Wednesday, January 20, 2010 – 6:00pm

It’s a new year, and a new opportunity to get started with PowerShell. With the new year, we’re going to put a renewed focus on trying to make our meetings valuable to everyone. With that in mind, the first Rochester meeting of the upstate New York PowerShell users group in 2010 will feature two separate presentations:

Getting Started: The PowerShell Pipeline (100 Level)

• Different types of commands in PowerShell
• Variable assignment and Pipeline arguments
• Understanding the Pipe
• Why PowerShell’s pipeline is different

• Exploiting pipelines for fun and profit

Creating PowerShell script Modules (300 Level)

• Turning your script or function into a module
• What makes a function an “advanced function”
• Providing in-line help for functions
• Controlling what functions and variables your module exports

• Why you should use Module Metadata files

Everyone is welcome, from beginners to pros! Bring a friend and introduce them to PowerShell. As always, our meeting will be at New Horizons’ in Henrietta, 50 Methodist Hill Drive, Suite 50.

In order to keep things on time, we’ll be starting our presentations at 6:30 exactly, after a short pizza, wings, and networking time starting at 6pm. We only ask that if you can come early for the food you let us know ahead of time and come ready to pitch in a few dollars to help cover the cost.

PS: There will be swag. We have several books, an Arc Mouse, and copies of Windows 7 and Office 2007 to give away…

I was under the impression that module manifests allow only DATA stuff. In fact, if you try to use cmdlets or variables in them, you get the usual Data-language errors, like this:

Import-Module : The module manifest ‘C:\Modules\Test\Test.psd1’ could not be processed because it is not a valid PowerShell restricted language file. Please remove the elements that are not permitted by the restricted language: The command ‘Get-ChildItem’ is not in allowed in restricted language mode or a Data section.

Import-Module : The module manifest ‘C:\Modules\Test\Test.psd1’ could not be processed because it is not a valid PowerShell restricted language file. Please remove the elements that are not permitted by the restricted language: A variable that cannot be referenced in restricted language mode or a Data section is being referenced. Variables that can be referenced include the following: $PSCulture, $PSUICulture, $true, $false, and $null.

BUT THEN I came across this in the Windows 7 built-in BitsTransfer module:

RequiredAssemblies=Join-Path $psScriptRoot “Microsoft.BackgroundIntelligentTransfer.Management.Interop.dll”

AND IT WORKS?!

Well, that’s very weird, because Join-Path is certainly not allowed in a normal “restricted language” file, and (as you can tell from above) neither is $PsScriptRoot — in fact, as far as I know, you shouldn’t have to do that at all, since RequiredAssemblies knows enough to look in your module folder… but nevermind that, why does it work?

The Rest of the Story

So I went digging, and it turns out that although they are parsed as Restricted Language (like a “data section,” see about_Data_Sections), module manifests are allowed extra cmdlets: “Import-LocalizedData”, “ConvertFrom-StringData”, “Write-Host”, “Out-Host”, “Join-Path” and even special variables that aren’t normally allowed in data sections: specifically $PsScriptRoot which is the ModuleBase (the parent folder of the psd1) and environment variables ($Env:), plus the usual $PSCulture, $PSUICulture, and of course $true, $false, and $null.

However, although you can use those variables, you can’t embed them in strings, so if you wanted to use $Env:Windir or $Env:Temp as part of a path (for instance), you need to take advantage of the availability of Join-Path.

Now, I can’t find this documented anywhere (although I did add it to the module manifest documentation on MSDN), but it’s true, nonetheless — you’ll just have to trust me Yeah, PowerShell is starting to drive me crazy again.

Related articles by Zemanta

• A question about PowerShell Module Manifests (huddledmasses.org)
• Another Module Manifest Gotcha (huddledmasses.org)

Shay Levy, ScriptFanatic, wrote about the PowerShellHostVersion on his blog this morning, explaining how it is not a field you should use in your module manifests.

Of course, there’s an exception: if your module is dependent on a specific host. For instance, if you’ve written a module for PoshConsole which exploits the WpfHost display features or the BgHost hotkeys feature … or if you’ve written ISEPack based on the menu and script-editor features of PowerShell ISE), then it makes sense to specify the PowerShellHostName and the PowerShellHostVersion together. However, if you aren’t taking a dependency on a specific host … you should definitely never specify PowerShellHostVersion by itself.

RequiredModules

I’ve got another one that I think you should not use. PowerShell has several ways of specifying prerequisites for your modules, such as assemblies that should be loaded, modules that must be nested, etc. In every case except RequiredModules, PowerShell will actually import those things for you.

That is, if you specify that you RequiredAssemblies = “System.Windows.Forms” it will automatically load it. If you specify NestedModules = “BitsTransfer” it will automatically load that. But if you specify RequiredModules = “BitsTransfer” you’re sore out of luck.

Not only does PowerShell not load them for you, it doesn’t give you all the information you need to load them when it fails. For example, consider that you have a module called “ReallyRequired” and one called “TestModule” which requires ReallyRequired. It’s metadata file is very simple, it looks like this:

Before we try to import it, we might try to check which modules it requires, and since it doesn’t appear to require any, we’ll go ahead and import it:

Apparently, there’s just no way to tell which module(s) you need to preload (or even whether there are any) until you get an error message. You might think you could just write a script to test for that error and then run Import-Module ReallyRequired … but what you don’t know is that after you load your copy of the ReallyRequired module:

If you’re like me, right now you’re tearing out your hair wondering why the error message didn’t tell you that in the first place. Then you have to go off and try to find a newer version of your RequiredModules. And that’s not even the crazy thing! The crazy thing is, if you find the wrong one, you could still get a message about the GUID not matching next.

So yeah, RequiredModules is really unlikely to be helpful right now. You’re better off just putting an Import-Module ReallyRequired -version 2.5 -ErrorAction Stop at the top of you script module (although that won’t help you if you’re writing a binary cmdlet module).

Some good news

This is just one of the things that the new PoshCode is being designed to address, so I’ll be blogging more about this shortly, but for now, you can use this function to list RequiredModules:

Related articles by Zemanta

• A question about PowerShell Module Manifests (huddledmasses.org)

This is old news and new news … but I figured it’s about time I break it on my blog, since it was mostly my doing (and since my Northeast Developer Evangelist Jim O’Neil already mentioned it on his blog).

We have created an official Virtual PowerShell Group (official, in the sense that it’s recognized by Microsoft’s User Group Support Services out of the unofficial #PowerShell channel on irc.freenode.net and have created a website to support it including a page where you can join us online via web-based chat any time you like.

Of course, this isn’t exactly headline news: The #PowerShell IRC channel has been around for years and has always been helpful and polite. What’s new is that we are officially a User Group, and have swag to give away (more on that later)! We’re hoping that this will help some of you justify joining us, and using IRC as a resource for your PowerShell learning and for those sticky questions!

I just posted an article to the FAQ on the PoshCode Wiki answering this question that came up again on our IRC PowerShell user group today. It came up in the context of determining whether a function was the last function on the pipeline, because one of our users was looking for a way (other than creating ps1xml files) to output objects onto the pipeline for use in other functions, but still format those objects nicely if they were output directly.

Before I give the solution, I just want to say: don’t change your output based on where you are in a pipeline.

There are numerous scenarios where your function will be the last one on a pipeline, but still be participating in further pipelines, including formatting and output modification. For example, take our function Test-Pipeline (defined below) in these three scenarios below. In none of these scenarios would it be appropriate for the function to write formatted output instead of outputting the raw object, but in each case, the function is the last function in the pipeline.

However, if you want to determine your function’s position in the pipeline for some other reason, the answer is simple. You need to use $MyInvocation and compare the PipelineLength and PipelinePosition properties:

So, the Windows PowerShell 2.0 Software Development Kit is basically a collection of samples, and it has been released separately from the Windows Platform SDK for a change, making the download a tiny 2.35MB …

There are lots of examples in there in C# (no other languages), and honestly, some of them ought to make it into PSCX or some other project where people could grab pre-compiled versions

• A Template for creating PSProviders, and a sample PSProvider (for Access databases).
• A sample of participating in Transactions (a set of “transacted comment” cmdlets for creating comments that go along with a transaction).
• Example cmdlets: Select-Object, Select-String, Get-Process, Stop-Process
• Dealing with issues with Serialization in PowerShell
• PowerShell Eventing:
  • Deriving from ObjectEventRegistrationBase to create Register-FileSystemEvent
• Receiving notifications from PowerShell Events on remote computers.
• Hosting APIs, including (among others):
• Restricted runspaces
• Runspace pools
• Remote runspaces (and remote runspace pools)
• Running commands in parallel or sequentially
• Calling Cmdlets and passing parameters

• Reproducing the default PowerShell.exe output

Honestly, if you’re a developer that’s been wondering about learning to code cmdlets, or host PowerShell as a scripting engine, now’s the time.

I know that I just wrote a post last week about XPath and namespaces in PowerShell, but at the time I left out one possible way of dealing with namespaces, because it’s not the right way of doing things. However, sometimes it’s nice to have options, and when you’re working on the command-line in PowerShell, or just trying to figure out a proof-of-concept call to a web service, you really don’t need to deal with namespaces correctly, you just need it to work.

With that in mind, I present to you the fourth option: just strip the namespaces out! The simplest way to do that is to run the XML through an XSL stylesheet which just outputs the local-name() of each node (including attributes), and remove any namespace definitions (processing instructions).

That stylesheet and the basic steps of the process will work anywhere, from Java to C# to the web … but since my current language of choice for prototyping is PowerShell, I’ll show you how to implement it there as Remove-XmlNamespace. Once you have that, I think you’ll see that it was relatively simple for me to write a new Select-XML which adds a parameter RemoveNamespace which is implemented by calling this Remove-XmlNamespace …

That actually allows you to call Select-Xml with the -RemoveNamespace parameter just as though the namespaces didn’t exist. Of course, the returned XML nodes will, in fact, NOT have namespaces … so they may not be quite the same as the source, but the data will all be there.

The style I use has a nice black background...

A while back Thell Fowler (with a little help, and a lot of testing from me) wrote a very good PowerShell Lexer for Notepad++ 5.2 and later… it’s very thorough, has good code-folding, and full support for PowerShell 2.0 syntax highlighting.

I mention this because Notepad++ 5.6 just released yesterday, and it has built-in support for PowerShell syntax courtesy of Scintilla ... but it’s very, very bad. The scintilla PowerShell lexer is probably the most minimal PowerShell lexer I’ve seen (it’s worse than the old “user style” I had created for Notepad++) and has no support for:

• The ` escape character
• Here-strings (which can contain quotes, etc)
• The difference between strings and literal strings and literal here-strings
• The begin/process/end block keywords and Param()
• PowerShell operators (like -is or -gt or -notcontains)
• [System.Namespace.Class]::Method() syntaxes
• Nested $variables inside strings
• Nested $( code blocks ) inside strings (with strings inside those, and …)
• Any of the new PowerShell 2 syntax like:
  • multi-line comments
  • [Parameter()] and [Alias()] and [Validate …. ]

• [CmdletBinding()]

There’s probably more, but I couldn’t be bothered to spend more than a couple of minutes with it. As you can probably guess … all of those features are supported by the external PowerShell Lexer plugin that Thell wrote, so if you’re a PowerShell and Notepad++ user, I apologize for not drawing your attention to our PowerShell Lexer for Notepad++ before .

Incidentally, I stuck a screenshot in this post so you can see how I use it, but there’s one a more complete example of the PowerShell Syntax Highlighting on that lexer download page.