Huddled Masses

I’ve written this up on the new website for our Rochester PowerShell group and on the old blog, and on UGSS Facebook and Twitter … and now that there’s only two days left, it’s time to mention it again, on my blog

This Wednesday, January 20, 2010 – 6:00pm

It’s a new year, and a new opportunity to get started with PowerShell. With the new year, we’re going to put a renewed focus on trying to make our meetings valuable to everyone. With that in mind, the first Rochester meeting of the upstate New York PowerShell users group in 2010 will feature two separate presentations:

Getting Started: The PowerShell Pipeline (100 Level)

• Different types of commands in PowerShell
• Variable assignment and Pipeline arguments
• Understanding the Pipe
• Why PowerShell’s pipeline is different

• Exploiting pipelines for fun and profit

Creating PowerShell script Modules (300 Level)

• Turning your script or function into a module
• What makes a function an “advanced function”
• Providing in-line help for functions
• Controlling what functions and variables your module exports

• Why you should use Module Metadata files

Everyone is welcome, from beginners to pros! Bring a friend and introduce them to PowerShell. As always, our meeting will be at New Horizons’ in Henrietta, 50 Methodist Hill Drive, Suite 50.

In order to keep things on time, we’ll be starting our presentations at 6:30 exactly, after a short pizza, wings, and networking time starting at 6pm. We only ask that if you can come early for the food you let us know ahead of time and come ready to pitch in a few dollars to help cover the cost.

PS: There will be swag. We have several books, an Arc Mouse, and copies of Windows 7 and Office 2007 to give away…

I was under the impression that module manifests allow only DATA stuff. In fact, if you try to use cmdlets or variables in them, you get the usual Data-language errors, like this:

Import-Module : The module manifest ‘C:\Modules\Test\Test.psd1’ could not be processed because it is not a valid PowerShell restricted language file. Please remove the elements that are not permitted by the restricted language: The command ‘Get-ChildItem’ is not in allowed in restricted language mode or a Data section.

Import-Module : The module manifest ‘C:\Modules\Test\Test.psd1’ could not be processed because it is not a valid PowerShell restricted language file. Please remove the elements that are not permitted by the restricted language: A variable that cannot be referenced in restricted language mode or a Data section is being referenced. Variables that can be referenced include the following: $PSCulture, $PSUICulture, $true, $false, and $null.

BUT THEN I came across this in the Windows 7 built-in BitsTransfer module:

RequiredAssemblies=Join-Path $psScriptRoot “Microsoft.BackgroundIntelligentTransfer.Management.Interop.dll”

AND IT WORKS?!

Well, that’s very weird, because Join-Path is certainly not allowed in a normal “restricted language” file, and (as you can tell from above) neither is $PsScriptRoot — in fact, as far as I know, you shouldn’t have to do that at all, since RequiredAssemblies knows enough to look in your module folder… but nevermind that, why does it work?

The Rest of the Story

So I went digging, and it turns out that although they are parsed as Restricted Language (like a “data section,” see about_Data_Sections), module manifests are allowed extra cmdlets: “Import-LocalizedData”, “ConvertFrom-StringData”, “Write-Host”, “Out-Host”, “Join-Path” and even special variables that aren’t normally allowed in data sections: specifically $PsScriptRoot which is the ModuleBase (the parent folder of the psd1) and environment variables ($Env:), plus the usual $PSCulture, $PSUICulture, and of course $true, $false, and $null.

However, although you can use those variables, you can’t embed them in strings, so if you wanted to use $Env:Windir or $Env:Temp as part of a path (for instance), you need to take advantage of the availability of Join-Path.

Now, I can’t find this documented anywhere (although I did add it to the module manifest documentation on MSDN), but it’s true, nonetheless — you’ll just have to trust me Yeah, PowerShell is starting to drive me crazy again.

Related articles by Zemanta

• A question about PowerShell Module Manifests (huddledmasses.org)
• Another Module Manifest Gotcha (huddledmasses.org)

Shay Levy, ScriptFanatic, wrote about the PowerShellHostVersion on his blog this morning, explaining how it is not a field you should use in your module manifests.

Of course, there’s an exception: if your module is dependent on a specific host. For instance, if you’ve written a module for PoshConsole which exploits the WpfHost display features or the BgHost hotkeys feature … or if you’ve written ISEPack based on the menu and script-editor features of PowerShell ISE), then it makes sense to specify the PowerShellHostName and the PowerShellHostVersion together. However, if you aren’t taking a dependency on a specific host … you should definitely never specify PowerShellHostVersion by itself.

RequiredModules

I’ve got another one that I think you should not use. PowerShell has several ways of specifying prerequisites for your modules, such as assemblies that should be loaded, modules that must be nested, etc. In every case except RequiredModules, PowerShell will actually import those things for you.

That is, if you specify that you RequiredAssemblies = “System.Windows.Forms” it will automatically load it. If you specify NestedModules = “BitsTransfer” it will automatically load that. But if you specify RequiredModules = “BitsTransfer” you’re sore out of luck.

Not only does PowerShell not load them for you, it doesn’t give you all the information you need to load them when it fails. For example, consider that you have a module called “ReallyRequired” and one called “TestModule” which requires ReallyRequired. It’s metadata file is very simple, it looks like this:

Before we try to import it, we might try to check which modules it requires, and since it doesn’t appear to require any, we’ll go ahead and import it:

Apparently, there’s just no way to tell which module(s) you need to preload (or even whether there are any) until you get an error message. You might think you could just write a script to test for that error and then run Import-Module ReallyRequired … but what you don’t know is that after you load your copy of the ReallyRequired module:

If you’re like me, right now you’re tearing out your hair wondering why the error message didn’t tell you that in the first place. Then you have to go off and try to find a newer version of your RequiredModules. And that’s not even the crazy thing! The crazy thing is, if you find the wrong one, you could still get a message about the GUID not matching next.

So yeah, RequiredModules is really unlikely to be helpful right now. You’re better off just putting an Import-Module ReallyRequired -version 2.5 -ErrorAction Stop at the top of you script module (although that won’t help you if you’re writing a binary cmdlet module).

Some good news

This is just one of the things that the new PoshCode is being designed to address, so I’ll be blogging more about this shortly, but for now, you can use this function to list RequiredModules:

Related articles by Zemanta

• A question about PowerShell Module Manifests (huddledmasses.org)

So, I’m building the next PoshCode around modules, with CPAN as my model …

Scripts are still there, but they’re still basically done the way they are now: you just upload a single file.

Modules, however, will be treated specially. You’ll have to have an account to upload, but you’ll be able to just upload a .zip file of your module folder using your login and not fill in any other forms. Our system will take care of parsing the metadata out of the manifest in your module folder.

The problem is that I need a little data that isn’t a part of the standard module manifest format… and I can’t add it because PowerShell won’t load a module with extra fields in the manifest metadata hashtable: https://connect.microsoft.com/PowerShell/feedback/ViewFeedback.aspx?FeedbackID=421837 — My request was closed “by design” early in the beta cycle and I wasn’t able to convince them to rethink that.

At a minimum, PoshCode requires a LICENSE field, and a CATEGORY field (think of the categories on CPAN or TechNet Script Center).

So I’m having an informal poll. (you’ll have to “vote” by commenting or tweeting to me, because I want more than just “I choose A”).

Which option do you prefer, or can you think of a better way:

Require the License and Category to be part of the PRIVATEDATA field in the standard manifest

This might require authors to rewrite parts of their modules, because we’d be forcing PrivateData to be a Hashtable, and to contain keys that they don’t need in the module.

Some existing modules use an array in PrivateData, or even a simple string, rather than a Hashtable. However, it’s not a huge difference, and might be the cleanest method: it wouldn’t require you to manage a second file of data, and the additional data will be easily available to users and scripts via the standard Get-Module output.

Add a “doc comment” system for Module manifest files.

The upside of this is that PoshCode wants to create module-level documentation anyway. If we use documentation comments like those used on functions we would be able to just create our own standard for them, and add any extra fields we need. It would be guaranteed not to conflict with anything you’re already doing, and in addition to missing metadata, you could have some module-level documentation beyond just the Description field of the metadata.

The problem with this is that there’s no built-in way to parse that, and doing so isn’t trivial, so you would need to just read it on our website, or read the source of the file, or have our PoshCode cmdlets in order to make any sense of it once you had a module on your system. It doesn’t integrate with PowerShell in any meaningful way.

Embed the extra data right in the manifest hashtable, using a special comment to hide it from Get-Module

@{
author=“Joel Bennett”
description=“My latest crazy module”
<#!PoshCode
License=“Ms-PL”
Category=“WPF”,“GUI”,“Toolkit” #>
CompanyName=“Huddled Masses”
...
}


I like this because it’s fairly trivial for us to strip out the comment and just turn that into a plain-old DATA section. It also lends itself to reminding the PowerShell team that these fields are missing, and maintains the existing simple syntax of the manifest.

The problem here is, again, that the data doesn’t appear using any of the standard PowerShell tools — but getting it out is significantly easier than getting out document comments…

Require a separate “ReadMe.psd1” (or “PoshCode.pds1” or “Metadata.psd1”) metadata file

You could start with a copy of your module manifest, and then add the extra stuff that PoshCode needs. This would be nice because we would be able to add any extra fields we wanted as mandatory members, and we could include the “documentation” stuff I mentioned earlier…

But the downside is that it would be a third file (second manifest) that authors would have to maintain and keep current.

Any thoughts? Suggestions?

I’ve put some further thoughts about CPAN and the data problem on the wiki.

I’ve just created PowerShellGroup.org — a new site for PowerShell user groups (starting with: Rochester, Syracuse, Atlanta, Ottawa and of course, the Virtual Group).

It’s open to any PowerShell user groups or script clubs. If you’re running a PowerShell user group and want simple web-hosting with:

• top-of-the-site billing for your group (with a url like http://powershellgroup.org/rochester-ny )
• group signups
• events, calendars and event signups (including sending out RSVP emails to your group), and email reminders of events
• a semi-private group section for posting notices, wiki pages, etc.

• optional twitter notification for your posts

Just let me know who you are after you sign up on the site, and I can set you up with “group administrator” rights.

Also, if you have your own web hosting for your user group, but would like to be listed on our site, we can handle that too

This is old news and new news … but I figured it’s about time I break it on my blog, since it was mostly my doing (and since my Northeast Developer Evangelist Jim O’Neil already mentioned it on his blog).

We have created an official Virtual PowerShell Group (official, in the sense that it’s recognized by Microsoft’s User Group Support Services out of the unofficial #PowerShell channel on irc.freenode.net and have created a website to support it including a page where you can join us online via web-based chat any time you like.

Of course, this isn’t exactly headline news: The #PowerShell IRC channel has been around for years and has always been helpful and polite. What’s new is that we are officially a User Group, and have swag to give away (more on that later)! We’re hoping that this will help some of you justify joining us, and using IRC as a resource for your PowerShell learning and for those sticky questions!

Well, ok, we already know that the answer is 42 … which, in binary, is 101010 …

Clearly we need to make sure that everyone celebrates The Hitchhikers’ Guide to the Galaxy on October 10 this year, but I’m not sure what to suggest as an appropriate celebration. Should I just recommend you read through the series again? Should we have group readings on that day? Try blogging in binary? The one thing I do know, is that I won’t even suggest viewing the movie(s). Anyone have some good ideas? They must be Mostly Harmless of course, but I’m sure some of you will have some good ideas . We’re definitely having a party at my house!

Related articles by Zemanta

• The Hitchhiker’s Guide to the Galaxy, 30 years on: why we should still be reading it (guardian.co.uk)
• Don’t Panic! (news.bbc.co.uk)

I just posted an article to the FAQ on the PoshCode Wiki answering this question that came up again on our IRC PowerShell user group today. It came up in the context of determining whether a function was the last function on the pipeline, because one of our users was looking for a way (other than creating ps1xml files) to output objects onto the pipeline for use in other functions, but still format those objects nicely if they were output directly.

Before I give the solution, I just want to say: don’t change your output based on where you are in a pipeline.

There are numerous scenarios where your function will be the last one on a pipeline, but still be participating in further pipelines, including formatting and output modification. For example, take our function Test-Pipeline (defined below) in these three scenarios below. In none of these scenarios would it be appropriate for the function to write formatted output instead of outputting the raw object, but in each case, the function is the last function in the pipeline.

However, if you want to determine your function’s position in the pipeline for some other reason, the answer is simple. You need to use $MyInvocation and compare the PipelineLength and PipelinePosition properties:

So, the Windows PowerShell 2.0 Software Development Kit is basically a collection of samples, and it has been released separately from the Windows Platform SDK for a change, making the download a tiny 2.35MB …

There are lots of examples in there in C# (no other languages), and honestly, some of them ought to make it into PSCX or some other project where people could grab pre-compiled versions

• A Template for creating PSProviders, and a sample PSProvider (for Access databases).
• A sample of participating in Transactions (a set of “transacted comment” cmdlets for creating comments that go along with a transaction).
• Example cmdlets: Select-Object, Select-String, Get-Process, Stop-Process
• Dealing with issues with Serialization in PowerShell
• PowerShell Eventing:
  • Deriving from ObjectEventRegistrationBase to create Register-FileSystemEvent
• Receiving notifications from PowerShell Events on remote computers.
• Hosting APIs, including (among others):
• Restricted runspaces
• Runspace pools
• Remote runspaces (and remote runspace pools)
• Running commands in parallel or sequentially
• Calling Cmdlets and passing parameters

• Reproducing the default PowerShell.exe output

Honestly, if you’re a developer that’s been wondering about learning to code cmdlets, or host PowerShell as a scripting engine, now’s the time.

Consider this a reminder that December is ScriptClub month at our local PowerShell User Group.

What is Script Club?

Script clubs are like a hands on lab with no set topic or teacher.

You bring an idea for a script, and ask your fellow PowerShell users for help getting the script written. Pretty much anything goes — just bear in mind that it may be hard to test things like Exchange scripts or Active Directory management scripts unless you have a nice virtual lab on your laptop

Please join us for PowerShell Script Club this Wednesday, and bring your friends! Script Club is a great way to learn PowerShell. Beginners through advanced users are equally welcome.

What are the Rules of Script Club?

1. You always talk about script club
2. You always talk about script club
3. If someone asks for help, and you can help, you help
4. Two people help one person at one time
5. One module per person per night
6. All scripts, all PowerShell
7. Scripts will be as short as they can be

1. If this is your first time at Script Club, You have to script!

Please RSVP so we can get a head count (especially if you want Pizza)!

As usual, our meeting will be on the Third Wednesday of the month, with Pizza at 6pm, followed by Script Club at 6:30. And of course, we’ll be meeting at the New Horizon’s training center at 50 Methodist Hill Drive, Suite 50, Rochester, NY. Click here for directions and a map.