Well, after some extensive hacking this long weekend, I’m happy to announce the opening of the refurbished (and renamed) PowerShell Code repository at it’s new domain: PoshCode.org with a few new features and a slightly overhauled look.

I expect that soon we’ll be replacing the back end with something which has a few enhancements over this version (at least in terms of tracking the many versions of a particular code contribution), but even without that, we hope that the current feature set will be enough to push the repository into mainstream usage as the primary place to share PowerShell code.

The new PoshCode site is still based on the old pastebin GPL source code, but I’ve hacked it up so much you’ll hardly recognize it. Sadly, I haven’t kept it as generically clean as I could have, so although I’m making the source code available, you’ll have to clean up the “layout” files if you want to run your own repository.

New Features

The most obvious new feature is that the front page now prominently features the most recent contributions. This list is filtered to only show the last in a series of contributions (so if you make a new contribution based on a previous one, the front page won’t be swamped with iterations of a single script the way the old one was).

The best new feature is that you can now browse by author (currently, you simply get a list of their 25 most recent contributions, in the future we’ll add some paging functionality as the need arises).

Finally, I’ve changed the description field (which previously allowed any html) to use only a restricted subset of Textile. I’ll put up some documentation about this later, but for now: you can make paragraphs by leaving a blank line, and can use asterisks for bold and underscores for italics … and starting each line with an asterisk or a pound sign will get you bulleted or numbered lists. Any links will be rel=“nofollow” and you can’t use raw HTML at all.

Earlier Enhancements

In addition to these new features, there are a few other enhancements that I added on to the original code: a couple of extra fields, permanent storage, search functionality, and a recent contributions RSS (Really Simple Syndication) feed.

Incidentally, the search functionality allows you to search the author, description, title, and main content fields using MySQL’s full-text indexing search, and is accessible via the new PoshCode script functions (available as both a PowerShell v2 CTP-compatible PoshCode module and a v1 PoshCode script).

There’s been a lot of discussion on the #PowerShell IRC channel about code signing lately — mostly in response to my earlier posts (which I won’t link to here, because I should probably retract some of it after the discussions we’ve had). The discussion mostly revolves around how code signing applies to the PowerShell script repository and users’ ability to trust scripts that they download.

In order to keep the conversation going, and hopefully, include some of the Microsoft MVPs and PowerShell developers that don’t participate in the IRC channel at all, I’m going to post a summary of the conclusions we’ve drawn so far, and then outline a proposal for the repository.

In case you don’t know, PowerShell implements code-signing for scripts using x509 Certificates (basically, SSL (Secure Sockets Layer) certs with “Extended” use properties), but the signatures are just embedded in the script as a comment, much like PGP (Pretty Good Privacy) signatures in email. Depending on your settings, the engine will actually check the signature on a script before executing it, and refuse to execute it if the signature isn’t valid, for instance, if the script has been modified, or the certificate used to sign it can’t be traced to a trusted Certificate Authority.

PowerShell code-signing is inconvenient, and expensive. Scripts are only Valid if they are signed, unmodified, and the certificate traces back to a CA(Certificate Authority) that is in your Trusted Roots store (by default, basically, Microsoft’s, VeriSign, Thawte, and Comodo).

You can create your own “test” certificate (that is, one that you sign yourself), but scripts signed by that certificate will only work on that computer. This system works fine for large companies which typically have either a subscription for purchasing certificates, or their own in-house trusted CA, but not for small private developers or open source script sharing communities without financial motivation.

Because of this, to take advantage of PowerShell’s code-signing on your computer (that is, to set your ExecutionPolicy to “AllSigned” or “RemoteSigned”) you must do one of these things:

1. Make sure that all script authors rent and use code-signing certs which cost more than $150 a year.
2. Add a non-commercial root CA certificate (like CACert) to (all of) your computer(s), and offer your authors the option of going through the process of getting one of those instead.
3. Create a “community” CA … and try to figure out how to make sure the certificate is secure enough to convince people to use it (yeah, ok, this was partly my idea, and it’s really a bad idea).
4. Generate your own code-signing certificate (which can be self-signed) and just sign every script that you trust.

So far, it seems like that last option is the only one that’s going to fly, because although some developers are willing to buy certificates to sign software they’re going to sell … most scripters and sys-admins aren’t willing to buy one to share their certificates for free.

So. Here’s the base fact:

The PowerShell Code repository is a repository of user-provided code.

We can’t actually afford to audit all of the code that users submit, nor take responsibility for verifying the identity of every contributor, but we’d like to have a way to do so. The best way would be to give everyone a free CodeSigning certificate from Microsoft or one of the established CAs, but we can’t afford to do that, so here’s our proposal (although it may sound like it, none of this is written yet, I’m just trying to get some feedback on the ideas):

Rather than rely strictly on user login authentication, we will require that all (non anonymous) submissions be SIGNED, and the signing certificate (key) will be the user authentication mechanism. Essentially, any unsigned code will show up as written by “anonymous” ... and any signed code will show up with the identity from the signing certificate — the certificate must include an email address, and the first post by each key will be held back until the email address can be authenticated.

In addition to the author’s signature on a script, other users who have used the code (or read through it without finding any issues), can also sign it as a show of trust — they simply submit the exact same script as a modification, but with their signature in place of the author’s.

So users will sign scripts to say they trust them, and when you browse or search the repository, you’ll see a lists of the users who have signed each code-snippet. You can download a copy of the code with any of the signatures and verify the signature yourself…

Additionally, there will be a Get-PoshCode cmdlet which probably should:

• Mark the downloaded scripts as ‘remote’.
• Add the download permalink url in a comment upon download (to aid in discovering other signatures and in adding your signature).
• Handle a cache of “trusted” key/certs and choose a trusted author (if there is one) from multiple available downloads, and ask you if you want to add any of the signers to your trusted list — particularly if there isn’t already a trusted signature available for the script.
• Validate the signature(s) after download, and provide a report (eg: valid/invalid).
• Offer to replace the signature (with yours) if it’s not a trusted x509 authenticode signature.

What certificate system should we use?

You may have assumed that we’re planning on using Microsoft’s code signing which is built into PowerShell, but when we were discussing this in the IRC channel, one of the things that kept coming up is that one way or another, we’re essentially creating an alternative way for people to “trust” the certificate — since we’re assuming that most of these certificates will be self-signed.

That is, if we use x509, we’ll STILL be creating an alternate way for people to say that they trust the x509 certs, and thus, using them in a way they weren’t intended. If we use PGP, we’ll basically be using them as intended, and will have peer key-signing as a trust mechanism, but we’re STILL introducing a secondary way for users to establish trust for PowerShell scripts which doesn’t integrate with the engine’s policy settings, thus making things more complicated.

Let me sum up the positive and negative sides of what I see as our two options.

x509 (aka “Authenticode”) signatures — supported by Microsoft.

• Probably self-signed and therefore won’t “work” properly in PowerShell anyway.
• Some of them would probably be commercial certificates issued by trusted CAs like Thawte or Comodo, giving you instant trust in the identity of the author. In that case, they would additionally be:
• Verifiable by PowerShell natively, and therefore, they:
• Don’t require a separate signature in order to execute when the ExecutePolicy is set to AllSigned.

PGP signatures (supported by the Open Source community and used in Debian, etc)

• Just as (un)trustworthy as any other self-signed certificate ... except that:
• PGP certificates (keys) are inherently user-signable. The community can vouch for each other’s identities, and create a web-of-trust for authentication purposes (which is all x509 really accomplishes anyway — you trust that Verisign makes sure that they are who they say they are, and live where they say they live).
• Not inherently usable by PowerShell.
• AllSigned mode requires re-signing (probably with a self-signed cert). This also means that if you have an x509 authenticode-signed script to submit, we would still need you to re-sign it with a PGP certificate when you wanted to upload it.

Final thoughts

Of course, there’s a lot of outstanding issues with this idea. But I wanted to put it out there for some broader feedback, particularly because we really are thinking about using PGP certificates since they can be counter-signed by other users, and since the general consensus opinion is that actually expecting scripters to buy code-signing certificates is just out of the question.

So the first question is, can we do PGP signing in pure .Net for a cmdlet (maybe using bouncy castle), and can we easily create a Set-PGPSignature cmdlet?

Has there been any progress on a way to do PGP signing that’s compatible with XML files (eg: ps1xml files)?

Can we do signature checking in-memory, so we can download the script with many signatures embedded and stitch them together for verification without creating multiple temporary files?

Regardless, if you trust a signature, but the signature isn’t Authenticode (PGP) or isn’t Valid (self-issued), the PowerShell security measures will require you (the end user) to sign the script in order to allow it to execute — essentially, every signed script on your system will be signed by you (or your sysadmin) and so, signing essentially becomes the administrator’s equivalent of blessing a script, and is somewhat like setting the eXecute flag on a linux script — except that you have to re-set it every time you edit the script.

We need to make signing slightly easier.

It’s a pain to find your certificate, and the built-in AuthenticodeSignature cmdlets are horrible at parsing parameters.

We need to make generating these self-signing certs trivial for non-SDK machines

It’s really pretty easy with MakeCert, but that’s not redistributable, and the Windows SDK is a ridiculously large download for someone who just wants to sign their scripts. I’ve written a script to generate code-signing certs using OpenSSL, which is redistributable, but it’s still not great.

This continues a short series of posts about getting started with PowerShell … with a few tips about things you can do to keep your PowerShell profile safe and organized. Your “profile” is the script that is automatically loaded when you start up PowerShell. Really, I should say that your profile is the set of scripts which are loaded by default when you start up PowerShell. “By default” because you can always skip loading them by passing the -NoProfile switch to PowerShell.exe, and a set because PowerShell does, in fact, attempt to load at least four scripts when you run it:

PowerShell loads “machine” profile scripts (which are located in the PowerShell folder) and “user” profile scripts (located in your Documents\WindowsPowerShell folder). But there’s a little more to it than that: PowerShell is a scripting engine which can be hosted inside any app, PowerShell.exe is a DOS-style console which is the default host. There are several third-party hosts available such as PowerShell Plus and PowerGUI and several open source hosts such as BgShell and PoshConsole … in order to support this ecosystem of hosts, the default PowerShell behavior is to load a host-specific profile script (for both the machine settings and the local-user settings). Not all hosts will do that, but anyway … the default host loads Microsoft.PowerShell_profile.ps1 and Profile.ps1 from both the user and machine locations.

By default, none of those profiles actually exist. Once you’ve installed everything as in Part 1, you should have a Profile.ps1 file provided by PSCX. This profile defines a whole bunch of values that are used by various PSCX cmdlets and scripts, so you may want to change some of it, but you should be careful about just deleting things until you’re well acquainted with the PSCX cmdlets. Over time, I’ve added settings to my profile for other snapins as well, and there gets to be a lot of noise in there that’s specific to different snapins, so instead of just leaving all of that in my main profile, I rename the Profile.ps1 file provided by PSCX and then dot-source it from a new blank profile script.

In fact, and I found that I started collecting a lot of scripts in my WindowsPowerShell folder so I created a sub-folder for them, and I automatically load everything that’s in that folder, so I don’t have to manually dot-source things when I add a new snapin profile.

Authenticode Signing your auto-load scripts

In order to make sure that automatically loading scripts doesn’t become a way for people to attack my computer, I made a decision awhile ago that I would only auto-load signed scripts. The how and why of this is a bit much to get into, and I wrote about Generating Windows Authenticode Code-Signing Certificates with OpenSSL a while back, so you can read that if you want more details, I want to review the simplest steps.

Edit: I should take a moment here to point out, in case you’re really new to PowerShell, that the first thing you’ll have to do is fix the error “File … cannot be loaded because the execution of scripts is disabled on this system.” I originally was just assuming you would have figured that out — it tells you to read the about_signing help, so you can just execute get-help about_signing and it explains in there what you need to do. In case you haven’t gotten that far, before you can execute the New-CodeSigningCert you’re going to need to run PowerShell as administrator (“elevated”) and execute this command: Set-ExecutionPolicy RemoteSigned … that will let you execute most scripts that you have saved locally without having signed them. You may want to tighten that up later and set it to “AllSigned,” but first you have to be able to sign scripts.

Generating a code-signing certificate

Edit: I’ve simplified this a bit, and modified the script to import the certificate to the Current User certificate store if you aren’t running in an elevated console (in Vista) — which should work fine if you’re the only user on your machine that needs to run the scripts — if you need them in the Local Machine store, you need to be running as administrator. The script will let you know which it does, so you’ll know, one way or another.

Step 1. Download my PoshCerts package and unpack it in your WindowsPowerShell folder — lets say in, WindowsPowerShell\PoshCerts\bin.

Step 2. Run the New-CodeSigningCert script to generate into the Certs folder. So if you’re in the WindowsPowerShell directory, you can just run it and you’ll be prompted for a few pieces of information that the certificates require, plus two passwords — and all of the keys will be generated into the Certs folder as files, and imported to your local store. Make sure you make a note of those passwords.

Edit: I came up with a Step 3. Under most circumstances (that is, unless you were planning on running a corporate Certificate Authority), you’ll have no further use for the CA certificate, If you want to make sure that nobody can abuse your root CA … you can and should just delete the private certificates, like so: Remove-Item *Root-CA.* -exclude *.crt

Signing Scripts

You can now sign scripts at will using the pfx file generated in step 2 with the Get-PfxCertificate cmdlet, or the Cert: provider (you have use -ImportAll when you call New-CodeSigningCert, not just -Import — that causes the import of your code-signing certificate to the “My” store … then you can do Get-ChildItem "Cert:\CurrentUser\My\$thumbprint" where $thumbprint is your certificate’s thumbprint).

Let’s rename our Profile.ps1 script and then sign it. We’ll create create a folder called “AutoModules” to stick your PSCX profile (and any future such scripts) in. If you’re still on PowerShell 1, you’ll just put the .ps1 files into that folder, and we’ll dot-source them.

Edit: The simplest

Checking for valid signatures

Once you’ve got that set up, we need to create the new profile script and modify it to load not just the PSCX module, but any other signed scripts we put in there. Paste this into your new Profile.ps1, and don’t forget to sign it.

If you’re testing PowerShell 2, you’ll want to treat these scripts as Modules … SnapIns themselves can be loaded as modules in PowerShell 2, but so can any script which needs to be dot-sourced. In this case, the easiest thing is to create a subfolder for each one and then use Add-Module with the folder name. There are some problems with that, which I’ll go into in Part 3 (no link until I’ve written it this time, I learned my lesson).

I’ve decided to spend part of this weekend organizing my sock drawer PowerShell profile, and since there doesn’t seem to be a lot of best practices or prescriptive advice on setting up your PowerShell profile, I thought I’d write about how I do it, to help any new users out there get started.

If you are a .Net developer or a Computer Science researcher, and haven’t started using PowerShell, let me just make this argument for learning it:

PowerShell cmdlets are simple to write. The additional code required to make a .Net class into a cmdlet is very minor (at a minimum, a single method with a line of code to Write-Output something). Since cmdlets must derive from either Cmdlet or PSCmdlet you can’t just write every class as a cmdlet, but even non-cmdlet classes can be instantiated via New-Object and their methods and properties can be exercised.

After writing portions of a recent research project as PowerShell cmdlets, I would say that the ability to write a simple class and then use/execute from the command line and in scripts — plus the built in automatic argument parsing (PowerShell assigns pipeline input and command line parameters to object properties for you) makes PowerShell cmdlets quite useful for research and development. Because the overhead is so minimal, it minimizes the distractions of creating a user interface and allows you to focus your actual research goals. Additionally, because the PowerShell-specific code can be restricted to attributes and a method or two, the actual classes are still usable for other purposes. That is, your class is invokable within PowerShell, but it’s also reusabe within any .Net application, whether Windows.Forms, WPF, or even web apps.

Of course, since I’m primarily using PowerShell for research and development, I’m using the current CTP 2 release of PowerShell v2, some of what I’m doing in my profile won’t work until PowerShell 2 is released (next year?). I’ll try to mark the CTP-only features appropriately and offer PowerShell 1-compatible workarounds.

The first step, of course, is to install the software. To take full advantage of PowerShell 2, you need to have the .Net Framework 3.5 although you could use the .Net Framework 2.0 instead if you’re just using PowerShell 1.0. PowerShell 1 is only available as a “patch” so you have to choose the correct download for your platform through the Microsoft Download Center. However, since it’s still in CTP release, there is an installer for PowerShell 2 CTP2.

I also use the PowerShell Community Extensions and recommend them, although there’s a lot in there you may not use, there’s also a lot of extremely useful things — some of which are practically requirements. This package needs updating to work well with PowerShell 2, because PowerShell 2 has implemented natively some of the most popular features in PSCX, but it’s just a few minor changes. If you’re just getting started (or want to follow along with this series of posts), you should let PSCX install it’s profile (if you already have a profile, then in order to follow along with these posts, you should MOVE your profile out of the way before you install PSCX, and go ahead and let it create a profile from scratch).

Part 2

The cool thing about the way authenticode signatures are implemented is that even if a script is signed with a self-issued certificate, you can still tell if the script has been tampered with… Check this out:

As you can see, in the case of a UNTRUSTED, but correct, signature, you get the UnknownError status. If you checked the output object, it has a StatusMessage which says “A certificate chain could not be built to a trusted root authority”. If the script has been altered (as in my SampleBAD.ps1 scripts) then the signature is incorrect, you get the HashMismatch status, and the corresponding StatusMessage is: “The contents of file SCRIPTS:\TrustedCert\SampleBAD.ps1 may have been tampered because the hash of the file does not match the hash stored in the digital signature. The script will not execute on the system…”

One odd thing is that the messages are inaccurate about not executing the script: if you have your execution policy set to Unrestricted, the signatures aren’t checked at all, and if you have it set to RemoteSigned they are only checked for remote scripts. Furthermore: if you do have your execution policy set to AllSigned, neither the UnknownError nor the HashMismatch script will execute — only the one Valid scripts will.

So what?

The bottom line is: you can verify that nothing has happened to the script — even if you don’t trust the person who signed it nor the person, group, or company that issued a certificate to them. Why does this matter? Well, I recently wrote a post about generating self-signed code-signing certificates which can be used for signing PowerShell scripts, and if you chose to distribute scripts signed with one of those certificates, nobody would be able to verify the root CA(Certificate Authority) and so the signatures would never come out as valid.

Is there any usefulness in this? Well, I guess that depends on your perspective, but basically, I think that if I published my scripts signed and tell you on my blog what my certificate thumbprint is … that you’d be more able to trust those scripts than you are now (when they’re not signed at all). Of course, I could go one step further, and publish my own self-signed root CA certificate so you could choose to trust that …

I was recently having a conversation about the future of the PowerShell Script Repository and it involved some discussion of whether it would be safe to use the Repository Scripts to download dependencies automatically… The answer, obviously, is no.

But it started me thinking again about scripts being signed. If you had already chosen to run a script provided by me (which was signed by a certificate you couldn’t verify), maybe you’d be willing to trust other scripts signed by the same certificate, so we could automatically download them. Well, maybe even then you wouldn’t want to trust it, but lets assume that you were running a copy of the PowerShell Script Repository internally at your company …

Would you use automatic dependency downloading?

We could easily have a function that takes the script name and verifies that you have that script available — and if not, it could fetch the script from your designated repository and verify that the signature is valid even if the certificate isn’t signed by a root certificate authority you trust.

Of course, such automatically downloaded scripts would need to be marked as “Remote” so if you had your Execution Policy set to AllSigned or Remote Signed, then the script would only run if you had trusted it’s author (and you wouldn’t even be offered the option if you hadn’t trusted the CA(Certificate Authority) that issued his script. In that case you would need to review the script and re-sign it yourself — or manually remove the “remote” bit.

Imagine something like this:

When you tried to execute Get-Paste, it would check for Get-Webfile, and if it couldn’t find it, would attempt to download it (presumably this would involve asking your permission, and placing it in some specific location that was in your PATH, so that the script could find it when it tried to execute it on the next line).

Or maybe, an Apt-Get?

Perhaps instead of this mechanism, we could use the new embeddable “Data Language” to provide a list of dependencies, like: DATA Dependencies { scripts = Get-WebFile } and run a Resolve-Dependencies function against each script before trying to execute it — this way, if you downloaded a script from the repository using Get-Paste, it could automatically Resolve-Dependencies and offer to download the other scripts at the same time.

The fact is that doing this correctly will require some major reworking of the script repository to allow tracking new versions of scripts better, and to let the script repository track dependencies explicitly so that you don’t have to download the whole script to find out what it’s dependencies are, but this could be done, if people are actually interested in it.

A web of trust?

Ad I’m thinking about this, I’m wondering again about the possibility of creating an informal web-of-trust style code-signing certificate tree. The idea would be that the Script Repository would have a CA certificate of it’s own, and would issue code-signing certificates to PowerShell developers cheaply (free?) by skipping over some of the usual verification steps. In an ideal world, Microsoft would issue the PowerShell Community a “SubCA” certificate signed by their root — in the interests of promoting code signing for PowerShell …

However, if we couldn’t get a SubCA certificate for “free” or cheap, we could simply generate and self-sign our own, and publish it on the Script Repository website, requiring users to download and import it into their trusted roots if they wanted to use trust permissions. Regardless of whether they chose to trust it or not, they could still verify the scripts were valid, which is better than what we have now — the rest would be up to the user.

Of course, if we were issuing certificates that were self-signed anyway, we could go a step further and sign SubCAs and distribute them to, say, the Microsoft PowerShell MVPs and trusted community leaders after verifying email addresses and physical mailing addresses etc … trusting them further to issue (less trusted) code-signing certificates to additional developers.

Call to action

All of this is extra work for the people maintaining the script repository web site (right now, that’s me), but it might be worth it if it makes it easier to use the script repository, easier to trust the scripts on it, and easier to verify that an author is who he says he is … what do you think?

1. Should we put in the work to set up a web of trust or should we leave it up to individual developers to self-sign and generate their certificates (and publish their public roots on their websites or something)?
2. Should we just leave it at that (scripters can sign their scripts if they feel like it), or should we push and promote script signing? As an incomplete example of promoting code-signing, I mean:
   1. We can use certificates as the primary (or only) way for authors to identify themselves (that is: no log-ins, unsigned scripts are marked “anonymous” ... but we track the thumbprints and allow you to browse scripts signed by the same author, etc).
   2. We can include the signature thumbprint with the short descriptions output on the search results and from the scripts which interact with repository.
   3. We can restrict “latest version” updates to only scripts which are signed, and optionally to new versions signed by the same certificate.
3. Is there any point (or hope) in trying to get a signed CA certificate? Can Microsoft help us out? Do any of you work at a certificate authority?

One of the major security features of PowerShell is the support for code signing of scripts, so that you can set an execution policy that requires scripts to be signed before they can be run. Of course, it goes a bit further than that. When a script has been signed by a certificate with a root Certificate Authority (CA) that you don’t already “know” or trust it can’t be run at all until you add the root CA to the system’s certificate store.

Even after you trust a specific authority, you haven’t trusted a script author — so any signed script you run will prompt you whether you want to allow it or not, like so:

The important thing to note here is that you’re really being asked not about the script, but about the author. If you choose the Never or Always options, the certificate that was used to sign the script is added to the appropriate certificate store (“Untrusted Certificates” or “Trusted Publishers”, respectively). To be clear: this happens for each every new author certificate, regardless of whether it’s signed by a self-signed cert (where you’ve already installed the root certificate in your root store) or a certificate issued by a commercial CA — there’s no loophole, no matter what anyone may have said in the past.