Tag Archives: Testing

Did you know PowerShell can use Selenium?

This is sort-of a place-holder for a full-length post that I really ought to write about driving web testing from PowerShell using Selenium. I actually have a little module around for doing that with WaTiN, but honestly the Selenium project seems to be a lot more active, and has quite a bit of muscle behind it since they’ve merged with WebDriver…


Add-Type -path ~\Downloads\selenium-dotnet-2.16.0\net40\WebDriver.dll

# Navigate to google in IE (or Firefox, Chrome, Opera, etc)
$driver = New-Object OpenQA.Selenium.IE.InternetExplorerDriver
$driver.Url = "http://google.com"

# Type PowerShell into the query box, the page will update via AJAX
# Note we won't hit enter or anything
$q = $driver.FindElementByname("q")
$q.SendKeys("PowerShell")

# Use a CSS selector to find the first result link and click it
$driver.FindElementByCssSelector("li.g h3.r a").Click()
 

One Catch

The Security tab of the Internet Options dialogIf you try this with IE and you get the error Unexpected error launching Internet Explorer. Protected Mode must be set to the same value (enabled or disabled) for all zones ... it means exactly what it says. You need to open “Internet Options” from your start menu (or from IE), and go through each “zone” and set the “Enabled Protected Mode” check box to the same value for each zone (either all checked, obviously the most secure, or all unchecked). I’m not going to debate whether setting them all unprotected is a good idea … I set mine to all protected, but I don’t generally use IE anyway.

If you want more help, Selenium’s documentation is great, and there’s a section on Getting Started with Selenium WebDriver which I found quite helpful (make sure your examples are in “csharp” and you can almost just copy and paste — someone should offer to do them in PowerShell).

If you want more information about the Internet Explorer driver and this problem in particular, the short answer is that “Protected Mode” is a security boundry, so if you cross over it the COM automation object doesn’t work — thus, you need to make sure you always stay on the same side. There’s a good discussion on the mailing list archive about how it works and why, as well a weird alternative documented on the Selenium JavaDocs

Arrange – Act – Assert: Intuitive Testing

Today I have a new module to introduce you to. It’s a relatively simple module for testing, and you can pick it up in short order and start testing your scripts, modules, and even compiled .Net code. If you put it together with WASP you can pretty much test anything ;-)

The basis for the module is the arrange-act-assert model of testing. First we arrange the things we’re going to test: set up data structures or whatever you need for testing. Then we act on them: we perform the actual test steps. Finally, we assert the expected output of the test. Normally, the expectation is that during the assert step we’ll return $false if the test failed, and that’s all there is to it. Of course, there’s plenty more to testing, but lets move on to my new module.

The module is called PSaint (pronounced “saint”), and it stands, loosely, for PowerShell Arrange-Act-Assert in testing. Of course, what it stands for isn’t important, just remember the name is PSaint :)

PSaint is really a very simple module, with only a few functions. There are two major functions which we’ll discuss in detail: Test-Code and New-RhinoMock, and then a few helpers which you may or may not even use:

Set-TestFilter

Sets filters (include and/or exclude) for the tests by name or category.

Set-TestSetup (alias “Setup”)

Sets the test setup ScriptBlock which will be run before each test.

Set-TestTeardown (alias “Teardown”)

Sets the test teardown ScriptBlock which will be run after each test.

Assert-That

Assserts something about an object (or the output of a scriptblock) and throws if that assertion is false. This function supports asserting that an exception should be thrown, or that a test is false … and supports customizing the error message as well.

Assert-PropertyEqual

This is a wrapper around Compare-Object to compare the properties of two objects.

How to test with PSaint: Test-Code

Test-Code (alias “Test”) is the main driver of functionality in PSaint, and you use it to define the tests that you want to run. Let’s jump to an example or two so you can see the usefulness of this module.

Let’s start with an extremely simple function that we want to write: New-Guid. We want a function that generates a valid random GUID as a string. We’ll start by writing a couple of tests. First we’ll test that the output of the function is a valid GUID.

test "New-Guid outputs a Guid" {
   act {
      $guid = New-Guid
   }
   assert {
      $guid -is [string]
      New-Object Guid $guid
   }
}
 

Now, to verify that the test works, you should define this function (the GUID-looking thing is one letter short) and then run that test:

function New-Guid { "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaa" }
 

Another proof that it works would be that it should fail on this function too, because “x” is not a valid character in a Guid:

function New-Guid { "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" }
 

So, let’s write a minimal New-Guid that actually generates a valid Guid:

function New-Guid { "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa" }
 

If you run our test on that, you will see:

   Result: Pass

Result Name                          Category
------ ----                          --------
Pass   New-Guid outputs a Guid
 

If you don’t like the fact that the Category is empty, you could add a category or two to the end of our test. We should also switch to using Assert-That if we want to know which test in the assert failed. Finally, we want to write another test which would test that New-Guid doesn’t just return the same Guid every time, the way ours does right now:

test "New-Guid outputs a Guid" {
   act {
      $guid = New-Guid
   }
   assert {
      Assert-That { $guid -is [string] } -FailMessage "New-Guid returned a $($guid.GetType().FullName)"
      New-Object Guid $guid  # Throws relevant errors already
   }
} -Category Output, ValidGuid

test "New-Guid outputs different Guids" {
   arrange {
      $guids = @()
      $count = 100
   }
   act {
      # generate a bunch of Guids
      for($i=0; $i -lt $count; $i++) {
         $guids += New-Guid
      }
   }
   assert {
      # compare each guid to all the ones after it
      for($i=0; $i -lt $count; $i++) {
         for($j=$i+1; $j -lt $count; $j++) {
            Assert-That ($guids[$i] -ne $guids[$j]) -FailMessage "There were equal Guids: $($guids[$i])"
         }
      }
   }
} -Category Output, RandomGuids
 

Now, we have to actually fix our New-Guid function to generate real random Guids:

function New-Guid { [System.Guid]::NewGuid().ToString() }
 

And at that point, we should have a function, and a couple of tests that verify it’s functionality…

The finer points of assertions

One thing you’ll notice the first time you use Get-Member after loading the PSaint module is a few script properties have been added to everything. I did this because I found myself writing the same Assert-That calls over and over and decided that it would be slicker to make these extension methods than to write new functions for each one:

MustBeA([Type]$Expected,[string]$Message)
MustBeFalse([string]$Message)
MustBeTrue([string]$Message)
MustEqual([Object]$Expected,[string]$Message)
MustNotEqual([Object]$Expected,[string]$Message)
 

There’s also a MustThrow([Type]$Expected, [string]$Message) which can be used on script blocks (note that this function executes the ScriptBlock immediately, so be careful how you use it).

We can use these to tidy up our tests quite a bit, while still getting good error messages when tests fail:

test "New-Guid outputs a Guid String" {
   act {
      $guid = New-Guid
   }
   assert {
      $guid.MustBeA( [string] )
      New-Object Guid $guid # Throws relevant errors already
   }
} -Category Output, ValidGuid

test "New-Guid outputs different Guids" {
   arrange {
      $guids = @()
      $count = 100
   }
   act {
      # generate a bunch of Guids
      for($i=0; $i -lt $count; $i++) {
         $guids += New-Guid
      }
   }
   assert {
      # compare each guid to all the ones after it
      for($i=0; $i -lt $count; $i++) {
         for($j=$i+1; $j -lt $count; $j++) {
            $guids[$i].MustNotEqual($guids[$j])
         }
      }
   }
} -Category Output, RandomGuids
 

COM Objects

PSaint also has a wrapper for COM objects to help with testing them. It adds GetProperty and SetProperty methods to allow you to access COM object properties which don’t show up on boxed COM objects (a common problem when working with MSOffice, for instance). It also adds InvokeMethod for COM objects to invoke methods that don’t show up for similar reasons. These, of course, only help you if you’re already fairly literate with the COM object in question.

Mock Objects

PSaint includes New-RhinoMock, a function for generating a new mock object using RhinoMocks (which is included). Rhino Mocks is a BSD-licensed dynamic mock object framework for the .Net platform. Its purpose is to ease testing by allowing the developer to create mock implementations of custom objects and verify the interactions using unit testing.

I have to admit that this New-RhinoMock function is incomplete, and exposes only a fraction of the options and power in RhinoMocks, but it’s been sufficient for the few times when I’ve wanted to actually mock objects from PowerShell, so I’m including it here.

For those of you (developers) who want to know why RhinoMocks instead of your favorite mocking framework, the answer is astonishingly simple: it had the least number of necessary generic methods (which are impossible to call in PowerShell 2).

Using PowerShell and WatiN (PowerWatin?)

This is an update to a previous article.

Someone asked (on Twitter) about using WatiN from PowerShell, and pointed to this old post by Scott Hanselman saying he was having the same problems … so I wrote this to help them out:

WatiN requires -STA mode

Note: WatiN requires Single Threaded Apartment mode, so you need to be using PowerShell 2.0 (currently in CTP3) in order for any of this to work, and you need to pass the -STA parameter to PowerShell. Regardless, I thought I’d throw two tips out here:

Don’t use LoadFile, use LoadFrom

I’m not 100% sure when it’s appropriate to use LoadFile in PowerShell, but I can tell you that if your assembly is in a folder with a bunch of other assemblies upon which it depends … you need to use [Reflection.Assembly]::LoadFrom( $path ) instead — because the .net loader will be able to find the dependencies.

Generate some functions to help yourself out

It’s trivial to do code-generation in PowerShell, and WatiN is not friendly to the PowerShell syntax, so you’re going to want to generate a bunch of them. To get you started, here’s a set of Find-* functions to let you find each type of element that WatiN recognizes and automates… by name, id, class, style … well, by any attribute, really:

Continue reading