Awhile back I wrote a Get-Web script, which I later converted to a compiled Get-Web cmdlet and module (pssnapin) ... adding support for POST forms. At the time, it seemed like GET and POST with some string variables pretty much covered what I’d ever want to do with web pages, however I found a few places where I needed to POST a file attachment, and then a couple of full-on REST APIs, which use not just GET and POST, but PUT and DELETE as well.

While I was playing with scripting the REST API for DekiWiki … I experimented with the MindTouch Dream SDK … and as a result, I’ve refactored the part of that module that actually does the HTTP work out into a little PowerShell 1.0 compatible script library. Of course, it depends on several dll’s (mindtouch.core.dll, mindtouch.dream.dll, and SgmlReaderDll.dll and log4net.dll) from the Dream SDK, which you can download from SourceForge.

Example Uses


## Retrieve Google search results:
function Get-Google {
  Invoke-Http GET http://www.google.com/search @{q=$args} |
    Receive-Http Xml "//h3[@class='r']/a" | Select href, InnerText
}

## Download a file from a web page
function Get-WebFile($url,$cred) {
  Invoke-Http GET $url -auth $cred | Receive-Http File  
}

## Upload a PowerShell script to PasteBin
function Send-Paste {
PARAM($PastebinURI="http://posh.jaykul.com/p/",[IO.FileInfo]$file)
PROCESS {
  if($_){[IO.FileInfo]$file=$_}

  if($file.Exists) {
    $ofs="`n"
    $result = Invoke-Http POST $PastebinURI @{
      format="posh"           # PowerShell
      expiry="d"              # (d)ay or (m)onth or (f)orever
      poster=$([Security.Principal.WindowsIdentity]::GetCurrent().Name.Split("\")[-1])
      code2="$((gc $file) -replace "http://","http``://")" # To get past the spam filter.
      paste="Send"
    } -Type FORM_URLENCODED -Wait
    $xml = $result.AsDocument().ToXml()
    write-output $xml.SelectSingleNode("//*[@class='highlight']/*").href
  } else { throw "File Not Found" }
}}
#########################################################

## Example, to download my PasswordRequired.dll:
$cred = new-object System.Management.Automation.PSCredential `
                   "test", $(ConvertTo-SecureString "password")
Get-WebFile http://huddledmasses.org/downloads/PasswordRequired.dll $cred

## Example, to upload a script called TestScript.ps1 to the Pastebin we use on IRC:
ls TestScript.ps1 | Send-Paste "http://posh.jaykul.com/p/"

 

There are basically two main functions: Invoke-Http and Receive-Http. There are also a few extra functions which are mostly just for use by these two functions, and then Set-HttpDefaultUrl and Set-HttpCredential which you can use to simplify your scripts. I intend to rewrite these as cmdlets in C# and contribute them to the PowerShell Community Extensions along with some of my other networking cmdlets (Get-DNS), so if you have any suggestions, I’d love to hear them!

Invoke-Http

Invokes HTTP verbs (like GET, POST, PUT, and DELETE) against a URL with optional parameters (eg: name = value), input, etc … and returns the output as a MindTouch.Dream.Result or (optionally) as a MindTouch.Dream.DreamMessage …

Syntax

Invoke-Http [-Verb] <String> [[-Path] <String>] [[-With] <Hashtable>] [[-Content] <Object>] [[-Type] <String>] [[-Authenticate] <String>] [-WaitForResponse]

Parameters

Verb

The HTTP verb you want to invoke. Defaults to GET, but can be anything your HTTP server supports.

Path

The URL you want to invoke against. Can be a partial URL, if you first call Set-HttpDefaultURL first. You can also specify this as an array or a hashtable. This is particularly useful if you’re specifying a partial URL, and your URL is built based on logic in your code. If you pass an array, the values in the array are joined with forward-slashes. If you pass a hashtable, the keys and values are appended joined with forward-slashes, but the values are double encoded (by UrlEncode), ie: $key/$(UrlEncode(UrlEncode($value))

With

The parameters to be passed with the verb call, as a hashtable. (these are particularly used for POST, etc). These are passed in the query as named parameters, with the values encoded.

Content

Anything you want to pass as content. In actuality, we really only handle three things: XML Documents, file paths (the file will be used as content), and everything else is passed as simple text. If you need other data types, the “DreamMessage” that the scripts depend on can handle all sorts of data types, so you’ll just need to add it around line 50 of the script.

Type

This is the type of content. We can automatically deduce this for XML or File objects, but if you want to force a type, you should use this parameter. For instance, to fake a web form submission, you need to specify the type as FORM or FORM_URLENCODED but you can also specify the type as any of: ATOM, BINARY, BMP, CSS, DREAM_EXCEPTION, FORM_URLENCODED, GIF, HTML, HTTP, JPEG, JS, JSON, MULTIPART_MIXED, NOTHING (mimetype -/-), ANY (mimetype */*), PDF, PHP, PNG, RDF, RELAXNG, SVG, ANY_TEXT (mimetype text/*), TEXT (mimetype text/plain, with the ISO-8859-1 encoding), TEXT_UTF8 (mimetype text/plain, with UTF8 encoding), TEXT_XML, TIFF, VCAL, VERSIT, XHTML, XML, XRDS.

Authenticate

You can pass either a boolean (in which case the HTTPRest scripts will use a default $global:HttpRestCredential which it will prompt for if you haven’t set it), or an actual PSCredential or NetworkCredential object. In any case, this causes the invoke to contain network credentials. Note that sometimes you may need to pre-authenticate before making calls to a web-service.

WaitForResponse

A switch parameter that causes the query to wait until the response completes downloading, and return a DreamResponse instead of returning an incomplete Result ... this is useful when you know the response is a simple XML file and you need to parse it to determine whether your call has succeeded. If you need to actually get the data out, or if you just need to know the method was invoked successfully, you shouldn’t pass this parameter.

Receive-Http

Receives the response or result from a Dream.Result or DreamMessage. as XML, XDoc, text, raw bytes, or even to file. This function is really only useful to receive the output of Invoke-Http when you need the response content.

Syntax

Receive-Http [[-Output] <string:[Xml|File|Text|Bytes]>] [[-Path] <string>] [[-InputObject] <MindTouch.Dream.Result<MindTouch.Dream.DreamMessage>>]

Receive-Http [[-Output] <string:[Xml|File|Text|Bytes]>] [[-Path] <string>] [[-InputObject] <MindTouch.Dream.DreamMessage>]

Parameters

Output

The TYPE of output you want:

  • Xml will return an XmlDocument — or XmlNode(s) if you pass a Path.
  • File will write to file — currently, you must pass a path, I didn’t make it guess it automatically.
  • Text will output text — in the case where the actual output is XML, this returns the InnerText.
  • Bytes returns an array of bytes…
  • XDoc returns a MindTouch.Dream.XDoc (not the .Net native class), which has a boatload of extra web-related methods and features you might find useful such as accepting XPath operators as indexers.

Path

A string containing the path for file output or XPath selector for XML, XDoc, and Text output

InputObject

The input must be either a Result<DreamMessage> or a DreamMessage itself. Values passed on the pipeline are accepted into this parameter.

Set-HttpDefaultUrl

This takes the passed URI and sets it as the base (default) URL for the Invoke-Http function (if you set this, any path passed to Invoke-Http is treated as relative to this path).

Set-HttpCredential

This takes a PSCredential or NetworkCredential and stores it as a PSCredential for use by the Invoke-Http function. Particularly useful when you want to script a bunch of authenticated methods without being re-prompted. Note that if you don’t pass a credential, you’ll be prompted for one the usual way, so calling just Set-HttpCredential is sufficient when working on the command-line.

Miscellaneous

There are other functions in here, a few should be considered “internal” functions (when this is a module, they would not be exported: Get-DreamMessage, Get-DreamPlug, Get-HttpCredential) and a few of which came out of my utilities functions (like Encode-Twice, and Join-Url, Get-FileName, Get-UtcTime, and Get-CredentialBetter)...

Code

3 Responses to “Using REST API’s from PowerShell with the Dream SDK”