Well, after multiple attempts at a wget PowerShell script (the last one works very well for downloading web pages, files, as long as you don’t need to send post parameters or anything like that) ... I found myself writing a script last week that included a custom HTTP POST function as well as using some prior functionality I wrote (ConvertFrom-Html) to convert HTML files to XML documents — which PowerShell can deal with nicely.

So I’ve taken my Huddled.Html code and worked it in together with a few extra bits … and I’m releasing it under the new name “PoshHttp” (which has the sole benefit of being short, in the case when you need to type the full command names, like: PoshHttp\Get-Web.

In lieu of providing proper documentation for this, there’s a script below which shows off some of the features of the Get-Web cmdlet … but first, you’re going to want to download PoshHttp and I can give you a basic overview.

Get-Web

Downloads a file from the web URI (Uniform Resource Identifier), optionally passing POST or GET parameters which it accepts as a Hashtable. If the resulting file is an xml or html file, it converts it to XML and outputs it as an XmlDocument object. If they are not, it saves them to file, and outputs the FileInfo object.

• Plain will prevent the conversion to xml
• Save will force saving to file (either xml or binaries)
• Path lets you specify the file name/path (normally, it selects a path based on what you’re downloading).
• Force will prevent it from asking if you want to overwrite files

ConvertFrom-Html

Converts HTML to XML. I’ll eventually tweak this so it supports passing it a file path and even saving the xml in-place on the original file … but for now you can use it on a stream of text, like this: (Get-Content file.html | ConvertFrom-Html).save( "file.xml" ). Note that it outputs an xmldocument object … you can get the xml as text to the console using Get_OuterXml().

ConvertTo-Hashtable

Converts a url-style query string or NameValueCollection into a Hashtable. It’s mainly to make it easier to use the Get-Web in cases where you’re getting the URL or POST data from elsewhere.

Here’s a script with a few example uses of Get-Web: (more…)

Edit: I made a mistake …

I wrote this a few weeks ago, and made an error in the script which causes it to not work properly with most binary files … I’ve fixed the script down below now, and while I was at it, I went ahead and added a download progress report

About a year ago, I wrote a script to download files from the web using PowerShell, but it was so simple that you had to specify the url to download and the file to save to. At the time I knew there was a way to find the file name of the download in the header, but I couldn’t remember how to get to the headers, and it’s not possible using System.Net.WebClient so I just dropped it.

Then the other day I saw an old post from Script Fanatic about querying Http status codes, and it reminded me of my annoyance with my wget script. So I fixed it, and I’m letting you have it.

The new version of wget for PowerShell (or Get-WebFile) is on the PowerShell Repository and it’s been completely rewritten to use System.Net.HttpWebRequest and HttpWebResponse which gives us access to the the name of the file in those cases where the URL is something like .../scripts/?dl=189 (assuming the server puts the file name in the headers as it should).

It still doesn’t support authentication, and even though the headers include the size (and I have to do the streams myself so I could easily read a few bytes at a time and give you progress reports)

Update: You should look at this other WGet for PowerShell script if what you want is to just download a file and have it work…

I’m probably the last person on the planet to figure out how easy it is to get a basic wget working in PowerShell … but just in case I’m not … here’s a PowerShell script which will download web files to your hard drive.

That’s all it does. This script doesn’t support authentication or anything (even though it does use your computer’s proxy settings), but I think it will be obvious how to extend it, since it’s basically just .Net Framework stuff.

That’s all there is to it. Actually, you really only need the last two lines (and the first line), as long as you understand that the default path will be your user directory rather than the current directory. (like: C:\Users\Joel OR C:\Documents and Settings\Joel\). That means that if you leave off that if(!(Split-Path -parent $path)) stuff, you’ll always have to specify the file starting with a .\ if you want it saved in the folder you’re in (otherwise it gets dumped in your user directory).

Those three lines check if the specified path has a folder in it (and if it’s a valid folder) and if not, they just save the file to the local current folder.

Here are some examples of how you’d use it, assuming you save that into a file called “wget.ps1” and put it in your path. Incidentally, just like in ‘nix shells, ~ represents your user directory … oh, and the generally more PowerShell-like name for the script would be “Get-Url.ps1” or something…