I’ve been having problems with the search functionality on the PoshCOde repository, and I just thought I’d throw this up here because I just now solved the biggest problem: ranking. Up until now, the results have not been returned in order of relevance — this is because the search works using MySQL’s FULLTEXT BOOLEAN search, which doesn’t return in relevance order, nor does it return an extra ‘score’ column.

I’ve fixed that, and weighted the search so that words in the title count more than words in the code by creating a relevance column by hand:

Incidentally, the FULLTEXT index means that words shorter than 4 characters don’t count (I’m going to try to get this changed, but it’s an option for MySQL, so it has to be changed in the config file) in the meantime you can search for words using the wildcard character, like: SQL* and it sort-of works. The PoshCode cmdlet actually was adding *‘s to the query (although I’ve just decided that’s not a good idea, because it means that queries from the cmdlet appear to have different results than queries on the website.

MySQL’s FULLTEXT BOOLEAN search has all sorts of features (and limitations): there is a stopword list, maximum and minimum word lengths, and all sorts of operators for setting word precedence or negating words, or weighting them negatively … to REQUIRE that a word be present, it must have a + in front, and in order to mark a word as more important, you have to put > in front, not just put it first… I’ve been thinking about trying to apply a few of those tricks myself (eg: put * on words under four characters, and put > on the first 30% of words and < on the last 30% to try to simulate weighting them …) but my original feeling was that the search is more powerful if you just know that it’s a fulltext boolean search and can write your queries accordingly.

If anyone has any ideas for how to improve search in MySQL … or opinions on whether I should try to apply boolean operators to queries which don’t already have them … please let me know.

There are two requests that I get with overwhelming regularity for the PowerShell script repository: first, that I would add some sort of “browsing” functionality, and second, that it should have an RSS feed. Well, browsing may have to wait until a future iteration of the repository, but feeds are easy, because they’re basically just a hack-up of the “recent” items that are in the repository sidebar already, so I did it this evening.

Presenting …

PowerShellCentral.com/scripts/feed

Right now the number of items is limited to about ten, but if traffic picks up I can increase that easily — you can limit the count by passing the number of items you want to list like this: ?list=2.

You can also create a feed for specific search results (the feed will show items in relevance order, but the date is in the RSS so if you want to sort by that, you can). For instance, if you wanted to keep up to date on scripts that used SecureStrings, you would append your search terms like this: ?q=*securestring*.

As usual, my modifications to the PasteBin.com site are available on the repository site, and without much in the way of documentation. I will say this: in addition to creating the feed.php file, I moved some of the common translation functions into a translate.php file, and modified the dates that were being returned from db.mysql.class.php — it’s all under the Affero General Public License that Paul Dixon wrote PasteBin under in the first place.

So, people keep talking about a better PowerShell script repository, and there’s at least 3 or 4 different people working on improvements, but for now, the one we have is working ok, and already has hundreds of scripts in it — but there hasn’t been a way to add or retrieve scripts from within PowerShell.

So while I was taking a break from working on my own replacement script repository I whipped up some scripts to Send-Paste, Get-Paste and Find-Paste. These scripts still need some work, but I think you’ll find them useful, so I’ve posted a version that is compatible with PowerShell v1 (and works fine on the 2.0 CTP) and doesn’t need any additional scripts (it includes the latest Get-WebFile script for downloading the files).

The Find-Paste function takes a search string, and adds wildcards — but otherwise performs the same search as you can already do on the web page. It returns the results in a PowerShell friendly PSObject which by default is just displayed in the host.

In order to download a script using Get-Paste, you must know the ID of the script — which you can get from the output of Find-Paste. Note that because Find-Paste does a full-text search, you may get results that are not the script you’re looking for (if they call that script, for instance), and you may get many versions of the script (and newer versions are not necessarily at the top of the search results).

The Send-Paste script is most useful for sending things to the temporary PowerShell pastebin on my site but can be used successfully to send scripts to the permanent PowerShell Script Repository … in fact, that’s how I uploaded the PowerShell Pastebin Functions to the repository … you just have to be careful to specify the title and description when you do that. Edit: I attached the script here on my site also because I made a mistake and deleted the original copy from the repository.

So, you now have something a little bit more like a proper script repository. Hopefully the Usage comments in the script will be enough to help you figure out how to use the various commands, (and I did blog about the Send-Paste script about a month ago) but just in case you need help, let me give you a simple example.

Suppose I need to find a script to encrypt text so I can store some SecureString objects (like the passwords in a PSCredential object) in a file…

It looks like I found two, but clearly the second one is not the right one (I mentioned this in one of my examples in the pastebin scripts, and amusingly enough, that screwed up my example). So to download the script I want, I just do:

Simple enough, right? I could have used the -Passthru option to see the script in the console, but since I was pretty sure it was the only one that might be useful (and since using -Passthrough just to display it on the console would mean downloading it a second time, if I actually wanted it). Another option would be to use the Tee-Object cmdlet, which would let us download it, show it in the host, and also save it in the file, all at once!

Incidentally, I have a version of this which uses my PoshHttp snapin and Get-Web cmdlet, but it’s also written as a module and therefore requires PowerShell 2.0 CTP2 — for various reasons, I’m choosing to release that one separately later. If you are using the CTP, you could take advantage of the WPF UI abilities and try piping Find-Script ... | "Select-Grid":http://huddledmasses.org/wpf-from-powershell-select-grid/ | Get-Script to pick the script you want visually.