Before I give the solution, I just want to say: don’t change your output based on where you are in a pipeline.

There are numerous scenarios where your function will be the last one on a pipeline, but still be participating in further pipelines, including formatting and output modification. For example, take our function Test-Pipeline (defined below) in these three scenarios below. In none of these scenarios would it be appropriate for the function to write formatted output instead of outputting the raw object, but in each case, the function is the last function in the pipeline.

However, if you want to determine your function’s position in the pipeline for some other reason, the answer is simple. You need to use $MyInvocation and compare the PipelineLength and PipelinePosition properties:

Of scripts and functions

The first thing to know is that in PowerShell, there’s really no difference between a script (just a file with a .ps1 ending) and a function as you’ll see written below. If you take a function Get-Square, and remove the first and last lines ( Function Get-Square { … } ) you could put them in a file called “Get-Square.ps1” in your PATH, you use them exactly the way you would the function that was pre-loaded into memory by dot-sourcing or pasting it on the command line. Of course, doing that easily requires writing the function parameters on their own line using PARAM(...) syntax, which is why I recommend doing that.

Of functions and the pipeline

When your script or function is used on the pipeline its begin block is called once when the pipeline starts up, and then the process block is called repeatedly: once to process each pipeline object, and finally, the end block is called after all the objects have been processed through the whole pipeline. If you don’t understand that, you should play with this function, try calling it by passing a series of numbers through multiple instances of it:

Incidentally, if you don’t specify the begin, process, or end blocks, the body of your function is treated as the end block. This is so that you can use the special $Input variable, which collects all the things passed in on the pipeline, and thus only works in the end block. That would allow the following function to behave the same way regardless of whether it was invoked on the pipeline or by passing an $InputObject. Notice, however, the difference between this, and the function above.

Our challenge

The basic idea here is to rewrite that function such that it can be used to process a set of numbers from either the pipeline or an argument, without interfering with the processing of other parameters. We require that the function process items as they come in, rather than waiting until it’s received all input before processing them the way Test-SquareEnd does.

In PowerShell 2.0 this would be easy

In the current CTP 3 of PowerShell 2, you just specify ValueFromPipeline=$true for the parameter you want to set from the pipeline, and the function will work the same way whether you pass the numbers as a parameter or along the pipeline — you can even control which attribute of the objects on the pipeline will be used, but that’s a whole other article.

Of course, PowerShell 2.0 is still in beta status, and even after it’s released you may need to write scripts that are backwards compatible to PowerShell 1.0, and the excercise of doing so may help you to understand more about how PowerShell functions work, and particularly how they behave in the pipeline.

Our solution

Since in a PowerShell 1.0 function it’s not really supported directly, we need to do some extra work:

An explanation

Of course, this is just an example method, squaring things isn’t that exciting — but what’s special about it is that the output is almost exactly the same whether you call it with parameters Get-Square 1,2,3,4 or on the pipeline: 1,2,3,4 | Get-Square.

The trick is that it actually executes the same way in either case:

1. If you call it by passing the int (or array of ints) as an argument ($InputObject), it calls itself and passes those values on the pipeline.
2. When the integers are passed on the pipeline, the special pipeline iterator variable $_ is set, and the process block is executed.

1. When it has to (re)invoke itself, it passes any other parameters as parameters, which means you need to have default values for them.

There is one tiny difference in the processing, which in real-world use is practically never noticeable (you can see it in our example if you call it like this):

You’ll see that unlike the PowerShell 2.0 pipeline function, when you pass the numbers as a parameter, the first one of them actually gets passed through the process block before the begin block of the second function on the pipeline is called. This usually doesn’t have any effect, but it’s something to keep in the back of your head.

A few precautions:

• You have to default values for parameters, because you’ll be passing them all as named parameters, in the re-invoke step, and $null can cause problems.
• If you need to do additional processing in the begin block, you should only do so in an ELSE case: when $InputObject is null. That way, the code will only execute once each time you call the function.
• The same goes for the end block: you have to keep your code in an If(!$InputObject) block to avoid executing it twice (when you pass the values as an argument, and it re-invokes itself).

• The test cases in the process block must wrap all of your process block code, so that you don’t process the arguments twice, and you shouldn’t refer to $InputObject, but instead should use the automatic $_ variable which is the value passed when the function is (re)invoked via the pipeline.

Here’s some sample output, in case you’re wondering:

Improvements to the underlying web search API

You’ve always been able to pass a LIST parameter to the API, and get more results by specifying a higher number. But it never worked with the “path” notation (until now).

That is, you used to be able to do:

• http://poshcode.org/api1?q=start&list=10

• http://poshcode.org/api1?q=start&list=100

To make the API a little easier to use I’ve enhanced it just now:

1. You can now page the search results.
2. You can use the word “limit” instead of “list”
3. If you specify limit=0 (or list=0) I’ll give you everything I’ve got. Please use a little precaution about that, as it could be a LOT of data. I’d much rather you retrieve, say … 25, and then get the second page if you want more.

1. You can use path notation.

So, you can use any of these URLs:

• http://poshcode.org/api1/start/list/25/page/1
• http://poshcode.org/api1/start/limit/25/page/2
• http://poshcode.org/api1?q=start&list=25&page=3

• http://poshcode.org/api1?q=start&limit=25&page=4

There are a lot of search results for “start” ... feel free to play with enhancing the PoshCode module, or incorporating this into your apps, etc.

I should add that you don’t have to specify the limit or page number. By default you’ll get the first 10 items, which should be enough.

user: Can you explain the distinction between $_ and a param in a script context?
Jaykul: I sure can. And I think I can do it simply … take an example script|function … actually, we’ll just use a scriptblock
Jaykul: it can have pipeline blocks, or not (the pipeline blocks are the BEGIN{...} PROCESS{...} END{...})
user: yeah, I’m with you
Jaykul: so if you pass a value via an argument, it’s present in all three blocks, and has the same value the whole time (unless you change it in your script)
user: really
Jaykul: But when you put that script in a pipeline, your BEGIN {} block is called first, and then for EACH item in the pipeline, powershell sets $_ and calls your PROCESS block, and finally, your END{} block is called after all pipeline items have been processed
Jaykul: Here, check this out (run it so you can see)

user: that really helps, thank you
Jaykul: no problem. Incidentally … in a script-block like that, you can use the Params() statement, but I wanted to loop over $args
user: based on that I’m left wondering what the value of parameters is.
user: If I’m writing a script, chances are that I’m going to want to use it in a pipeline
user: I guess I can use parameters, but put it in a foreach-object block
user: $arr | %{myscript $_}
Jaykul: Well, if you use the foreach-object block and pass arguments, your script doesn’t persist through the life of the pipeline, so you can’t do things like counting, or comparing to the previous object, etc.
Jaykul: But you can only pass one thing at a time via the pipeline, so you still need arguments in a script like the one I just used to paste that script to the pastebin (which is here: http://huddledmasses.org/powershell-send-paste-script/ )
user: right. So I guess the concept is that scripts grab stuff off the pipeline themselves?
Jaykul: sort-of, yeah. I mean, it’s actually PowerShell that does it, but basically, the process block grabs each item off the pipeline without needing it to be specified as an argument.

Jaykul: In v2, you can be more explicit about this, because you can write a “cmdlet” in script which lets you specify parameters which should get the pipeline value
user: okay, I totally get it now. That’s why you’d write a cmdlet instead of a script?
Jaykul: Yeah! The particularly cool thing about cmdlet parameter handling … is you can do this:
Jaykul: [Parameter(FromPipelineByPropertyValue)] [Alias(“FullName”)] $FileName
Jaykul: a parameter like that would accept a string as a parameter, but if you did gci * | scriptcmdlet instead of getting the [System.IO.FileInfo] object you would get the STRING corresponding to that object’s FullName property
user: right. sweet!
Jaykul: so you can even do tricky stuff like getting multiple properties from it (as different parameters to your script)

user: I noticed from http://powershellcentral.com/scripts/184 that you’re testing the type of the pipeline object
Jaykul: yeah
user: FIleInfo vs. string
Jaykul: because it lets me do this: ls *.ps1 | Send-Paste
Jaykul: or this: get-history -count 5 | % { $_.CommandLine } | Send-Paste
Jaykul: or even send multiple files as a single paste, like: get-content *.ps1 | Send-Paste
Jaykul: (note that in this script, that’s VERY different than ls *.ps1 | Send-Paste …. which would create a paste for each file)
user: wow, this is good stuff, thanks for the time…

Incidentally, this script is also on PowerShellCentral scripts Repository where I will probably post any future modifications …

Every once in a while the question of how to best use the process block of a function to process pipeline objects comes up on IRC, and although I’m sure others have already written this up on the web in the past, we’ve been polishing up this example sending it back and forth to each other for a couple of weeks, and it seems to me it’s about time to publish it a little more prominently.

The basic idea was to write a function that could be used to process a set of inputs from either the pipeline or an argument, while still allowing other arguments to be passed and processed. This is something that’s very easy for a cmdlet, because you can specify that a certain parameter will receive pipeline input (and even control which attribute of the objects on the pipeline will be used), but in a script it’s not really supported. In addition, we require that the function process items as they come in, rather than waiting until it’s received all input before processing them.

Some explanation

This is just an example method, it doesn’t really do much, except that it outputs exactly the same thing whether you call it like Test-Pipeline -inputObject "Foo","Bar","Baz" "Kill Each Process" or like: ""Foo","Bar","Baz" | Test-Pipeline "Kill Each Process".

It works by calling itself with the inputObject argument passed on the pipeline when you specify it as an argument, so that code is processed exactly right. This means that the first line you see in the process block must remain the first line, even if you add additional code to the block.

We use a trick to make sure that additional unnamed parameters work normally by using a switch parameter -inputObject followed by a second parameter $io which we treat as inputObject if inputObject is set. This works because switch parameters don’t expect values, so we’re able to sort-of hijack it to emulate the cmdlet behavior. Note that if it is not set, we move the value of the $in parameter into the $args array which contains any additional parameters. This arrangement allows it to behave rather like a cmdlet would, but it’s a little delicate: if you’re going to pass input objects, you must specify the switch as the first argument, and pass the $io (inputObjects) immediately following (as though they were the value of the inputObjects parameter). All of that is necessary so that we can optionally pass additional parameters.

You can also add additional named parameters — but you would need to either always specify their names when you called them, or carefully adjust the code so that they get shifted if -inputObject isn’t specified.

Here’s some sample output, in case you’re wondering:

Credit where it’s due

I wanted to officially acknowledge that /\/\o\/\/ and Gaurhoth and Oisin and Brandon have all contributed to the polishing and testing of this script in various forms … thanks to all

The question was: why doesn’t this have any output? Well, the answer is: it can’t have any output. The filter outputs true or false for each item passed into it, but the where scriptblock doesn’t actually pass anything into the filter! Of course we quickly fixed his problem the right way by rewriting the filter to output the items, and getting rid of the where-object call completely … but when I started wondering where he had gotten the bizarre idea to use a filter like that, his answer was: “I read about_filter.” So I went and looked, and sure enough, the example pasted above is straight out of about_filter. And it’s as wrong as it can possibly be.

Let me fix it, and then offer some clarification below. In about_filter there is an example like this: Get-Process | where {$_.processname -like "[a-m]*"} and then a table describing each element of the Where-Object command, and just below it, there is some text about the filter command. Mentally replace that text with this:

You can create a predefined filter by creating a special type of function. When defining the function, you may specify that it is a filter. For the filter script block, you should output items if they pass the test you saw in the preceding example. For instance, the following command creates a filter named process-a-m:

filter process-a-m
    {
        if( $_.processname -like "[a-m]*" ) { Write-Output $_ }
    }

You can then use the filter name in place of your Where-Object command to retrieve the filtered data, as shown in the following example:

Get-Process | process-a-m

The values returned by the command are the same as they would have been had you specified the conditions using the Where-Object cmdlet as above, but the filter is much easier to reuse on multiple calls, particularly if the conditions were more complicated.

Of course, a filter function can take parameters, just like any other function, so you could make the range of starting letters more flexible:

filter process-range([char]$a="a",[char]$m="m")
    {
        if( $_.processname -like "[$a-$m]*" ) { Write-Output $_ }
    }

Now you can use this filter to select between any two letters quite easily:

Get-Process | process b j

Functions vs. Filters

To be clear, functions in PowerShell aren’t like functions in other programming languages, although they can just have a body (and be like other functions) they can instead have pipeline syntax. A function (just like a cmdlet) which is meant to be used in the pipeline has three separate bodies: begin, process, and end, but a filter has only the process part of the body.

Of course, there are other uses for filters than just limiting which objects come through … anything that you could do with foreach-object or where-object, you can basically do with a filter in a much more reusable way. So you don’t have to write the same thing to the output as what came in the input, you could actually add to it, or remove from it. A filter for processes could look use Win32_Process to look up their ‘owner’ and append it as a property, or could return the process path and name as text …

If you’re new to all of this, let me just present an example function to hopefully clear things up:

A function without an internal pipeline block is basically like the begin block of a pipeline-enabled function: the only things present are the parameters to the function (if there are any). Then, process is called once for each item in the pipeline, with the current item set as the $_ variable. Finally, end is called, and you still have the last item in the pipeline present. The scope persists through each part of the function (so you will see that $counter maintains it’s value). The important distinction between this and a function without a process body is that a normal function will only be called once when it is in a pipeline — regardless of how many things are in the pipeline. If you called DemoTheCount with a list of processes, like Get-Process g* | DemoTheCount, you would get something like this (depending on the processes running on your PC):

A filter doesn’t have separate parts, because the whole filter is the process part of the function. If we were to take the process portion of our DemoTheCount function and turn it into a filter, it would only work because the dynamic language would automatically initialize $counter to zero and let you increment it — anything more complex (or where the type couldn’t be deduced) would not. When you create a filter using the filter keyword, it’s pure syntax sugar. The filter process-a-m defined in my correction to the documentation is exactly the same as this function:

You can verify this by creating both of them and then comparing the contents of the resulting functions ((get-content function:\process-a-m) -eq (get-content function:\reprocess-a-m). The filter keyword simply creates a function and with the filter body as the process body for the function.