Comments on: PowerShell needs Shift operators … http://huddledmasses.org/powershell-needs-shift-operators/ You can do more than breathe for free... Sun, 14 Mar 2010 05:17:19 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: Klaus Schulte http://huddledmasses.org/powershell-needs-shift-operators/comment-page-1/#comment-212303 Klaus Schulte Tue, 03 Mar 2009 15:39:08 +0000 http://huddledmasses.org/?p=1094#comment-212303 Excellent! I did already add a snapin ( My first SnapIn !!! ) to powershell with exactly that functionality but only using the type long int. That's a nice excercise ... but the "Add-Type" cmdlet makes these things SO MUCH easier ... that's brilliant!!! I should go now and buy me the T-Shirt with the usual red heart, the "PS>_" and the "I love the PS team" on it! best regards, Klaus Excellent!
I did already add a snapin ( My first SnapIn !!! ) to powershell with exactly that functionality but only using the type long int. That’s a nice excercise … but the “Add-Type” cmdlet makes these things SO MUCH easier … that’s brilliant!!!

I should go now and buy me the T-Shirt with the usual red heart, the “PS>_” and the “I love the PS team” on it!

best regards, Klaus

]]> By: Joel 'Jaykul' Bennett http://huddledmasses.org/powershell-needs-shift-operators/comment-page-1/#comment-212268 Joel 'Jaykul' Bennett Sat, 28 Feb 2009 02:46:56 +0000 http://huddledmasses.org/?p=1094#comment-212268 Wow, nice work :) The speed is only a hair off the Add-Type one I wrote, as far as I can tell ... but yeah, that's about as complicated as things come. I think either one would work, but yours is a lot easier to use on PowerShell v1, since you wouldn't need the "New-Type script to stand in for Add-Type":http://poshcode.org/720 on top of the script I presented. There is one _other_ reason to use the Add-Type version, of course: when you need to call the actual "shift operators":http://msdn.microsoft.com/en-us/library/aa691377(VS.71).aspx on a type that overrides them with a custom implementation. Wow, nice work :)

The speed is only a hair off the Add-Type one I wrote, as far as I can tell … but yeah, that’s about as complicated as things come. I think either one would work, but yours is a lot easier to use on PowerShell v1, since you wouldn’t need the New-Type script to stand in for Add-Type on top of the script I presented.

There is one other reason to use the Add-Type version, of course: when you need to call the actual shift operators).aspx on a type that overrides them with a custom implementation.

]]> By: Ed Withers http://huddledmasses.org/powershell-needs-shift-operators/comment-page-1/#comment-212260 Ed Withers Fri, 27 Feb 2009 21:07:32 +0000 http://huddledmasses.org/?p=1094#comment-212260 Hi Joel, Ok, the more I think about it, the more I agree with your original premise, we need shift operators. Actually, you really caught my attention when you pointed out that PowerShell doesn't do "integer math", so I had to do some playing around, and you know how that can lead into strange directions, (like how Int16 behaves differently than Int32 when you try to assign values with the high bit set.) To make a long story short, I decided to play with the bitwise operators and see if I could do the bit truncation needed to make my original multiply/divide trick work. It's a lot more work than it's worth. However, I finally hacked something together that I think is pretty stable and works quickly for any value in a [byte], [Int16], [Int32], or [Int64] regardless of the starting value or number of bits to shift. Here's my code. I didn't try to make it pipeline friendly, maybe I'll save that for another day. I did combine both left and right shift into a single function with -left and -right switches... <code lang="posh"> function Shift-Bit { ###################################################################### ## ## Shift-Bit ## ## Accepts an input value in any of the integer types (byte, Int16, ## Int32, and Int64) and the number of bits to shift it to the left ## ## Input: ## $x - Value in any of the integer types ## (byte, Int16, Int32, and Int64) ## [Defaults to 1] ## ## $y - Number of bits to shift between 1 and bitsize of $x -1 ## ## $left - Switch to do left shift ## $right- Switch to do right shift ## [Default is right, if both are set, right takes priority] ## ## v1 B. Edward Withers 27-Feb-2009 ## Initial code ## ####################################################################### param ($x = 1, $y, [switch] $left, [switch] $right) # PowerShell does some strange casting of Int16, so we have to handle # it specially. [bool] $flagInt16 = $false # Only process if the number of bits to shift is valid if ($y -gt 0) { # Check the input variable type Switch ($x.gettype().fullname) { System.Byte { $maxbits = 8 [byte] $highbit = 0x80 [byte] $allbits = 0xFF [byte] $maskclear = 1 [byte] $maskhigh = 1 [byte] $maskrest = 1 } System.Int16 { $maxbits = 16 [int16] $highbit = -1 * 0x8000 # Int16 behaves differently [int16] $allbits = -1 [int16] $maskclear = 1 [int16] $maskhigh = 1 [int16] $maskrest = 1 [bool] $flagInt16 = $true } System.Int32 { $maxbits = 32 [int32] $highbit = 0x80000000 [int32] $allbits = -1 [int32] $maskclear = 1 [int32] $maskhigh = 1 [int32] $maskrest = 1 } System.Int64 { $maxbits = 64 [int64] $highbit = 0x8000000000000000 [int64] $allbits = -1 [int64] $maskclear = 1 [int64] $maskhigh = 1 [int64] $maskrest = 1 } # For any other variable type, alert the user and exit default { "Cannot process" + ($x.gettype().fullname) + " variables in a bitwise fashion." return } } # End of Switch # Only process if the shift distance is valid (less than variable length) if ($y -lt $maxbits) { # Check the shift direction, default to right, and if both are set, # give precedence to right if ($left.IsPresent -and (-not $right.IsPresent)) { # create a bitmask for just the bits we want to keep for ($count=1; $count -le $maxbits - $y -1; $count += 1) { $maskhigh = $maskhigh * 2 # a mask with only the high bit set $maskrest = $maskrest * 2 + 1 # a mask with all kept bits set } # Now do some mask magic to get the high bits and the rest of the bits # separated into variables $maskrest = $maskrest -bxor $maskhigh # clear the high bit from $maskrest $maskrest = $maskrest -band $x # then get just those bits from input $maskhigh = $maskhigh -band $x # and get the high kept bit from input # Now shift left by multiplying by 2 for ($count=1; $count -le $y; $count += 1) { $maskrest = $maskrest * 2 } # Set the low-order bits of the return value $x = $maskrest # if the high input bit was set, then set it on output if ($maskhigh -ne 0) { $x = $highbit -bor $x } } else { # Assume we want a right shift if Left wasn't selected # create a bitmask for the bits we will be losing # (this prevents underflow and conversion into a floating point # format) for ($count=1; $count -le $y -1; $count += 1) { $maskclear = $maskclear * 2 + 1 # a mask of bits to clear } # Invert the mask so the bits to clear are zero $maskclear = $maskclear -bxor $allbits # Reduce the input to just the bits we want $maskclear = $maskclear -band $x # Now shift to the right by dividing by 2 for ($count=1; $count -le $y; $count += 1) { $maskclear = $maskclear / 2 } # Set the return value $x = $maskclear } } else { "Cannot shift a " + ($x.gettype().fullname) + " by $y bits." } } # End of check for shift length # Handle the special case by forcing the output to be cast back if ($flagInt16) {[Int16]$x = $x} # Output the result $x } # End of Function</code> Hi Joel,

Ok, the more I think about it, the more I agree with your original premise, we need shift operators.

Actually, you really caught my attention when you pointed out that PowerShell doesn’t do “integer math”, so I had to do some playing around, and you know how that can lead into strange directions, (like how Int16 behaves differently than Int32 when you try to assign values with the high bit set.)

To make a long story short, I decided to play with the bitwise operators and see if I could do the bit truncation needed to make my original multiply/divide trick work. It’s a lot more work than it’s worth. However, I finally hacked something together that I think is pretty stable and works quickly for any value in a [byte], [Int16], [Int32], or [Int64] regardless of the starting value or number of bits to shift.

Here’s my code. I didn’t try to make it pipeline friendly, maybe I’ll save that for another day. I did combine both left and right shift into a single function with -left and -right switches…

function Shift-Bit {
######################################################################
##
##   Shift-Bit
##
##   Accepts an input value in any of the integer types (byte, Int16,
##       Int32, and Int64) and the number of bits to shift it to the left
##
##   Input:
##       $x -    Value in any of the integer types
##               (byte, Int16, Int32, and Int64)
##               [Defaults to 1]
##
##       $y -    Number of bits to shift between 1 and bitsize of $x -1
##
##       $left - Switch to do left shift
##       $right- Switch to do right shift
##               [Default is right, if both are set, right takes priority]
##
##   v1  B. Edward Withers 27-Feb-2009
##       Initial code
##
#######################################################################
    param ($x = 1, $y, [switch] $left, [switch] $right)

    # PowerShell does some strange casting of Int16, so we have to handle
    # it specially.
    [bool] $flagInt16 = $false

    # Only process if the number of bits to shift is valid
    if ($y -gt 0) {

        # Check the input variable type
        Switch ($x.gettype().fullname) {
            System.Byte {
                $maxbits = 8
                [byte] $highbit = 0x80
                [byte] $allbits = 0xFF
                [byte] $maskclear = 1
                [byte] $maskhigh = 1
                [byte] $maskrest = 1
            }
            System.Int16 {
                $maxbits = 16
                [int16] $highbit = -1 * 0x8000  # Int16 behaves differently
                [int16] $allbits = -1
                [int16] $maskclear = 1
                [int16] $maskhigh = 1
                [int16] $maskrest = 1
                [bool] $flagInt16 = $true
            }
            System.Int32 {
                $maxbits = 32
                [int32] $highbit = 0x80000000
                [int32] $allbits = -1
                [int32] $maskclear = 1
                [int32] $maskhigh = 1
                [int32] $maskrest = 1
            }
            System.Int64 {
                $maxbits = 64
                [int64] $highbit = 0x8000000000000000
                [int64] $allbits = -1
                [int64] $maskclear = 1
                [int64] $maskhigh = 1
                [int64] $maskrest = 1
            }
            # For any other variable type, alert the user and exit
            default {
                "Cannot process" + ($x.gettype().fullname) + " variables in a bitwise fashion."
                return
            }
        } # End of Switch

        # Only process if the shift distance is valid (less than variable length)
        if ($y -lt $maxbits) {

            # Check the shift direction, default to right, and if both are set,
            # give precedence to right
            if ($left.IsPresent -and (-not $right.IsPresent)) {

                # create a bitmask for just the bits we want to keep
                for ($count=1; $count -le $maxbits - $y -1; $count += 1) {
                    $maskhigh = $maskhigh * 2 # a mask with only the high bit set
                    $maskrest = $maskrest * 2 + 1 # a mask with all kept bits set
                }

                # Now do some mask magic to get the high bits and the rest of the bits
                # separated into variables
                $maskrest = $maskrest -bxor $maskhigh # clear the high bit from $maskrest
                $maskrest = $maskrest -band $x # then get just those bits from input
                $maskhigh = $maskhigh -band $x # and get the high kept bit from input

                # Now shift left by multiplying by 2
                for ($count=1; $count -le $y; $count += 1) {
                    $maskrest = $maskrest * 2
                }

                # Set the low-order bits of the return value
                $x = $maskrest

                # if the high input bit was set, then set it on output
                if ($maskhigh -ne 0) {
                    $x = $highbit -bor $x
                }
             }
             else { # Assume we want a right shift if Left wasn't selected
                # create a bitmask for the bits we will be losing
                # (this prevents underflow and conversion into a floating point
                # format)
                for ($count=1; $count -le $y -1; $count += 1) {
                    $maskclear = $maskclear * 2 + 1 # a mask of bits to clear
                }
                # Invert the mask so the bits to clear are zero
                $maskclear = $maskclear -bxor $allbits
                # Reduce the input to just the bits we want
                $maskclear = $maskclear -band $x
                # Now shift to the right by dividing by 2
                for ($count=1; $count -le $y; $count += 1) {
                    $maskclear = $maskclear / 2
                }
               
                # Set the return value
                $x = $maskclear                
             }
        }
        else {
            "Cannot shift a " + ($x.gettype().fullname) + " by $y bits."
        }
    } # End of check for shift length

    # Handle the special case by forcing the output to be cast back
    if ($flagInt16) {[Int16]$x = $x}

    # Output the result
    $x

} # End of Function

]]> By: Joel 'Jaykul' Bennett http://huddledmasses.org/powershell-needs-shift-operators/comment-page-1/#comment-212145 Joel 'Jaykul' Bennett Mon, 23 Feb 2009 21:23:28 +0000 http://huddledmasses.org/?p=1094#comment-212145 Well, no, using math wouldn't be easier ... because PowerShell doesn't seem to have integer multiplication, so your version will incorrectly overflow into a double. Lets call yours Shift-LeftMath, and try this: <code lang="posh"> [1]: $x = get-random; $x 1766273347 [2]: Shift-Left $x 1 -762420602 [3]: Shift-LeftMath $x 1 3532546694 </code> Of course, the same thing happens in the other direction: <code lang="posh"> [4]: Shift-RightMath 42 4 2.625</code> You can fix the shift-right easily by using either @[Math]::DivRem@ or @[Math]::Floor@ ... but shift-left is trickier, because you have to detect the overflow and truncate bits, or else you're not doing the same left shift that you would be doing in C# (or any other programming language?). If I thought through all of that (as I just did, thanks to you), I could have written something like this (notice I made these pipeline capable the same as the originals posted above): <code lang="posh"> function Shift-LeftMath { Param ($x =1, $y) BEGIN { if($y) { $result = $x (1..$y)|%{$result = [Math]::Floor($result * 2)} while($result -gt [int]::MaxValue) { $result = $result - [int]::MaxValue - [int]::MaxValue - 2 } $result } } PROCESS { if($_){ $result,$y = $_,$x (1..$y)|%{$result = [Math]::Floor($result * 2)} while($result -gt [int]::MaxValue) { $result = $result - [int]::MaxValue - [int]::MaxValue - 2 } Write-Host $result } } } function Shift-RightMath { Param ($x =1, $y) BEGIN { if($y) { $result = $x (1..$y)|%{$result = [Math]::Floor($result / 2)} $result } } PROCESS { if($_){ $result,$y = $_,$x (1..$y)|%{$result = [Math]::Floor($result / 2)} $result } } }</code> But after all of that, the math version still takes many times as long to run, even for the Shift-Right. The Shift-Left can take a _very_ long time, because the solution I came up with for rounding down involves iterating in a while loop and if you shifted a large number 16 bits left it can take thousands of loops to come clean. I'm sure there's a better way than that, but I can't think of one right now. Anyway, here's a comparison of the relatively fast Shift-Right, doing the task I started out trying to do -- get the high-order word: <code lang="posh"> $val = [System.Windows.Media.RenderCapability]::Tier 1..1000 | %{ $x = Shift-Right $val 16 }; $x 1..1000 | %{ $x = Shift-RightMath $val 16 }; $x </code> notextile. <code><pre> Duration Average Commmand -------- ------- -------- 0.23438 0.00023 1..1000 | %{ $x = Shift-Right $val 16 }; $x 1.65625 0.00166 1..1000 | %{ $x = Shift-RightMath $val 16 }; $x </pre></code> Obviously the actual time is tiny, if you were doing thousands of these, you'd regret it -- and as I said before, I can't find a way to make the mathematical left shift do the right thing when it overflows that doesn't take at least half a second for a simple @Shift-LeftMath 0x42000001 16@: notextile. <code><pre> Duration Average Commmand -------- ------- -------- 0.43750 0.43750 Shift-LeftMath 0x42000001 16 0.00000 0.00000 Shift-Left 0x42000001 16 </pre></code> Well, no, using math wouldn’t be easier … because PowerShell doesn’t seem to have integer multiplication, so your version will incorrectly overflow into a double. Lets call yours Shift-LeftMath, and try this:

[1]: $x = get-random; $x
1766273347
[2]: Shift-Left $x 1
-762420602
[3]: Shift-LeftMath $x 1
3532546694  

Of course, the same thing happens in the other direction:

[4]: Shift-RightMath 42 4
2.625

You can fix the shift-right easily by using either [Math]::DivRem or [Math]::Floor … but shift-left is trickier, because you have to detect the overflow and truncate bits, or else you’re not doing the same left shift that you would be doing in C# (or any other programming language?). If I thought through all of that (as I just did, thanks to you), I could have written something like this (notice I made these pipeline capable the same as the originals posted above):

function Shift-LeftMath {
Param ($x =1, $y)
BEGIN {
   if($y) {
      $result = $x
      (1..$y)|%{$result = [Math]::Floor($result * 2)}
      while($result -gt [int]::MaxValue) {
         $result = $result - [int]::MaxValue - [int]::MaxValue - 2  
      }
      $result
   }
}
PROCESS {
   if($_){
      $result,$y = $_,$x
      (1..$y)|%{$result = [Math]::Floor($result * 2)}
      while($result -gt [int]::MaxValue) {
         $result = $result - [int]::MaxValue - [int]::MaxValue - 2  
      }
      Write-Host
      $result
   }
}
}

function Shift-RightMath {
Param ($x =1, $y)
BEGIN {
   if($y) {
      $result = $x
      (1..$y)|%{$result = [Math]::Floor($result / 2)}
      $result
   }
}
PROCESS {
   if($_){
      $result,$y = $_,$x
      (1..$y)|%{$result = [Math]::Floor($result / 2)}
      $result
   }
}
}

But after all of that, the math version still takes many times as long to run, even for the Shift-Right. The Shift-Left can take a very long time, because the solution I came up with for rounding down involves iterating in a while loop and if you shifted a large number 16 bits left it can take thousands of loops to come clean. I’m sure there’s a better way than that, but I can’t think of one right now. Anyway, here’s a comparison of the relatively fast Shift-Right, doing the task I started out trying to do — get the high-order word:

$val = [System.Windows.Media.RenderCapability]::Tier
1..1000 | %{ $x = Shift-Right $val 16 }; $x
1..1000 | %{ $x = Shift-RightMath $val 16 }; $x
 

Duration Average Commmand
-------- ------- --------
 0.23438 0.00023 1..1000 | %{ $x = Shift-Right $val 16 }; $x
 1.65625 0.00166 1..1000 | %{ $x = Shift-RightMath $val 16 }; $x

Obviously the actual time is tiny, if you were doing thousands of these, you’d regret it — and as I said before, I can’t find a way to make the mathematical left shift do the right thing when it overflows that doesn’t take at least half a second for a simple Shift-LeftMath 0x42000001 16:

Duration Average Commmand
-------- ------- --------
 0.43750 0.43750 Shift-LeftMath 0x42000001 16
 0.00000 0.00000 Shift-Left 0x42000001 16

]]> By: Ed Withers http://huddledmasses.org/powershell-needs-shift-operators/comment-page-1/#comment-212139 Ed Withers Mon, 23 Feb 2009 17:21:08 +0000 http://huddledmasses.org/?p=1094#comment-212139 Ok, I take the point, and I'll grant that Add-Type is cool, but wouldn't it have been easier to just: <code lang="posh"> function shift-left { Param ($x =1, $y) (1..$y)|%{$x = $x * 2} $x }</code> (and divide for shift-right)? Ok, I take the point, and I’ll grant that Add-Type is cool, but wouldn’t it have been easier to just:

function shift-left {
  Param ($x =1, $y)
  (1..$y)|%{$x = $x * 2}
  $x
}

(and divide for shift-right)?

]]> By: Joel 'Jaykul' Bennett http://huddledmasses.org/powershell-needs-shift-operators/comment-page-1/#comment-212122 Joel 'Jaykul' Bennett Mon, 23 Feb 2009 04:58:53 +0000 http://huddledmasses.org/?p=1094#comment-212122 Add-Type is awesome, but it would be really cool if we could add more operators ourselves ... how is it that shift operators aren't in, but -split and -join are *operators*? Add-Type is awesome, but it would be really cool if we could add more operators ourselves … how is it that shift operators aren’t in, but -split and -join are operators?

]]> By: Jeffrey Snover http://huddledmasses.org/powershell-needs-shift-operators/comment-page-1/#comment-212121 Jeffrey Snover Mon, 23 Feb 2009 04:31:06 +0000 http://huddledmasses.org/?p=1094#comment-212121 Boy I'm glad we added Add-Type to V2! Boy I’m glad we added Add-Type to V2!

]]>