Should-BeNull: Strange behaviour when $null

[johlju]

Checklist

• Issue has a meaningful title
• I have searched the existing issues. See all issues
• I have tested using the latest version of Pester. See Installation and update guide.

What is the issue?

I'm getting that the $result passed to Should-BeNull is not $null, but the verbose output I added to debug this do say it is not an array and it is indeed $null. 🤔 Not sure if I'm doing something wrong. I have seen this in another place as well but ignored it there, when I got it here I thought this must be an error somewhere. 🙂

The Out-Diff currently only writes Write-Information messages and returns no values.
I output verbose message to determine that it is indeed $null and not an array, still Should-BeNull thinks it is.

It 'Should output to console' {
    $expected = @(
        'My String that is longer than actual'
    )
    $actual = @(
        'My string that is shorter'
    )

    $result = Out-Diff -ExpectedString $expected -ActualString $actual
    Write-Verbose ($result -is [array]) -Verbose
    Write-Verbose ($null -eq $result) -Verbose
    $result | Should-BeNull
}

VERBOSE: False
VERBOSE: True
[-] Out-Diff.When output is made as informational message.Should output to console 124ms (122ms|2ms)
 Expected $null, but got [Object[]] '@()'.
 at $result | Should-BeNull -Debug -Verbose, /Users/johlju/source/Viscalyx.Common/tests/Unit/Public/Out-Diff.tests.ps1:94

Expected Behavior

Should pass test as the actual value is $null.

Steps To Reproduce

Running this reproduces the issue:

BeforeAll {
    function Out-Something
    {
        param
        (
        )
    }
}

Describe 'Out-Something' {
    It 'Should not output anything' {
        $result = $null
        $result = Out-Something

        Write-Verbose ("Result is null: {0}" -f ($null -eq $result)) -Verbose
        Write-Verbose ("Result is array: {0}" -f ($result -is [array])) -Verbose

        $result | Should-BeNull
    }
}

Returns:

Starting discovery in 1 files.
Discovery found 1 tests in 101ms.
Running tests.
VERBOSE: Result is null: True
VERBOSE: Result is array: False
[-] Out-Something.Should not output anything 118ms (109ms|9ms)
 Expected $null, but got [Object[]] '@()'.
 at $result | Should-BeNull, /Users/johlju/source/Viscalyx.Common/tests/Unit/Public/Out-Something.tests.ps1:18

Describe your environment

Pester version     : 6.0.0-alpha4 /Users/johlju/source/Viscalyx.Common/output/RequiredModules/Pester/6.0.0/Pester.psm1  
PowerShell version : 7.4.4
OS version         : Unix 14.5.0

Possible Solution?

Hopefully I'm just doing something wrong. 🙂

[johlju]

If I actually return $null in the function Out-Something then it works. 🤔

BeforeAll {
    function Out-Something
    {
        param
        (
        )

        $null
    }
}

Describe 'Out-Something' {
    It 'Should not output anything' {
        $result = $null
        $result = Out-Something

        Write-Verbose ("Result is null: {0}" -f ($null -eq $result)) -Verbose
        Write-Verbose ("Result is array: {0}" -f ($result -is [array])) -Verbose

        $result | Should-BeNull
    }
}

Returns:

Starting discovery in 1 files.
Discovery found 1 tests in 11ms.
Running tests.
VERBOSE: Result is null: True
VERBOSE: Result is array: False
[+] /Users/johlju/source/Viscalyx.Common/tests/Unit/Public/Out-Something.tests.ps1 420ms (5ms|405ms)
Tests completed in 421ms
Tests Passed: 1, Failed: 0, Skipped: 0, Inconclusive: 0, NotRun: 0

[johlju]

For some reason $result is not actually $null in the reproducible example, although the verbose statements says it is - but it is not... 🤔

This doesn't seem to be a Pester issue, but probably how PowerShell works... I just don't understand why it behaves as it does. If someone could shed som light it would be appreciated. 🙂

[nohwnd]

Those are artifacts of using the pipeline, when you use pipeline we collect it by $input, this automatic variable is populated when you have parameter that has ValueFromPipeline. $input always contains an object[] which has 3 interesting states:

it is empty
it has one item
it has more than one item

When it is empty there was no input, we check for being called via pipeline (not in the function below, but in the real Collect-Input function in Pester), so the only case when that happens is when you get called with empty collection e.g. @().

When you pass $null or 1 or @(1) the array has 1 item, this allows us to distinguish null from empty collection, but we cannot distinguish single value wrapped in collection @(1) from single value not wrapped in collection 1.

When it has more items it there are no edge cases, it is simply a collection.

function empty ()  { }

function returnNull () { return $null }

function check () {
    param(
        [Parameter(Position = 1, ValueFromPipeline = $true)]
        $actual
    )

    "is null: $($null -eq $local:input)"    
    "has type: $($local:input.GetType().Name)"
    "count: $($local:input.Count)"
}

$null | check

"--- empty function:" 
empty | check

"--- empty array:"
@() | check

"--- null:"
$null | check

"--- return null:"
returnNull | check

"--- item:"
1 | check

"--- item in array:"
@(1) | check

--- empty function:
is null: False
has type: Object[]
count: 0
--- empty array:
is null: False
has type: Object[]
count: 0
--- null:
is null: False
has type: Object[]
count: 1
--- return null:
is null: False
has type: Object[]
count: 1
--- item:
is null: False
has type: Object[]
count: 1
--- item in array:
is null: False
has type: Object[]
count: 1

PowerShell also behaves differently when passing values in pipeline and when assigning to a variable. Which is how you are debugging this and why this is giving you different results:

function arr () { @() }; 
$e = arr
"is `$e null: $($null -eq $e)"

# BUT here we don't receive `$null` we receive empty
arr | check

I agree that this is unexpected, but is is how powershell binds the methods via pipeline and in assignments.

You will also observe similar difference when using the parameter syntax of Should-BeNull:

# this passes, the returned @() is expanded to null, same as when we assign
Should-BeNull -Actual (arr)

[nohwnd]

Now that is the technical reasoning, but how do we make it usable in practice?