How to group by all properties in PowerShell?

Question

I'm planning to remove duplicate firewall rules on Windows. I know there are a set of firewall cmdlets like get-netfirewallrule, but it's too slow when I try to pass a firewall rule to get additional information like Get-NetFirewallRule -displayname xxx | Get-NetFirewallApplicationFilter. So I plan to use the old-school way, the netsh advfirewall firewall command set.

Now I've parsed the output of netsh advfirewall firewall as an array of objects.

$output = netsh advfirewall firewall show rule name=all verbose | Out-String
$output = [regex]::split($output.trim(), "\r?\n\s*\r?\n");

$objects = @(foreach($section in $output) {
    $obj = [PSCustomObject]@{}

    foreach($line in $($section -split '\r?\n')) {
        if($line -match '^\-+$') {
            continue
        }
        $name, $value = $line -split ':\s*', 2
        $name = $name -replace " ", ""
        
        $obj | Add-Member -MemberType NoteProperty -Name $name -Value $value
    }

    $obj
})

But the problem is that the objects in the array have different properties. For example, this is one object:

RuleName       : HNS Container Networking - ICS DNS (TCP-In) - B15BF139-F18D-471C-A18C-92DFD33350F1 - 0
Description    : HNS Container Networking - ICS DNS (TCP-In) - B15BF139-F18D-471C-A18C-92DFD33350F1 - 0
Enabled        : Yes
Direction      : In
Profiles       : Domain,Private,Public
Grouping       :
LocalIP        : Any
RemoteIP       : Any
Protocol       : TCP
LocalPort      : 53
RemotePort     : Any
Edgetraversal  : No
Program        : C:\WINDOWS\system32\svchost.exe
Service        : sharedaccess
InterfaceTypes : Any
Security       : NotRequired
Rulesource     : Local Setting
Action         : Allow

This is another one:

RuleName       : HNS Container Networking - DNS (UDP-In) - 91EC1DEF-8CB8-4C2A-A6D4-91480448AE97 - 0
Description    : HNS Container Networking - DNS (UDP-In) - 91EC1DEF-8CB8-4C2A-A6D4-91480448AE97 - 0
Enabled        : Yes
Direction      : In
Profiles       : Domain,Private,Public
Grouping       :
LocalIP        : Any
RemoteIP       : Any
Protocol       : UDP
LocalPort      : 53
RemotePort     : Any
Edgetraversal  : No
InterfaceTypes : Any
Security       : NotRequired
Rulesource     : Local Setting
Action         : Allow

As you can see, the first rule use ports as well as program to define the rule and the second only use ports. How can I group the objects and find duplicated items (so that I can remove corresponding firewall rules)? I can't simply use $objects | Group-Object -Property rulename,enabled,....., because the properties are not the same for all objects.

Btw, the duplicate firewall rules are created by some imperfect script I written in the past.

Answer 1

You have learned (or remembered) the hard way of parsing command output before PowerShell was a thing 😛 But you've done the bulk of the work, we just need a few modifications to your attempt to get common properties on all of the returned objects:

$output = ( netsh advfirewall firewall show rule name=all verbose |
  Out-String ).Trim() -split '\r?\n\s*\r?\n'
$propertyNames = [System.Collections.Generic.List[string]]::new()

$objects = @( $(foreach($section in $output ) {
    $obj = @{}

    foreach( $line in ($section -split '\r?\n') ) {
        if( $line -match '^\-+$' ) {
            continue
        }
        $name, $value = $line -split ':\s*', 2
        $name = $name -replace " ", ""
        
        $obj.$name  = $value
        if( $propertyNames -notcontains $name ) {
            $propertyNames.Add( $name )
        }
    }
    
    $obj
}) | ForEach-Object {
    foreach( $prop in $propertyNames ) {
        if( $_.Keys -notcontains $prop ) {
            $_.$prop = $null
        }
    }
    [PSCustomObject]$_
})

---

I'll explain the adjustments I made to your code:

• Set the $output in one line (made multiline for readability here). Not strictly necessary.
• Use the -split operator instead of [regex]::split() for consistency with the rest of your code.
• Created a list called $propertyNames. This will be used to track all property names that have been read from your output.
• Sub-express $() your first foreach loop to prevent needing an intermediate variable and let us pass the foreach output down the pipeline. This is necessary at the end. It must be a sub-expression and not just surrounded by ().
• Make $obj a hashtable instead of a PSCustomObject. It's easier to work with while modifying the object, and convertable to PSCustomObject at the end.
• Add-Member is no longer required since we have a hashtable, so $obj.$name = $value will suffice for setting the property name and value.
  • Note: If you apply this portion of the answer in other areas, watch for pitfalls if the property name is anything other than a string. Alternatively, you can use traditional array-accessor syntax if this is a concern: $obj[$name] = $value.
• If $name is not in the known property names tracked in $propertyList, add it. Once the foreach section loop is complete, we will have all of the possible properties returned by your netsh command.
• Now we can return $obj for each section of the netsh output. We still might need to change this so don't turn it into a PSCustomObject just yet.
• Pipe the output of your sub-expressed foreach loop to ForEach-Object. Again, this prevents the need for an intermediate variable. We need to loop over every $obj because....
• Within ForEach-Object, loop over the $propertyList and check that every known property is present as a key on each $obj (represented as $_ or $PSitem within the ScriptBlock. If it does not exist, add it with a value of $null.
• Lastly, convert $_ (which will be each $obj returned from the first loop) to a PSCustomItem as we return it.

Here's a diff of your code vs. my changes:

Now, every returned object should have the same properties on it, even if they didn't show in the original output for that rule.