Multiple outputs to same csv file in Powershell

Question

I'm trying to export my firewall rules that are specified in multiple group policy objects and would like to include the Name of the GPO in the exported file. So I tried to take my string variable $Policy and jam it into the csv file each time a new gpo is parsed but all I'm getting is the gpo name and not the fields from Get-NetFirewallRule. Of course if I remove the $policy | Out-File $env:temp\gpos.csv -Append -Force line then I get all of the fields from Get-NetFirewallRule - but they're all on a large csv file and I can't determine their source GPO.

foreach ($policy in $PolicyObjects) {
$GPO = Open-NetGPO -PolicyStore "contoso.com\$policy"
$policy | Out-File $env:temp\gpos.csv  -Append -Force
Get-NetFirewallRule -GPOSession $GPO  | 
Select Name,
DisplayName,
DisplayGroup,
@{Name='Protocol';Expression={($PSItem | Get-NetFirewallPortFilter).Protocol}},
@{Name='LocalPort';Expression={($PSItem | Get-NetFirewallPortFilter).LocalPort}},
@{Name='RemotePort';Expression={($PSItem | Get-NetFirewallPortFilter).RemotePort}},
@{Name='RemoteAddress';Expression={($PSItem | Get-NetFirewallAddressFilter).RemoteAddress}},
Enabled,
Profile,
Direction,
Action | Export-CSV $env:temp\gpos.csv -Append -Force
}
Start-Process notepad $env:temp\gpos.csv 

Answer 1

It's not possible to append data with Export-Csv when the target file does not match the structure.

In your case, Export-Csv tried to append values from column $policy, which doesn't exist. Therefore it added $null.

For a valid CSV file you have to combine everything into one object before using Export-Csv.

Try the below approach, which should work much faster, since it does not use Get-NetFirewallAddressFilter several times for the same item, but only once for per GPO-Session.

Unfortunately I can't test it with a GPO-Session, but since the Parameter -GPOSession exists for Get-NetFirewallPortFilter and Get-NetFirewallAddressFilter as well, it may work. If you want to test it just locally, you just need to remove the outer foreach-loop and all GPOSession related commands and parameters. Open-NetGPO and -GPOSession

Comparison between my approach and retrieving the Port- and Address-Filter for each rule separately (even when only once per rule) when building a report for local firewall rules:

My approach: < 2 seconds
Other : > 3 minutes (using Get-NetFirwall*Filter for each rule once)

The code requires an elevated console!

#requires -RunAsAdministrator
$ruleReport = [System.Collections.Generic.List[psobject]]::new()
foreach ($policy in $PolicyObjects) {
    <#
        if you want to append to file after each policy, place the list here instead of above
        $ruleReport = [System.Collections.Generic.List[psobject]]::new()
    #>

    $GPO   = Open-NetGPO -PolicyStore "contoso.com\$policy"
    $rules = Get-NetFirewallRule -GPOSession $GPO

    # retrieving filters once in advance speeds up the execution extremely! (Requires elevated session to get all filters)
    # local execution time with that approach reduces the execution time from   >3 minutes  TO   <2 seconds
    $portFilters    = Get-NetFirewallPortFilter -All -GPOSession $GPO
    $addressFilters = Get-NetFirewallAddressFilter -All -GPOSession $GPO

    # build dictionary for port filters for rule lookups
    $portFilterDict    = [System.Collections.Generic.Dictionary[[string], [ciminstance]]]::new()
    foreach ($portFilter in $portFilters) {
        $portFilterDict.Add($portFilter.InstanceID, $portFilter)
    }

    # build dictionary for addresss filters for rule lookups
    $addressFilterDict = [System.Collections.Generic.Dictionary[[string], [ciminstance]]]::new()
    foreach ($addressFilter in $addressFilters) {
        $addressFilterDict.Add($addressFilter.InstanceID, $addressFilter)
    }


    foreach ($rule in $rules) {
        $ruleInstanceId    = $rule.InstanceID
        $rulePortFilter    = $portFilterDict[$ruleInstanceId]
        $ruleAddressFilter = $addressFilterDict[$ruleInstanceId]

        # build combined object
        $ruleReportItem = [pscustomobject][ordered]@{
            Policy        = $policy
            Name          = $rule.Name
            DisplayName   = $rule.DisplayName
            DisplayGroup  = $rule.DisplayGroup
            Protocol      = $rulePortFilter.Protocol
            LocalPort     = $rulePortFilter.LocalPort
            RemotePort    = $rulePortFilter.RemotePort
            RemoteAddress = $ruleAddressFilter.RemoteAddress
            Enabled       = $rule.Enabled
            Profile       = $rule.Profiles
            Direction     = $rule.Direction
            Action        = $rule.Action
        }

        # append to list
        $ruleReport.Add($ruleReportItem)
    }

    <#
        if you want to append to file after each policy, place the export here instead of below
        $ruleReport | Export-Csv $env:temp\gpos.csv -Append
    #>
}

$ruleReport | Export-Csv $env:temp\gpos.csv

Local Firewall Rules

To test the approach just for local Firewall Rules, use this script.
This script adds all available information of each rule and requires an elevated console.

#requires -RunAsAdministrator
$ruleReport = [System.Collections.Generic.List[psobject]]::new()

$rules                = Get-NetFirewallRule
$portFilters          = Get-NetFirewallPortFilter -All
$addressFilters       = Get-NetFirewallAddressFilter -All

$applicationFilters   = Get-NetFirewallApplicationFilter -All
$interfaceFilters     = Get-NetFirewallInterfaceFilter -All
$interfaceTypeFilters = Get-NetFirewallInterfaceTypeFilter -All
$securityFilters      = Get-NetFirewallSecurityFilter -All
$serviceFilters       = Get-NetFirewallServiceFilter -All


$portFilterDict = [System.Collections.Generic.Dictionary[[string], [ciminstance]]]::new()
foreach ($filter in $portFilters) {
    $portFilterDict.Add($filter.InstanceID, $filter)
}

$addressFilterDict = [System.Collections.Generic.Dictionary[[string], [ciminstance]]]::new()
foreach ($filter in $addressFilters) {
    $addressFilterDict.Add($filter.InstanceID, $filter)
}

$applicationFiltersDict = [System.Collections.Generic.Dictionary[[string], [ciminstance]]]::new()
foreach ($filter in $applicationFilters) {
    $applicationFiltersDict.Add($filter.InstanceID, $filter)
}

$interfaceFiltersDict = [System.Collections.Generic.Dictionary[[string], [ciminstance]]]::new()
foreach ($filter in $interfaceFilters) {
    $interfaceFiltersDict.Add($filter.InstanceID, $filter)
}


$interfaceTypeFiltersDict = [System.Collections.Generic.Dictionary[[string], [ciminstance]]]::new()
foreach ($filter in $interfaceTypeFilters) {
    $interfaceTypeFiltersDict.Add($filter.InstanceID, $filter)
}


$securityFiltersDict = [System.Collections.Generic.Dictionary[[string], [ciminstance]]]::new()
foreach ($filter in $securityFilters) {
    $securityFiltersDict.Add($filter.InstanceID, $filter)
}


$serviceFiltersDict = [System.Collections.Generic.Dictionary[[string], [ciminstance]]]::new()
foreach ($filter in $serviceFilters) {
    $serviceFiltersDict.Add($filter.InstanceID, $filter)
}




$cimClassProperties = 'CimClass', 'CimInstanceProperties', 'CimSystemProperties'
$cimClassPropertiesLookup = [System.Collections.Generic.HashSet[string]]::new([System.StringComparer]::OrdinalIgnoreCase)
foreach ($cimClassProperty in $cimClassProperties) {
    [void] $cimClassPropertiesLookup.Add($cimClassProperty)
}


foreach ($rule in $rules) {
    $ruleInstanceId      = $rule.InstanceID
    $portFilter          = $portFilterDict[$ruleInstanceId]
    $addressFilter       = $addressFilterDict[$ruleInstanceId]
    $applicationFilter   = $applicationFiltersDict[$ruleInstanceId]
    $interfaceFilter     = $interfaceFiltersDict[$ruleInstanceId]
    $interfaceTypeFilter = $interfaceTypeFiltersDict[$ruleInstanceId]
    $securityFilter      = $securityFiltersDict[$ruleInstanceId]
    $serviceFilter       = $serviceFiltersDict[$ruleInstanceId]

    $ruleReportItemProps = [ordered]@{
        RuleDisplayName            = $rule.DisplayName
        RuleDescription            = $rule.Description
        RuleEnabled                = $rule.Enabled
        RuleProfile                = $rule.Profile
        RuleAction                 = $rule.Action
        RuleDirection              = $rule.Direction
        PortFilterProtocol         = $rulePortFilter.Protocol
        PortFilterLocalPort        = $rulePortFilter.LocalPort
        PortFilterRemotePort       = $rulePortFilter.RemotePort
        AddressFilterRemoteAddress = $ruleAddressFilter.RemoteAddress
        ApplicationFilterProgram   = $applicationFilter.Program
    }

    foreach ($prop in $rule.psobject.Properties) {
        $propName = 'Rule' + $prop.Name
        if (-not $ruleReportItemProps.Contains($propName ) -and -not $cimClassPropertiesLookup.Contains($prop.Name)) {
            $ruleReportItemProps.Add($propName , $prop.Value)
        }
    }

    foreach ($prop in $portFilter.psobject.Properties) {
        $propName = 'PortFilter' + $prop.Name
        if (-not $ruleReportItemProps.Contains($propName) -and -not $cimClassPropertiesLookup.Contains($prop.Name)) {
            $ruleReportItemProps.Add($propName, $prop.Value)
        }
    }

    foreach ($prop in $addressFilter.psobject.Properties) {
        $propName = 'AddressFilter' + $prop.Name
        if (-not $ruleReportItemProps.Contains($propName) -and -not $cimClassPropertiesLookup.Contains($prop.Name)) {
            $ruleReportItemProps.Add($propName, $prop.Value)
        }
    }

    foreach ($prop in $applicationFilter.psobject.Properties) {
        $propName = 'ApplicationFilter' + $prop.Name
        if (-not $ruleReportItemProps.Contains($propName) -and -not $cimClassPropertiesLookup.Contains($prop.Name)) {
            $ruleReportItemProps.Add($propName, $prop.Value)
        }
    }

    foreach ($prop in $interfaceFilter.psobject.Properties) {
        $propName = 'InterfaceFilter' + $prop.Name
        if (-not $ruleReportItemProps.Contains($propName) -and -not $cimClassPropertiesLookup.Contains($prop.Name)) {
            $ruleReportItemProps.Add($propName, $prop.Value)
        }
    }

    foreach ($prop in $interfaceTypeFilter.psobject.Properties) {
        $propName = 'InterfaceTypeFilter' + $prop.Name
        if (-not $ruleReportItemProps.Contains($propName) -and -not $cimClassPropertiesLookup.Contains($prop.Name)) {
            $ruleReportItemProps.Add($propName, $prop.Value)
        }
    }

    foreach ($prop in $securityFilter.psobject.Properties) {
        $propName = 'SecurityFilter' + $prop.Name
        if (-not $ruleReportItemProps.Contains($propName) -and -not $cimClassPropertiesLookup.Contains($prop.Name)) {
            $ruleReportItemProps.Add($propName, $prop.Value)
        }
    }

    foreach ($prop in $serviceFilter.psobject.Properties) {
        $propName = 'ServiceFilter' + $prop.Name
        if (-not $ruleReportItemProps.Contains($propName) -and -not $cimClassPropertiesLookup.Contains($prop.Name)) {
            $ruleReportItemProps.Add($propName, $prop.Value)
        }
    }

    $ruleReport.Add([pscustomobject]$ruleReportItemProps)
}

$ruleReport | Sort-Object RuleEnabled, RuleDisplayName, RuleProfile | ogv

Answer 2

Seems your $PolicyObjects is a list of your group policy displaynames. I'd tighten up your code in one of the following manners.

$PolicyObjects | ForEach-Object {

    $GPO = Open-NetGPO -PolicyStore "contoso.com\$_"

    foreach($rule in Get-NetFirewallRule -GPOSession $GPO)
    {
        $portfilter    = $rule | Get-NetFirewallPortFilter
        $addressfilter = $rule | Get-NetFirewallAddressFilter

        [PSCustomObject]@{
            GPOName       = $_
            RuleName      = $rule.name
            DisplayName   = $rule.displayname
            DisplayGroup  = $rule.displaygroup
            Protocol      = $portfilter.Protocol
            LocalPort     = $portfilter.LocalPort
            RemotePort    = $portfilter.RemotePort
            RemoteAddress = $addressfilter.RemoteAddress
            Enabled       = $rule.enabled
            Profile       = $rule.profile
            Direction     = $rule.direction
            Action        = $rule.action
        }
    }
} | Export-CSV $env:temp\gpos.csv -Force

Start-Process notepad $env:temp\gpos.csv 

or

$csvdata = foreach($policy in $PolicyObjects)
{
    $GPO = Open-NetGPO -PolicyStore "contoso.com\$policy"

    foreach($rule in Get-NetFirewallRule -GPOSession $GPO)
    {
        $portfilter    = $rule | Get-NetFirewallPortFilter
        $addressfilter = $rule | Get-NetFirewallAddressFilter

        [PSCustomObject]@{
            GPOName       = $policy
            RuleName      = $rule.name
            DisplayName   = $rule.displayname
            DisplayGroup  = $rule.displaygroup
            Protocol      = $portfilter.Protocol
            LocalPort     = $portfilter.LocalPort
            RemotePort    = $portfilter.RemotePort
            RemoteAddress = $addressfilter.RemoteAddress
            Enabled       = $rule.enabled
            Profile       = $rule.profile
            Direction     = $rule.direction
            Action        = $rule.action
        }
    }
}

$csvdata | Export-CSV $env:temp\gpos.csv -Force

Start-Process notepad $env:temp\gpos.csv 

In the first one we change the outer loop to a Foreach-Object to take advantage of the pipeline and piping straight to Export-Csv.

In the second we capture all the output then export.

In both we limit the execution time by limiting the opening/writing to file to one time, limit the portfilter calls to one per rule instead of 3, and we use the [PSCustomObject] type accelerator to construct our final object instead of piping to Select-Object with calculated expressions. Both should achieve your desired result if I understood correctly.

Answer 3

Does this make a difference?

$csv=foreach ($policy in $PolicyObjects) {
$GPO = Open-NetGPO -PolicyStore "contoso.com\$policy"
$policy | Out-File $env:temp\gpos.csv  -Append -Force
Get-NetFirewallRule -GPOSession $GPO  | 
Select Name,
DisplayName,
DisplayGroup,
@{Name='Protocol';Expression={($PSItem | Get-NetFirewallPortFilter).Protocol}},
@{Name='LocalPort';Expression={($PSItem | Get-NetFirewallPortFilter).LocalPort}},
@{Name='RemotePort';Expression={($PSItem | Get-NetFirewallPortFilter).RemotePort}},
@{Name='RemoteAddress';Expression={($PSItem | Get-NetFirewallAddressFilter).RemoteAddress}},
Enabled,
Profile,
Direction,
Action 
}
$csv | Export-CSV $env:temp\gpos.csv -Append -Force
Start-Process notepad $env:temp\gpos.csv