Beefcake
Beefcake

Reputation: 801

Programmatically find and remove object in JSON object

TL;DR How do I programatically search for an object which may not be in the same location for each file and remove it?

It's fine if I know exactly where the object will be inside the JSON, but if it changes locations it's difficult to script. At the moment we programtically build an ARM template and push it to Azure, but before we do we compare it to see the version in the cloud differs from the newly generated file, and if it does, either replace it or create a new one based on its version.

The problem is when the template spec is built with a new version Bicep, which then injects that information into the JSON file, which then adjusts the version and templateHash values. Sadly, I cannot prevent this from happening.

          "metadata": {
            "_generator": {
              "name": "bicep",
              "version": "0.4.1008.15138",
              "templateHash": "17593852815978663805"
            }

I used to do a Compare-Object to compare the two files after turning them into PowerShell Objects, but when the binary change came in it threw that idea out. Then I started looking into a more complicated PowerShell comparison where it would ignore lines based on patterns, but this morning that failed with some random error that I can't diagnose cos it's in Azure DevOps pipeline.

Write-Host "`n`n##[debug]šŸž`tLooking for changes  šŸ‘€"
$bicepAlteredLines  = @()
$templateSpecChange = $false
$patterns           = @(
    '\"version\"'
    '\"templateHash\"'
)


# newFile (${currentDir}.json) goes into old oldFile (portal.json)
$compare = Compare-Object `
    -ReferenceObject  $(Get-Content -Path "./portal.json") `
    -DifferenceObject $(Get-Content -Path "./${currentDir}.json")

# find the values where new goes into old
foreach($result in ($compare | Where-Object { $_.SideIndicator -eq "=>" })) {

    # loop over patterns that should be excluded because of bicep binary update
    foreach($pattern in $patterns) {

    # for each bicep binary metadata pattern, check to see if the difference line matches that pattern
        if($result.InputObject -match $pattern) {

            # create variable of old oldFile which is referenced on the same line
            $comparedResult = $compare |
                Where-Object {
                    ($_.SideIndicator -eq "<=") -and
                    ($_.InputObject.ReadCount -eq $result.InputObject.Readcount)
                }

            # check to see if the variable also contains the pattern that should be excluded
            if($comparedResult.InputObject -match $pattern) {

                $selectSearch = Select-String -Path "./${currentDir}.json" -Pattern $pattern -Context 2 |
                    Where-Object {
                        $_.LineNumber -eq $result.InputObject.Readcount
                    }

                # validate if the pattern relates to bicep and not just random string that looks like what we want
                foreach($item in $selectSearch) {

                    # we do this by checking the lines above the pattern, as this will contain the word 'bicep':
                    # "metadata": {
                    #     "_generator": {
                    #       "name": "bicep",
                    #       "version": "0.4.1008.15138",                <- if this line was the pattern, 1 up contains 'bicep'
                    #       "templateHash": "9914162448113979868"       <- if this line was the pattern, 2 up contains 'bicep'
                    #     }
                    foreach($bicepCheck in $item.Context.PreContext) {
                        if($bicepCheck -match "bicep") {

                            # if 'bicep' has been found, add array for readout later
                            $bicepAlteredLines += $item.LineNumber
                            Write-Host "Bicep binary change found at line $($item.LineNumber) for pattern ${pattern}. Adding to ignore list."
                        }
                        else {
                            # if the variable doesn't contain 'bicep', then obviously there's been a change in the JSON layout
                            $templateSpecChange = $true
                        }
                    }
                }
            }
            else {
                # if the variable doesn't contain the pattern, then obviously there's been a change in the JSON layout
                $templateSpecChange = $true
            }
        }
        else {
            # if the variable doesn't contain the pattern, then obviously there's been a change in the JSON layout
            $templateSpecChange = $true
        }
    }
}


Write-Host "`nThe following lines are ignored due Bicep binary change:"
$bicepAlteredLines | Sort-Object -Unique

Write-Host "`n`n##[debug]šŸž`tDifference found in file comparison = ${templateSpecChange}"

So I would like to either;

{
  "$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "metadata": {
    "_generator": {
      "name": "bicep",
      "version": "0.4.1111.15138",
      "templateHash": "asdfasdfasdf"
    }
  },
  "parameters": {
    "p_vnetPeeringsCompute": {
      "type": "array",
      "metadata": {
        "description": "List of peerings to the virtual network."
      }
    },
    "p_vnetPeeringsStorage": {
      "type": "array",
      "metadata": {
        "description": "List of peerings to the virtual network."
      }
    },
    "p_vnetPeeringsLinkFile": {
      "type": "array",
      "metadata": {
        "description": "List of peerings to the virtual network."
      }
    },
    "p_tags": {
      "type": "object",
      "defaultValue": {},
      "metadata": {
        "description": "List of tags applied to resources."
      }
    }
  },
  "functions": [],
  "variables": {
    "v_longLocation": "uksouth",
    "v_shortLocation": "uks",
    "v_templateDescription": "Azure Private DNS Template Spec",
    "v_templateDisplayName": "privateDns",
    "v_templateName": "privateDns",
    "v_templateVersion": "2.0.0",
    "v_templateTags": {
      "tsName": "[variables('v_templateName')]",
      "tsVersion": "[variables('v_templateVersion')]"
    },
    "v_tags": "[union(parameters('p_tags'), variables('v_templateTags'))]"
  },
  "resources": [
    {
      "type": "Microsoft.Resources/resourceGroups",
      "apiVersion": "2021-01-01",
      "name": "[format('{0}-{1}-compute-rsg', variables('v_shortLocation'), subscription().displayName)]",
      "location": "[variables('v_longLocation')]",
      "tags": "[variables('v_tags')]"
    },
    {
      "type": "Microsoft.Resources/resourceGroups",
      "apiVersion": "2021-01-01",
      "name": "[format('{0}-{1}-link-rsg', variables('v_shortLocation'), subscription().displayName)]",
      "location": "[variables('v_longLocation')]",
      "tags": "[variables('v_tags')]"
    },
    {
      "type": "Microsoft.Resources/resourceGroups",
      "apiVersion": "2021-01-01",
      "name": "[format('{0}-{1}-storage-rsg', variables('v_shortLocation'), subscription().displayName)]",
      "location": "[variables('v_longLocation')]",
      "tags": "[variables('v_tags')]"
    },
    {
      "type": "Microsoft.Resources/deployments",
      "apiVersion": "2020-06-01",
      "name": "privateCompute",
      "resourceGroup": "[format('{0}-{1}-compute-rsg', variables('v_shortLocation'), subscription().displayName)]",
      "properties": {
        "expressionEvaluationOptions": {
          "scope": "inner"
        },
        "mode": "Incremental",
        "parameters": {
          "p_tags": {
            "value": "[variables('v_tags')]"
          },
          "p_vnetPeeringsCompute": {
            "value": "[parameters('p_vnetPeeringsCompute')]"
          }
        },
        "template": {
          "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
          "contentVersion": "1.0.0.0",
          "metadata": {
            "_generator": {
              "name": "bicep",
              "version": "0.4.1008.15138",
              "templateHash": "17593852815978663805"
            }
          },
          "parameters": {
            "p_tags": {
              "type": "object",
              "defaultValue": {},
              "metadata": {
                "description": "List of tags applied to resources."
              }
            },
            "p_vnetPeeringsCompute": {
              "type": "array",
              "metadata": {
                "description": "List of peerings to the virtual network."
              }
            }
          },
          "functions": [],
          "resources": [
            {
              "type": "Microsoft.Network/privateDnsZones",
              "apiVersion": "2020-06-01",
              "name": "compute.bld.local",
              "tags": "[parameters('p_tags')]",
              "location": "Global"
            },
            {
              "copy": {
                "name": "vnl",
                "count": "[length(parameters('p_vnetPeeringsCompute'))]"
              },
              "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks",
              "apiVersion": "2020-06-01",
              "name": "[format('{0}/{1}', 'compute.bld.local', last(split(parameters('p_vnetPeeringsCompute')[copyIndex()].id, '/')))]",
              "tags": "[parameters('p_tags')]",
              "location": "Global",
              "properties": {
                "virtualNetwork": {
                  "id": "[parameters('p_vnetPeeringsCompute')[copyIndex()].id]"
                },
                "registrationEnabled": false
              },
              "dependsOn": [
                "[resourceId('Microsoft.Network/privateDnsZones', 'compute.bld.local')]"
              ]
            }
          ]
        }
      },
      "dependsOn": [
        "[subscriptionResourceId('Microsoft.Resources/resourceGroups', format('{0}-{1}-compute-rsg', variables('v_shortLocation'), subscription().displayName))]"
      ]
    },
    {
      "type": "Microsoft.Resources/deployments",
      "apiVersion": "2020-06-01",
      "name": "privateLink",
      "resourceGroup": "[format('{0}-{1}-link-rsg', variables('v_shortLocation'), subscription().displayName)]",
      "properties": {
        "expressionEvaluationOptions": {
          "scope": "inner"
        },
        "mode": "Incremental",
        "parameters": {
          "p_vnetPeeringsLinkFile": {
            "value": "[parameters('p_vnetPeeringsLinkFile')]"
          },
          "p_tags": {
            "value": "[variables('v_tags')]"
          }
        },
        "template": {
          "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
          "contentVersion": "1.0.0.0",
          "metadata": {
            "_generator": {
              "name": "bicep",
              "version": "0.4.1008.15138",
              "templateHash": "14603239311913824090"
            }
          },
          "parameters": {
            "p_vnetPeeringsLinkFile": {
              "type": "array",
              "metadata": {
                "description": "List of peerings to the virtual network."
              }
            },
            "p_tags": {
              "type": "object",
              "defaultValue": {},
              "metadata": {
                "description": "List of tags applied to resources."
              }
            }
          },
          "functions": [],
          "resources": [
            {
              "type": "Microsoft.Network/privateDnsZones",
              "apiVersion": "2020-06-01",
              "name": "[format('privatelink.file.{0}', environment().suffixes.storage)]",
              "tags": "[parameters('p_tags')]",
              "location": "Global"
            },
            {
              "copy": {
                "name": "vnl",
                "count": "[length(parameters('p_vnetPeeringsLinkFile'))]"
              },
              "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks",
              "apiVersion": "2020-06-01",
              "name": "[format('{0}/{1}', format('privatelink.file.{0}', environment().suffixes.storage), last(split(parameters('p_vnetPeeringsLinkFile')[copyIndex()].id, '/')))]",
              "tags": "[parameters('p_tags')]",
              "location": "Global",
              "properties": {
                "virtualNetwork": {
                  "id": "[parameters('p_vnetPeeringsLinkFile')[copyIndex()].id]"
                },
                "registrationEnabled": false
              },
              "dependsOn": [
                "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.file.{0}', environment().suffixes.storage))]"
              ]
            },
            {
              "type": "Microsoft.Network/privateDnsZones",
              "apiVersion": "2020-06-01",
              "name": "[format('privatelink{0}', environment().suffixes.sqlServerHostname)]",
              "tags": "[parameters('p_tags')]",
              "location": "Global"
            }
          ]
        }
      },
      "dependsOn": [
        "[subscriptionResourceId('Microsoft.Resources/resourceGroups', format('{0}-{1}-link-rsg', variables('v_shortLocation'), subscription().displayName))]"
      ]
    },
    {
      "type": "Microsoft.Resources/deployments",
      "apiVersion": "2020-06-01",
      "name": "privateStorage",
      "resourceGroup": "[format('{0}-{1}-storage-rsg', variables('v_shortLocation'), subscription().displayName)]",
      "properties": {
        "expressionEvaluationOptions": {
          "scope": "inner"
        },
        "mode": "Incremental",
        "parameters": {
          "p_tags": {
            "value": "[variables('v_tags')]"
          },
          "p_vnetPeeringsStorage": {
            "value": "[parameters('p_vnetPeeringsStorage')]"
          }
        },
        "template": {
          "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
          "contentVersion": "1.0.0.0",
          "metadata": {
            "_generator": {
              "name": "bicep",
              "version": "0.4.1008.15138",
              "templateHash": "12131108274222647600"
            }
          },
          "parameters": {
            "p_tags": {
              "type": "object",
              "defaultValue": {},
              "metadata": {
                "description": "List of tags applied to resources."
              }
            },
            "p_vnetPeeringsStorage": {
              "type": "array",
              "metadata": {
                "description": "List of peerings to the virtual network."
              }
            }
          },
          "functions": [],
          "resources": [
            {
              "type": "Microsoft.Network/privateDnsZones",
              "apiVersion": "2020-06-01",
              "name": "storage.bld.local",
              "tags": "[parameters('p_tags')]",
              "location": "Global"
            },
            {
              "copy": {
                "name": "vnl",
                "count": "[length(parameters('p_vnetPeeringsStorage'))]"
              },
              "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks",
              "apiVersion": "2020-06-01",
              "name": "[format('{0}/{1}', 'storage.bld.local', last(split(parameters('p_vnetPeeringsStorage')[copyIndex()].id, '/')))]",
              "tags": "[parameters('p_tags')]",
              "location": "Global",
              "properties": {
                "virtualNetwork": {
                  "id": "[parameters('p_vnetPeeringsStorage')[copyIndex()].id]"
                },
                "registrationEnabled": false
              },
              "dependsOn": [
                "[resourceId('Microsoft.Network/privateDnsZones', 'storage.bld.local')]"
              ]
            }
          ]
        }
      },
      "dependsOn": [
        "[subscriptionResourceId('Microsoft.Resources/resourceGroups', format('{0}-{1}-storage-rsg', variables('v_shortLocation'), subscription().displayName))]"
      ]
    }
  ]
}

At the moment I know it's in 2 standard locations, but I am not sure if it will appear in others.

āžœ $asdf.resources.properties.template.metadata

_generator
----------
@{name=bicep; version=0.4.1008.15138; templateHash=17593852815978663805}
@{name=bicep; version=0.4.1008.15138; templateHash=14603239311913824090}
@{name=bicep; version=0.4.1008.15138; templateHash=12131108274222647600}

compare
āžœ $asdf.metadata

_generator
----------
@{name=bicep; version=0.4.1111.15138; templateHash=asdfasdfasdf}

It would be more ideal if there was an option in PowerShell to find all instances of objects named metadata and remove them, but I cannot find an option like this in PowerShell. Any thoughts/ideas would be greatful.

Before anyone mentions that I should just remove all instances of $asdf.resources.properties.template.metadata and $asdf.metadata, I am wondering how to search for any instances of *.metadata that would/could be in other locations that may or may not come up. More of a "look for all of *.metadata and remove from object, where ever it is".

Upvotes: 1

Views: 727

Answers (2)

Tomalak
Tomalak

Reputation: 338326

In pure PowerShell you can use a recursive function to clean out any "metadata" property from objects at any nesting depth.

function StripMetadata {
    param($node)

    if ($node -is [pscustomobject]) {
        foreach ($prop in $node.PSObject.Properties) {
            if ($prop.Name -eq "metadata") {
                $node.PSObject.Properties.Remove($prop.Name)
            } else {
                StripMetadata $prop.Value
            }
        }
    } elseif ($node -is [array]) {
        foreach ($item in $node) {
            StripMetadata $item
        }
    }
}

Note that this modifies the object in-place, there is no result value.

$data = $yourJson | ConvertFrom-Json 
StripMetadata $data
$result = $data | ConvertTo-Json -Depth 100 -Compress

$result is (formatted):

{
    "$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "p_vnetPeeringsCompute": {
            "type": "array"
        },
        "p_vnetPeeringsStorage": {
            "type": "array"
        },
        "p_vnetPeeringsLinkFile": {
            "type": "array"
        },
        "p_tags": {
            "type": "object",
            "defaultValue": {}
        }
    },
    "functions": [],
    "variables": {
        "v_longLocation": "uksouth",
        "v_shortLocation": "uks",
        "v_templateDescription": "Azure Private DNS Template Spec",
        "v_templateDisplayName": "privateDns",
        "v_templateName": "privateDns",
        "v_templateVersion": "2.0.0",
        "v_templateTags": {
            "tsName": "[variables(\u0027v_templateName\u0027)]",
            "tsVersion": "[variables(\u0027v_templateVersion\u0027)]"
        },
        "v_tags": "[union(parameters(\u0027p_tags\u0027), variables(\u0027v_templateTags\u0027))]"
    },
    "resources": [{
        "type": "Microsoft.Resources/resourceGroups",
        "apiVersion": "2021-01-01",
        "name": "[format(\u0027{0}-{1}-compute-rsg\u0027, variables(\u0027v_shortLocation\u0027), subscription().displayName)]",
        "location": "[variables(\u0027v_longLocation\u0027)]",
        "tags": "[variables(\u0027v_tags\u0027)]"
    }, {
        "type": "Microsoft.Resources/resourceGroups",
        "apiVersion": "2021-01-01",
        "name": "[format(\u0027{0}-{1}-link-rsg\u0027, variables(\u0027v_shortLocation\u0027), subscription().displayName)]",
        "location": "[variables(\u0027v_longLocation\u0027)]",
        "tags": "[variables(\u0027v_tags\u0027)]"
    }, {
        "type": "Microsoft.Resources/resourceGroups",
        "apiVersion": "2021-01-01",
        "name": "[format(\u0027{0}-{1}-storage-rsg\u0027, variables(\u0027v_shortLocation\u0027), subscription().displayName)]",
        "location": "[variables(\u0027v_longLocation\u0027)]",
        "tags": "[variables(\u0027v_tags\u0027)]"
    }, {
        "type": "Microsoft.Resources/deployments",
        "apiVersion": "2020-06-01",
        "name": "privateCompute",
        "resourceGroup": "[format(\u0027{0}-{1}-compute-rsg\u0027, variables(\u0027v_shortLocation\u0027), subscription().displayName)]",
        "properties": {
            "expressionEvaluationOptions": {
                "scope": "inner"
            },
            "mode": "Incremental",
            "parameters": {
                "p_tags": {
                    "value": "[variables(\u0027v_tags\u0027)]"
                },
                "p_vnetPeeringsCompute": {
                    "value": "[parameters(\u0027p_vnetPeeringsCompute\u0027)]"
                }
            },
            "template": {
                "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
                "contentVersion": "1.0.0.0",
                "parameters": {
                    "p_tags": {
                        "type": "object",
                        "defaultValue": {}
                    },
                    "p_vnetPeeringsCompute": {
                        "type": "array"
                    }
                },
                "functions": [],
                "resources": [{
                    "type": "Microsoft.Network/privateDnsZones",
                    "apiVersion": "2020-06-01",
                    "name": "compute.bld.local",
                    "tags": "[parameters(\u0027p_tags\u0027)]",
                    "location": "Global"
                }, {
                    "copy": {
                        "name": "vnl",
                        "count": "[length(parameters(\u0027p_vnetPeeringsCompute\u0027))]"
                    },
                    "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks",
                    "apiVersion": "2020-06-01",
                    "name": "[format(\u0027{0}/{1}\u0027, \u0027compute.bld.local\u0027, last(split(parameters(\u0027p_vnetPeeringsCompute\u0027)[copyIndex()].id, \u0027/\u0027)))]",
                    "tags": "[parameters(\u0027p_tags\u0027)]",
                    "location": "Global",
                    "properties": {
                        "virtualNetwork": {
                            "id": "[parameters(\u0027p_vnetPeeringsCompute\u0027)[copyIndex()].id]"
                        },
                        "registrationEnabled": false
                    },
                    "dependsOn": ["[resourceId(\u0027Microsoft.Network/privateDnsZones\u0027, \u0027compute.bld.local\u0027)]"]
                }]
            }
        },
        "dependsOn": ["[subscriptionResourceId(\u0027Microsoft.Resources/resourceGroups\u0027, format(\u0027{0}-{1}-compute-rsg\u0027, variables(\u0027v_shortLocation\u0027), subscription().displayName))]"]
    }, {
        "type": "Microsoft.Resources/deployments",
        "apiVersion": "2020-06-01",
        "name": "privateLink",
        "resourceGroup": "[format(\u0027{0}-{1}-link-rsg\u0027, variables(\u0027v_shortLocation\u0027), subscription().displayName)]",
        "properties": {
            "expressionEvaluationOptions": {
                "scope": "inner"
            },
            "mode": "Incremental",
            "parameters": {
                "p_vnetPeeringsLinkFile": {
                    "value": "[parameters(\u0027p_vnetPeeringsLinkFile\u0027)]"
                },
                "p_tags": {
                    "value": "[variables(\u0027v_tags\u0027)]"
                }
            },
            "template": {
                "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
                "contentVersion": "1.0.0.0",
                "parameters": {
                    "p_vnetPeeringsLinkFile": {
                        "type": "array"
                    },
                    "p_tags": {
                        "type": "object",
                        "defaultValue": {}
                    }
                },
                "functions": [],
                "resources": [{
                    "type": "Microsoft.Network/privateDnsZones",
                    "apiVersion": "2020-06-01",
                    "name": "[format(\u0027privatelink.file.{0}\u0027, environment().suffixes.storage)]",
                    "tags": "[parameters(\u0027p_tags\u0027)]",
                    "location": "Global"
                }, {
                    "copy": {
                        "name": "vnl",
                        "count": "[length(parameters(\u0027p_vnetPeeringsLinkFile\u0027))]"
                    },
                    "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks",
                    "apiVersion": "2020-06-01",
                    "name": "[format(\u0027{0}/{1}\u0027, format(\u0027privatelink.file.{0}\u0027, environment().suffixes.storage), last(split(parameters(\u0027p_vnetPeeringsLinkFile\u0027)[copyIndex()].id, \u0027/\u0027)))]",
                    "tags": "[parameters(\u0027p_tags\u0027)]",
                    "location": "Global",
                    "properties": {
                        "virtualNetwork": {
                            "id": "[parameters(\u0027p_vnetPeeringsLinkFile\u0027)[copyIndex()].id]"
                        },
                        "registrationEnabled": false
                    },
                    "dependsOn": ["[resourceId(\u0027Microsoft.Network/privateDnsZones\u0027, format(\u0027privatelink.file.{0}\u0027, environment().suffixes.storage))]"]
                }, {
                    "type": "Microsoft.Network/privateDnsZones",
                    "apiVersion": "2020-06-01",
                    "name": "[format(\u0027privatelink{0}\u0027, environment().suffixes.sqlServerHostname)]",
                    "tags": "[parameters(\u0027p_tags\u0027)]",
                    "location": "Global"
                }]
            }
        },
        "dependsOn": ["[subscriptionResourceId(\u0027Microsoft.Resources/resourceGroups\u0027, format(\u0027{0}-{1}-link-rsg\u0027, variables(\u0027v_shortLocation\u0027), subscription().displayName))]"]
    }, {
        "type": "Microsoft.Resources/deployments",
        "apiVersion": "2020-06-01",
        "name": "privateStorage",
        "resourceGroup": "[format(\u0027{0}-{1}-storage-rsg\u0027, variables(\u0027v_shortLocation\u0027), subscription().displayName)]",
        "properties": {
            "expressionEvaluationOptions": {
                "scope": "inner"
            },
            "mode": "Incremental",
            "parameters": {
                "p_tags": {
                    "value": "[variables(\u0027v_tags\u0027)]"
                },
                "p_vnetPeeringsStorage": {
                    "value": "[parameters(\u0027p_vnetPeeringsStorage\u0027)]"
                }
            },
            "template": {
                "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
                "contentVersion": "1.0.0.0",
                "parameters": {
                    "p_tags": {
                        "type": "object",
                        "defaultValue": {}
                    },
                    "p_vnetPeeringsStorage": {
                        "type": "array"
                    }
                },
                "functions": [],
                "resources": [{
                    "type": "Microsoft.Network/privateDnsZones",
                    "apiVersion": "2020-06-01",
                    "name": "storage.bld.local",
                    "tags": "[parameters(\u0027p_tags\u0027)]",
                    "location": "Global"
                }, {
                    "copy": {
                        "name": "vnl",
                        "count": "[length(parameters(\u0027p_vnetPeeringsStorage\u0027))]"
                    },
                    "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks",
                    "apiVersion": "2020-06-01",
                    "name": "[format(\u0027{0}/{1}\u0027, \u0027storage.bld.local\u0027, last(split(parameters(\u0027p_vnetPeeringsStorage\u0027)[copyIndex()].id, \u0027/\u0027)))]",
                    "tags": "[parameters(\u0027p_tags\u0027)]",
                    "location": "Global",
                    "properties": {
                        "virtualNetwork": {
                            "id": "[parameters(\u0027p_vnetPeeringsStorage\u0027)[copyIndex()].id]"
                        },
                        "registrationEnabled": false
                    },
                    "dependsOn": ["[resourceId(\u0027Microsoft.Network/privateDnsZones\u0027, \u0027storage.bld.local\u0027)]"]
                }]
            }
        },
        "dependsOn": ["[subscriptionResourceId(\u0027Microsoft.Resources/resourceGroups\u0027, format(\u0027{0}-{1}-storage-rsg\u0027, variables(\u0027v_shortLocation\u0027), subscription().displayName))]"]
    }]
}

Upvotes: 3

pmf
pmf

Reputation: 36296

As jq is tagged, you can use walk to traverse the JSON document, and if it encounters an object, apply del to delete any .metadata field:

jq 'walk(if type == "object" then del(.metadata) else . end)' file.json
{
  "$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "p_vnetPeeringsCompute": {
      "type": "array"
    },
    "p_vnetPeeringsStorage": {
      "type": "array"
    },
    "p_vnetPeeringsLinkFile": {
      "type": "array"
    },
    "p_tags": {
      "type": "object",
      "defaultValue": {}
    }
  },
  "functions": [],
  "variables": {
    "v_longLocation": "uksouth",
    "v_shortLocation": "uks",
    "v_templateDescription": "Azure Private DNS Template Spec",
    "v_templateDisplayName": "privateDns",
    "v_templateName": "privateDns",
    "v_templateVersion": "2.0.0",
    "v_templateTags": {
      "tsName": "[variables('v_templateName')]",
      "tsVersion": "[variables('v_templateVersion')]"
    },
    "v_tags": "[union(parameters('p_tags'), variables('v_templateTags'))]"
  },
  "resources": [
    {
      "type": "Microsoft.Resources/resourceGroups",
      "apiVersion": "2021-01-01",
      "name": "[format('{0}-{1}-compute-rsg', variables('v_shortLocation'), subscription().displayName)]",
      "location": "[variables('v_longLocation')]",
      "tags": "[variables('v_tags')]"
    },
    {
      "type": "Microsoft.Resources/resourceGroups",
      "apiVersion": "2021-01-01",
      "name": "[format('{0}-{1}-link-rsg', variables('v_shortLocation'), subscription().displayName)]",
      "location": "[variables('v_longLocation')]",
      "tags": "[variables('v_tags')]"
    },
    {
      "type": "Microsoft.Resources/resourceGroups",
      "apiVersion": "2021-01-01",
      "name": "[format('{0}-{1}-storage-rsg', variables('v_shortLocation'), subscription().displayName)]",
      "location": "[variables('v_longLocation')]",
      "tags": "[variables('v_tags')]"
    },
    {
      "type": "Microsoft.Resources/deployments",
      "apiVersion": "2020-06-01",
      "name": "privateCompute",
      "resourceGroup": "[format('{0}-{1}-compute-rsg', variables('v_shortLocation'), subscription().displayName)]",
      "properties": {
        "expressionEvaluationOptions": {
          "scope": "inner"
        },
        "mode": "Incremental",
        "parameters": {
          "p_tags": {
            "value": "[variables('v_tags')]"
          },
          "p_vnetPeeringsCompute": {
            "value": "[parameters('p_vnetPeeringsCompute')]"
          }
        },
        "template": {
          "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
          "contentVersion": "1.0.0.0",
          "parameters": {
            "p_tags": {
              "type": "object",
              "defaultValue": {}
            },
            "p_vnetPeeringsCompute": {
              "type": "array"
            }
          },
          "functions": [],
          "resources": [
            {
              "type": "Microsoft.Network/privateDnsZones",
              "apiVersion": "2020-06-01",
              "name": "compute.bld.local",
              "tags": "[parameters('p_tags')]",
              "location": "Global"
            },
            {
              "copy": {
                "name": "vnl",
                "count": "[length(parameters('p_vnetPeeringsCompute'))]"
              },
              "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks",
              "apiVersion": "2020-06-01",
              "name": "[format('{0}/{1}', 'compute.bld.local', last(split(parameters('p_vnetPeeringsCompute')[copyIndex()].id, '/')))]",
              "tags": "[parameters('p_tags')]",
              "location": "Global",
              "properties": {
                "virtualNetwork": {
                  "id": "[parameters('p_vnetPeeringsCompute')[copyIndex()].id]"
                },
                "registrationEnabled": false
              },
              "dependsOn": [
                "[resourceId('Microsoft.Network/privateDnsZones', 'compute.bld.local')]"
              ]
            }
          ]
        }
      },
      "dependsOn": [
        "[subscriptionResourceId('Microsoft.Resources/resourceGroups', format('{0}-{1}-compute-rsg', variables('v_shortLocation'), subscription().displayName))]"
      ]
    },
    {
      "type": "Microsoft.Resources/deployments",
      "apiVersion": "2020-06-01",
      "name": "privateLink",
      "resourceGroup": "[format('{0}-{1}-link-rsg', variables('v_shortLocation'), subscription().displayName)]",
      "properties": {
        "expressionEvaluationOptions": {
          "scope": "inner"
        },
        "mode": "Incremental",
        "parameters": {
          "p_vnetPeeringsLinkFile": {
            "value": "[parameters('p_vnetPeeringsLinkFile')]"
          },
          "p_tags": {
            "value": "[variables('v_tags')]"
          }
        },
        "template": {
          "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
          "contentVersion": "1.0.0.0",
          "parameters": {
            "p_vnetPeeringsLinkFile": {
              "type": "array"
            },
            "p_tags": {
              "type": "object",
              "defaultValue": {}
            }
          },
          "functions": [],
          "resources": [
            {
              "type": "Microsoft.Network/privateDnsZones",
              "apiVersion": "2020-06-01",
              "name": "[format('privatelink.file.{0}', environment().suffixes.storage)]",
              "tags": "[parameters('p_tags')]",
              "location": "Global"
            },
            {
              "copy": {
                "name": "vnl",
                "count": "[length(parameters('p_vnetPeeringsLinkFile'))]"
              },
              "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks",
              "apiVersion": "2020-06-01",
              "name": "[format('{0}/{1}', format('privatelink.file.{0}', environment().suffixes.storage), last(split(parameters('p_vnetPeeringsLinkFile')[copyIndex()].id, '/')))]",
              "tags": "[parameters('p_tags')]",
              "location": "Global",
              "properties": {
                "virtualNetwork": {
                  "id": "[parameters('p_vnetPeeringsLinkFile')[copyIndex()].id]"
                },
                "registrationEnabled": false
              },
              "dependsOn": [
                "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.file.{0}', environment().suffixes.storage))]"
              ]
            },
            {
              "type": "Microsoft.Network/privateDnsZones",
              "apiVersion": "2020-06-01",
              "name": "[format('privatelink{0}', environment().suffixes.sqlServerHostname)]",
              "tags": "[parameters('p_tags')]",
              "location": "Global"
            }
          ]
        }
      },
      "dependsOn": [
        "[subscriptionResourceId('Microsoft.Resources/resourceGroups', format('{0}-{1}-link-rsg', variables('v_shortLocation'), subscription().displayName))]"
      ]
    },
    {
      "type": "Microsoft.Resources/deployments",
      "apiVersion": "2020-06-01",
      "name": "privateStorage",
      "resourceGroup": "[format('{0}-{1}-storage-rsg', variables('v_shortLocation'), subscription().displayName)]",
      "properties": {
        "expressionEvaluationOptions": {
          "scope": "inner"
        },
        "mode": "Incremental",
        "parameters": {
          "p_tags": {
            "value": "[variables('v_tags')]"
          },
          "p_vnetPeeringsStorage": {
            "value": "[parameters('p_vnetPeeringsStorage')]"
          }
        },
        "template": {
          "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
          "contentVersion": "1.0.0.0",
          "parameters": {
            "p_tags": {
              "type": "object",
              "defaultValue": {}
            },
            "p_vnetPeeringsStorage": {
              "type": "array"
            }
          },
          "functions": [],
          "resources": [
            {
              "type": "Microsoft.Network/privateDnsZones",
              "apiVersion": "2020-06-01",
              "name": "storage.bld.local",
              "tags": "[parameters('p_tags')]",
              "location": "Global"
            },
            {
              "copy": {
                "name": "vnl",
                "count": "[length(parameters('p_vnetPeeringsStorage'))]"
              },
              "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks",
              "apiVersion": "2020-06-01",
              "name": "[format('{0}/{1}', 'storage.bld.local', last(split(parameters('p_vnetPeeringsStorage')[copyIndex()].id, '/')))]",
              "tags": "[parameters('p_tags')]",
              "location": "Global",
              "properties": {
                "virtualNetwork": {
                  "id": "[parameters('p_vnetPeeringsStorage')[copyIndex()].id]"
                },
                "registrationEnabled": false
              },
              "dependsOn": [
                "[resourceId('Microsoft.Network/privateDnsZones', 'storage.bld.local')]"
              ]
            }
          ]
        }
      },
      "dependsOn": [
        "[subscriptionResourceId('Microsoft.Resources/resourceGroups', format('{0}-{1}-storage-rsg', variables('v_shortLocation'), subscription().displayName))]"
      ]
    }
  ]
}

Demo

Upvotes: 3

Related Questions