Pablo
Reputation: 2154
How to parse JSON objects that depend on array contents for their attributes with JQ?
I have this complex array of JSON objects that have attributes that are arrays which may or may not be populated.
I am looking for a way using jq to filter out the array elements that do not have any items in their updates attribute, but this is buried deep down the object tree.
Is there a way to produce the sample output from the input JSON array under it?
What I am trying to find is:
Select all top level objects from the array that have a baseBranch attribute and a non-empty config attribute and filter out the results to show me only the elements within the (gradle and gradle-wrapper) attributes within the config that have a non-empty deps array attribute. The result is a deps element that have a non-empty updates array in it.
Thanks in advance to anyone who can take this challenge!
Edit:
- Yes I have tried solving it on my own. I have been through the documentation and so far this is the expression I have:
.[] | . as $parent | select(.baseBranch != null and .config != null) | .config | [.gradle[] .deps[]] - This will filter out all the top level elements and get me to the dependency array, but I cannot figure out how to filter out the
depsarray and group them by the attribute where they are located.
Edit 2:
The query bellow is returning most of the information I want. But I'd like to figure out how to group the updates by the repository and file values.
jq '.[]
| . as $parent
| select(.baseBranch != null and .config != null)
| .config.gradle[]?.deps[]? as $deps
| [
{
repository: $parent.repository,
file:$deps.managerData.packageFile,
depName:$deps.depName,
currentVersion:$deps.currentValue,
newVersion:$deps.updates[]?.newVersion
}
]
'
/* Desired sample output */
[
{
"repository": "repositoryName",
"numberOfUpdates": 5,
"gradleUpdates": [
{
"fileName": "dependencies.gradle",
"package": "packageName.api:feature-flags", /* if packageName then display the depName */
"currentVersion": "0.20.0",
"newVersion": "0.21.0"
},
{
"fileName": "dependencies.gradle",
"package": "io.rest-assured:xml-path", /* if packageName then display the depName */
"currentVersion": "4.5.1",
"newVersion": "5.1.1"
},
{
"fileName": "build.gradle",
"package": "com.diffplug.spotless:com.diffplug.spotless.gradle.plugin", /* if packageName then display the depName */
"currentVersion": "6.9.0",
"newVersion": "6.10.0"
},
{
"fileName": "build.gradle",
"package": "org.springframework.boot:org.springframework.boot.gradle.plugin", /* if packageName then display the depName */
"currentVersion": "2.7.2",
"newVersion": "2.7.3"
}
],
"gradleWrapperUpdates": [
{
"fileName": "gradle/wrapper/gradle-wrapper.properties",
"currentVersion": "7.5",
"newVersion": "7.5.1"
}
]
}
]
/* This is the input */
[
{
"randomKey": "this is not what I am looking for",
"otherFields": "other object that does not match the query"
},
{
"repository": "myRepositoryName",
"baseBranch": "master",
"config": {
"gradle": [
{
"packageFile": "gradle.properties",
"datasource": "maven",
"deps": []
},
{
"packageFile": "dependencies.gradle",
"datasource": "maven",
"deps": [
{
"depName": "packageName:dependencyName",
"currentValue": "0.8.0",
"managerData": {
"fileReplacePosition": 611,
"packageFile": "dependencies.gradle"
},
"fileReplacePosition": 611,
"registryUrls": [
"https://mavenCentral/repository/release",
"https://mavenCentral/repository/maven-mirror",
"https://mavenCentral/repository/gradle-plugins"
],
"depType": "dependencies",
"depIndex": 0,
"updates": [],
"warnings": [],
"versioning": "gradle",
"currentVersion": "0.8.0",
"fixedVersion": "0.8.0"
},
{
"depName": "packageName.api:feature-flags",
"currentValue": "0.20.0",
"managerData": {
"fileReplacePosition": 1118,
"packageFile": "dependencies.gradle"
},
"fileReplacePosition": 1118,
"registryUrls": [
"https://mavenCentral/repository/release",
"https://mavenCentral/repository/maven-mirror",
"https://mavenCentral/repository/gradle-plugins"
],
"depType": "dependencies",
"depIndex": 8,
"updates": [
{
"bucket": "non-major",
"newVersion": "0.21.0",
"newValue": "0.21.0",
"releaseTimestamp": "2022-08-25T04:57:50.000Z",
"newMajor": 0,
"newMinor": 21,
"updateType": "minor",
"branchName": "renovate/all-minor-patch"
}
],
"warnings": [],
"versioning": "gradle",
"currentVersion": "0.20.0",
"isSingleVersion": true,
"fixedVersion": "0.20.0"
},
{
"depName": "io.rest-assured:xml-path",
"currentValue": "4.5.1",
"managerData": {
"fileReplacePosition": 3307,
"packageFile": "dependencies.gradle"
},
"fileReplacePosition": 3307,
"registryUrls": [
"https://mavenCentral/repository/release",
"https://mavenCentral/repository/maven-mirror",
"https://mavenCentral/repository/gradle-plugins"
],
"depType": "dependencies",
"depIndex": 39,
"updates": [
{
"bucket": "major",
"newVersion": "5.1.1",
"newValue": "5.1.1",
"releaseTimestamp": "2022-06-10T06:46:14.000Z",
"newMajor": 5,
"newMinor": 1,
"updateType": "major",
"branchName": "renovate/io.rest-assured-xml-path-5.x"
}
],
"warnings": [],
"versioning": "gradle",
"sourceUrl": "http://svn.sonatype.org/spice/tags/oss-parent-5",
"homepage": "https://rest-assured.io/",
"currentVersion": "4.5.1",
"isSingleVersion": true,
"fixedVersion": "4.5.1"
},
{
"depName": "io.rest-assured:json-schema-validator",
"currentValue": "5.1.1",
"managerData": {
"fileReplacePosition": 3376,
"packageFile": "dependencies.gradle"
},
"fileReplacePosition": 3376,
"registryUrls": [
"https://mavenCentral/repository/release",
"https://mavenCentral/repository/maven-mirror",
"https://mavenCentral/repository/gradle-plugins"
],
"depType": "dependencies",
"depIndex": 40,
"updates": [],
"warnings": [],
"versioning": "gradle",
"sourceUrl": "http://svn.sonatype.org/spice/tags/oss-parent-5",
"homepage": "http://maven.apache.org",
"currentVersion": "5.1.1",
"fixedVersion": "5.1.1"
}
]
},
{
"packageFile": "settings.gradle",
"datasource": "maven",
"deps": []
},
{
"packageFile": "build.gradle",
"datasource": "maven",
"deps": [
{
"depType": "plugin",
"depName": "com.diffplug.spotless",
"packageName": "com.diffplug.spotless:com.diffplug.spotless.gradle.plugin",
"registryUrls": [
"https://mavenCentral/repository/release",
"https://mavenCentral/repository/maven-mirror",
"https://mavenCentral/repository/gradle-plugins",
"https://plugins.gradle.org/m2/"
],
"commitMessageTopic": "plugin com.diffplug.spotless",
"currentValue": "6.9.0",
"managerData": {
"fileReplacePosition": 50,
"packageFile": "build.gradle"
},
"fileReplacePosition": 50,
"depIndex": 0,
"updates": [
{
"bucket": "non-major",
"newVersion": "6.10.0",
"newValue": "6.10.0",
"releaseTimestamp": "2022-08-24T14:40:44.000Z",
"newMajor": 6,
"newMinor": 10,
"updateType": "minor",
"branchName": "renovate/all-minor-patch"
}
],
"warnings": [],
"versioning": "gradle",
"currentVersion": "6.9.0",
"isSingleVersion": true,
"fixedVersion": "6.9.0"
},
{
"depType": "plugin",
"depName": "docker-publish-convention",
"packageName": "docker-publish-convention:docker-publish-convention.gradle.plugin",
"registryUrls": [
"https://mavenCentral/repository/release",
"https://mavenCentral/repository/maven-mirror",
"https://mavenCentral/repository/gradle-plugins",
"https://plugins.gradle.org/m2/"
],
"commitMessageTopic": "plugin docker-publish-convention",
"currentValue": "latest.release",
"managerData": {
"fileReplacePosition": 158,
"packageFile": "build.gradle"
},
"fileReplacePosition": 158,
"depIndex": 2,
"updates": [],
"warnings": [],
"versioning": "gradle",
"skipReason": "invalid-value"
},
{
"depType": "plugin",
"depName": "org.springframework.boot",
"packageName": "org.springframework.boot:org.springframework.boot.gradle.plugin",
"registryUrls": [
"https://mavenCentral/repository/release",
"https://mavenCentral/repository/maven-mirror",
"https://mavenCentral/repository/gradle-plugins",
"https://plugins.gradle.org/m2/"
],
"commitMessageTopic": "plugin org.springframework.boot",
"currentValue": "2.7.2",
"managerData": {
"fileReplacePosition": 217,
"packageFile": "build.gradle"
},
"fileReplacePosition": 217,
"depIndex": 3,
"updates": [
{
"bucket": "non-major",
"newVersion": "2.7.3",
"newValue": "2.7.3",
"releaseTimestamp": "2022-08-18T06:30:12.000Z",
"newMajor": 2,
"newMinor": 7,
"updateType": "patch",
"branchName": "renovate/all-minor-patch"
}
],
"warnings": [],
"versioning": "gradle",
"sourceUrl": "https://github.com/spring-projects/spring-boot",
"homepage": "https://spring.io/projects/spring-boot",
"currentVersion": "2.7.2",
"isSingleVersion": true,
"fixedVersion": "2.7.2"
}
]
}
],
"gradle-wrapper": [
{
"packageFile": "gradle/wrapper/gradle-wrapper.properties",
"deps": [
{
"depName": "gradle",
"currentValue": "7.5",
"replaceString": "https\\://mavenCentral/repository/gradle-dist/gradle-7.5-bin.zip",
"datasource": "gradle-version",
"versioning": "gradle",
"depIndex": 0,
"updates": [
{
"bucket": "non-major",
"newVersion": "7.5.1",
"newValue": "7.5.1",
"releaseTimestamp": "2022-08-05T21:17:56.000Z",
"newMajor": 7,
"newMinor": 5,
"updateType": "patch",
"branchName": "renovate/all-minor-patch"
}
],
"warnings": [],
"sourceUrl": "https://github.com/gradle/gradle",
"homepage": "https://gradle.org",
"currentVersion": "7.5",
"isSingleVersion": true,
"fixedVersion": "7.5"
}
]
}
]
}
}
]
Upvotes: 1
Views: 162
Answers (1)
pmf
Reputation: 36391
With some assumptions made for the edge cases, this should meet your requirements:
map(select(has("baseBranch") and .config != {}) | {
repository,
numberOfUpdates: 0,
gradleUpdates: .config.gradle,
gradleWrapperUpdates: .config."gradle-wrapper"
} | (.gradleUpdates, .gradleWrapperUpdates) |= map(
.packageFile as $fileName | .deps[] | .updates[] as $update | {
$fileName,
package: (.packageName // .depName),
currentVersion,
newVersion: $update.newValue
})
| .numberOfUpdates += ([.gradleUpdates, .gradleWrapperUpdates | length] | add)
)
[
{
"repository": "myRepositoryName",
"numberOfUpdates": 5,
"gradleUpdates": [
{
"fileName": "dependencies.gradle",
"package": "packageName.api:feature-flags",
"currentVersion": "0.20.0",
"newVersion": "0.21.0"
},
{
"fileName": "dependencies.gradle",
"package": "io.rest-assured:xml-path",
"currentVersion": "4.5.1",
"newVersion": "5.1.1"
},
{
"fileName": "build.gradle",
"package": "com.diffplug.spotless:com.diffplug.spotless.gradle.plugin",
"currentVersion": "6.9.0",
"newVersion": "6.10.0"
},
{
"fileName": "build.gradle",
"package": "org.springframework.boot:org.springframework.boot.gradle.plugin",
"currentVersion": "2.7.2",
"newVersion": "2.7.3"
}
],
"gradleWrapperUpdates": [
{
"fileName": "gradle/wrapper/gradle-wrapper.properties",
"package": "gradle",
"currentVersion": "7.5",
"newVersion": "7.5.1"
}
]
}
]
Upvotes: 2
Related Questions
- Parse an array of json object using jq
- Parse JSON Elements in Array with jq
- jq parsing complex json
- Parse dynamic array of objects with JQ
- jq process json where an element can be an array or object
- How to select/get object elements' (array of objects) keys based on value of an element in an object in the aforementioned array in jq?
- JQ to parse nested arrays
- How to parse an array of json object using jq
- jq parsing for elements from sub array
- Getting a list of keys based on another property in json with jq