Jenkins - Get repos and branches as choice parameter

Question

Hi I want to be able to grab a list of repos and present them as a selectable choice parameter for my pipeline.

What would be the best way to do this?

I would like to avoid having to run some sort of build that scrapes all repos and branches and creates a list in a xml or json file.

I also then want to be able to take all the branches in that repo and display them as another selectable choice parameter.

Edit:

Is there a way to add different type of project recognizers for a github organization. I don't want to use a jenkinsfile. Would rather look for a specific filename in the repo ("JenkinsProject.xml") as I have jenkinsFiles in other repos I dont want in this list.

Edit 2:

I found this sh command in jenkins.

resolveScm source: github(checkoutCredentialsId: 'SAME', id: '_', repoOwner: 'GITOWNER', repository: 'REPONAME', scanCredentialsId: 'CREDENTIALS'), targets: ['.']

Seems to find all the branches for the given repo. But how do I access the provided info as its only output to the console.

Edit 3

Okay So I think I will be using the github api. I can call this api call

GET /orgs/:org/repos

which returns all the repos in the given organization in json format. From here I can grab all the branches in a given repo with the api call

GET /repos/:owner/:repo/branches

Then I can checkout master of each repo. If master contains the "JenkinsProject.xml" I will save the repo name and branch names to a file which will be used as choice parameters hopefully.

Sadly I cant find another way to do this other than parsing through all the repos.

I will update this question again if this works out. Otherwise Im still open to more suggestions

Edit 4

I need help figuring out how to import this json file as parameters. Or maybe not use json. I might need to parse and update my buildscript then commit it.

Answer 1

So how I did this was using the api. I have a JenkinsProject.xml file inside of any of the games I want built with this job.

So I have a repo called JenkisDevOps with a jenkinsfile.

This jenkinsfile calls the github api and saves this to a file.

The api call to search for a file within an organization is

https://api.github.com/search/code?q=org:{ORGANIZATION_NAME}+filename:{FILENAME}

This will return every repo with this file in it. from here I can get each branch for these repos with.

https://api.github.com/repos/{ORGANIZATION_NAME}/{REPO_NAME}/branches

Then from here I save all this to a json file with

repos{
    repo:{
        name:"repoName",
        branches:{ 
            name:"branchname",
            name:"branchName2"
            ...
        }
    }
}

The full script is here. I still need to figure out how to convert the saved json file to parameter options now.

pipeline {
agent any

stages {
stage ('Create Build Parameters'){
    steps{
        sh 'echo !---SETUP---!'
        git credentialsId: '', url: 'https://github.com/GenesisGaming/DevOpsJenkins.git'
        httpRequest acceptType: 'APPLICATION_JSON', authentication: '', consoleLogResponseBody: false, contentType: 'APPLICATION_JSON', outputFile: 'Repos.json', responseHandle: 'NONE', url: 'https://api.github.com/search/code?q=org:GenesisGaming+filename:Project.xml&per_page=100'
        readFile 'Repos.json'
        stash includes: '**', name: 'build'
        script {
            node{
                unstash 'build'
                env.WORKSPACE = pwd()
                def buildConfig = load "GenerateBuildSelections.Groovy"
                def repos = buildConfig.GetListOfRepos("${env.WORKSPACE}/Repos.json")

                def dataMap = new HashMap<String,List>()
                for(i = 0; i < repos.size(); i++){
                    def repoName = repos[i]
                    httpRequest acceptType: 'APPLICATION_JSON', authentication: '', consoleLogResponseBody: false, contentType: 'APPLICATION_JSON', outputFile: "branches_${repoName}.json", responseHandle: 'NONE', url: "https://api.github.com/repos/GenesisGaming/${repoName}/branches"
                    env.WORKSPACE = pwd()
                    def branches = buildConfig.GetListOfBranches("${env.WORKSPACE}/branches_${repoName}.json")
                    dataMap.put("${repoName}", "${branches}")

                }
                def builder = new groovy.json.JsonBuilder(dataMap)
                new File("/home/service/BuildSelectionOptions/options.json").write(builder.toPrettyString())
            }
        }
    }
}
}
post {
always {
    sh 'echo !---Cleanup---!'
    cleanWs()
}
}
}

I used the active parameter plugin to get the selectable branches and repos.

For selectedGame( the repo ) I use a groovyscript choice wit hthis script.

#!/usr/bin/env groovy
import groovy.json.JsonSlurper

def inputFile = new File("/var/BuildOptions/options.json")
def InputJSON = new JsonSlurper().parseText(inputFile.text)
def output = []
InputJSON.each{key, val ->
    output.add(key)
}
return output;

and then for the branch parameter I use a active reactive parameter choice with this groovy script and I reference the selectedGame parameter

#!/usr/bin/env groovy
import groovy.json.JsonSlurper

def inputFile = new File("/var/BuildOptions/options.json")
def InputJSON = new JsonSlurper().parseText(inputFile.text)
def output = []
InputJSON.each{key, val ->
String strOut = ""
if(selectedGame == key){
    val.each { outChar ->
        if(outChar == "[" || outChar == " ")
        {
            strOut = ""
        }
        else if(outChar == "]" || outChar == ",") {
            output.add(strOut)
            strOut = ""
        }
        else
        {
            strOut+= outChar
        }
    }
}
}

return output;