CODEWITHSUNDEEP
httpdownloadgradleunzip
Peter Butkovic
Peter Butkovic

Reputation: 12149

gradle - download and unzip file from url

What would be the proper gradle way of downloading and unzipping the file from url (http)?

If possible, I'd like to prevent re-downloading each time I run the task (in ant.get can be achieved by skipexisting: 'true').

My current solution would be:

task foo {
  ant.get(src: 'http://.../file.zip', dest: 'somedir', skipexisting: 'true')
  ant.unzip(src: 'somedir' + '/file.zip', dest: 'unpackdir')
}

still, I'd expect ant-free solution. Any chance to achieve that?

Upvotes: 48

Views: 48035

Answers (7)

RaGe
RaGe

Reputation: 23677

Let's say you want to download this zip file as a dependency:

https://github.com/jmeter-gradle-plugin/jmeter-gradle-plugin/archive/1.0.3.zip

You define your ivy repo as:

repositories {
    ivy {
        url 'https://github.com/'

        patternLayout {
            artifact '/[organisation]/[module]/archive/[revision].[ext]'
        }

        // This is required in Gradle 6.0+ as metadata file (ivy.xml) 
        // is mandatory. Docs linked below this code section
        metadataSources { artifact() } 
    }
}

reference for required metadata here

The dependency can then be used as:

dependencies {
    compile 'jmeter-gradle-plugin:jmeter-gradle-plugin:1.0.3@zip'
    //This maps to the pattern: [organisation]:[module]:[revision]:[classifier]@[ext]         
}

To unzip:

task unzip(type: Copy) {

  def zipPath = project.configurations.compile.find {it.name.startsWith("jmeter") }
  println zipPath
  def zipFile = file(zipPath)
  def outputDir = file("${buildDir}/unpacked/dist")

  from zipTree(zipFile)
  into outputDir

}

optional:

If you have more than one repository in your project, it may also help (for build time and somewhat security) to restrict dependency search with relevant repositories.

Gradle 6.2+:

repositories {
    mavenCentral()
    def github = ivy {
        url 'https://github.com/'
        patternLayout {
            artifact '/[organisation]/[module]/archive/[revision].[ext]'
        }
        metadataSources { artifact() }
    }
    exclusiveContent {
        forRepositories(github)
        filter { includeGroup("jmeter-gradle-plugin") }
    }
}

Earlier gradle versions:

repositories {
    mavenCentral {
        content { excludeGroup("jmeter-gradle-plugin") }
    }
    ivy {
        url 'https://github.com/'
        patternLayout {
            artifact '/[organisation]/[module]/archive/[revision].[ext]'
        }
        metadataSources { artifact() }
        content { includeGroup("jmeter-gradle-plugin") }
    }
}

Upvotes: 98

HolyM
HolyM

Reputation: 312

    plugins {
        id 'de.undercouch.download' version '4.0.0'
    }

    /**
     * The following two tasks download a ZIP file and extract its
     * contents to the build directory
     */
    task downloadZipFile(type: Download) {
        src 'https://github.com/gradle-download-task/archive/1.0.zip'
        dest new File(buildDir, '1.0.zip')
    }

    task downloadAndUnzipFile(dependsOn: downloadZipFile, type: Copy) {
        from zipTree(downloadZipFile.dest)
        into buildDir
    }

https://github.com/michel-kraemer/gradle-download-task

Upvotes: 14

Daniel Alder
Daniel Alder

Reputation: 5372

"native gradle", only the download part (see other answers for unzipping)

task foo {
  def src = 'http://example.com/file.zip'
  def destdir = 'somedir'
  def destfile = "$destdir/file.zip"
  doLast {
    def url = new URL(src)
    def f = new File(destfile)
    if (f.exists()) {
      println "file $destfile already exists, skipping download"
    } else {
      mkdir "$destdir"
      println "Downloading $destfile from $url..."
      url.withInputStream { i -> f.withOutputStream { it << i } }
    }
  }
}

Upvotes: 1

Johny
Johny

Reputation: 869

This works with Gradle 5 (tested with 5.5.1):

task download {
    doLast {
        def f = new File('file_path')
        new URL('url').withInputStream{ i -> f.withOutputStream{ it << i }}
    }
}

Calling gradle download downloads the file from url to file_path.

You can use the other methods from other answers to unzip the file if necessary.

Upvotes: 5

Craig
Craig

Reputation: 2366

I got @RaGe's answer working, but I had to adapt it since the ivy layout method has been depreciated see https://docs.gradle.org/current/dsl/org.gradle.api.artifacts.repositories.IvyArtifactRepository.html#org.gradle.api.artifacts.repositories.IvyArtifactRepository:layout(java.lang.String,%20groovy.lang.Closure)

So to get it working I had to adjust it to this for a tomcat keycloak adapter:

ivy {
    url 'https://downloads.jboss.org/'
    patternLayout {
        artifact '/[organization]/[revision]/adapters/keycloak-oidc/[module]-[revision].[ext]'
    }
}

dependencies {
    // https://downloads.jboss.org/keycloak/4.8.3.Final/adapters/keycloak-oidc/keycloak-tomcat8-adapter-dist-4.8.3.Final.zip
    compile "keycloak:keycloak-tomcat8-adapter-dist:$project.ext.keycloakAdapterVersion@zip"
}

task unzipKeycloak(type: Copy) {

    def zipPath = project.configurations.compile.find {it.name.startsWith("keycloak") }
    println zipPath
    def zipFile = file(zipPath)
    def outputDir = file("${buildDir}/tomcat/lib")

    from zipTree(zipFile)
    into outputDir
}

Upvotes: 1

Matthias M
Matthias M

Reputation: 14830

Unzipping using the copy task works like this:

task unzip(type: Copy) {
  def zipFile = file('src/dists/dist.zip')
  def outputDir = file("${buildDir}/unpacked/dist")

  from zipTree(zipFile)
  into outputDir
}

http://mrhaki.blogspot.de/2012/06/gradle-goodness-unpacking-archive.html

Upvotes: 7

Peter Niederwieser
Peter Niederwieser

Reputation: 123910

There isn't currently a Gradle API for downloading from a URL. You can implement this using Ant, Groovy, or, if you do want to benefit from Gradle's dependency resolution/caching features, by pretending it's an Ivy repository with a custom artifact URL. The unzipping can be done in the usual Gradle way (copy method or Copy task).

Upvotes: 9

Related Questions