How can I download Maven artifacts within a plugin?

Question

I have a Maven plugin that takes a groupId, artifactId, and version in its confiugration.

I want to be able to download that artifact from the remote repositories and copy the file into the project. I can't figure out how to download the artifact though.

I understand that I can resolve dependencies using the dependency plugin, but I need it to happen inside my plugin. How can I do this?

Answer 1

I'm facing similar issue. I want to extract specific files from plugin's dependencies in order not to pollute main artifact's dependencies.

The solution using "old" Maven API is obvious and straightforward. Except the deprecation.

I decided to investigate new ways to do it using new API.

My main inspiration was Maven Dependency Plug-In especially GetMojo, UnpackDependenciesMojo and UnpackUtil.

In short, what has to be done:

1. Create org.apache.maven.shared.transfer.dependencies.DependableCoordinate which points to desired GAV. Nothing special here; use org.apache.maven.shared.transfer.dependencies.DefaultDependableCoordinate POJO.
2. Resolve the artifact. This is probably the most complex task. We need to set-up repositories, which can be obtained from Project's remote repositories and injecting proxies and authorization stuff. With this done we can use org.apache.maven.shared.transfer.dependencies.resolve.DependencyResolver to do the job.
3. Extracting resolved artifact. org.codehaus.plexus.archiver.UnArchiver is our friend. To get an instance, we can use org.codehaus.plexus.archiver.manager.ArchiverManager#getUnArchiver(). There are two options: by type or by file extension.

I developed/tested the following code with Maven 3.8.6 and Maven 3.9.1.

Common injection

    @Parameter(defaultValue = "${project.build.directory}/download", required = true)
    private File outputDirectory;

    @Parameter
    private List<String> includes = new ArrayList<>();

    @Inject
    private ArchiverManager archiverManager;

    @Inject
    private RepositorySystem repositorySystem;

    @Inject
    private DependencyResolver dependencyResolver;

    @Parameter(defaultValue = "${plugin}", readonly = true, required = true)
    private PluginDescriptor pluginDescriptor;

    @Parameter(defaultValue = "${project}", readonly = true, required = true)
    private MavenProject project;

    @Parameter(defaultValue = "${session}", required = true, readonly = true)
    private MavenSession session;

Creation of repositories for DependencyResolver

    private static List<ArtifactRepository> createRepositories(
            final MavenProject project,
            final MavenSession session,
            final RepositorySystem repositorySystem) {
        final List<ArtifactRepository> repositories = new ArrayList<>();

        final var policy = new ArtifactRepositoryPolicy(
                true,
                ArtifactRepositoryPolicy.UPDATE_POLICY_ALWAYS,
                ArtifactRepositoryPolicy.CHECKSUM_POLICY_FAIL);

        project.getRemoteArtifactRepositories().forEach(
                remoteRepository -> repositories.add(
                        new MavenArtifactRepository(
                                remoteRepository.getId(),
                                remoteRepository.getUrl(),
                                remoteRepository.getLayout(),
                                policy,
                                policy)));

        final var settings = session.getSettings();
        repositorySystem.injectMirror(repositories, settings.getMirrors());
        repositorySystem.injectProxy(repositories, settings.getProxies());
        repositorySystem.injectAuthentication(repositories, settings.getServers());

        return repositories;
    }

Resolving DependableCoordinate

    private static Iterable<ArtifactResult> resolveArtifact(
            final DependableCoordinate artifact,
            final List<ArtifactRepository> repositories,
            final MavenSession session,
            final DependencyResolver dependencyResolver)
            throws DependencyResolverException {
        final ProjectBuildingRequest buildingRequest
                = new DefaultProjectBuildingRequest(session.getProjectBuildingRequest());
        buildingRequest.setRemoteRepositories(repositories);
        return dependencyResolver.resolveDependencies(buildingRequest, artifact, null);
    }

The extraction itself

    private static void extract(
            final Artifact artifact,
            final File directory,
            final FileSelector[] selectors,
            final ArchiverManager archiverManager)
            throws IOException, MojoFailureException {
        try {
            if (!directory.exists()) {
                if (!directory.mkdirs()) {
                    throw new IOException(String.format("Failed to create directory [%s].", directory));
                }
            }

            UnArchiver unArchiver;
            try {
                unArchiver = archiverManager.getUnArchiver(artifact.getType());
            } catch (final NoSuchArchiverException ex) {
                unArchiver = archiverManager.getUnArchiver(artifact.getFile());
            }

            unArchiver.setFileSelectors(selectors);
            unArchiver.setSourceFile(artifact.getFile());
            unArchiver.setDestDirectory(directory);
            unArchiver.extract();
        } catch (final NoSuchArchiverException ex) {
            throw new MojoFailureException(
                    String.format("Could not determine archive type for artifact [%s].", artifact),
                    ex);
        }
    }

And finally the control loop

    private static final Logger LOGGER = LoggerFactory.getLogger(DependencyDownloadMojo.class);

    private static final List<String> DEFAULT_INCLUDES = Collections.singletonList("*.txt");

    @Override
    public void execute() throws MojoExecutionException, MojoFailureException {
        LOGGER.debug("About to download resources from dependencies.");

        final var repositories = createRepositories(project, session, repositorySystem);
        final var dependencies = getPluginDependencies(pluginDescriptor.getPlugin());
        final var selectors = createSelectors(includes);

        LOGGER.info("Using resource selectors: [{}].", (Object) selectors);

        for (final var dependency : dependencies) {
            try {
                LOGGER.info("About to download resources from dependency [{}].", dependency);

                LOGGER.debug("About to resolve Artifact [{}].", dependency);
                final var artifactResults = resolveArtifact(dependency, repositories, session, dependencyResolver);
                if (LOGGER.isDebugEnabled()) {
                    LOGGER.debug("Dependency resolves to [{}].", artifactResults);
                    for (final var artifactResult : artifactResults) {
                        LOGGER.debug("Dependency resolves to [{}]; location: [{}].",
                                artifactResult.getArtifact(),
                                artifactResult.getArtifact().getFile());
                    }
                }

                for (final var artifactResult : artifactResults) {
                    final var artifact = artifactResult.getArtifact();
                    LOGGER.info("About to extract resources from artifact [{}].", artifact);

                    try {
                        extract(artifact, outputDirectory, selectors, archiverManager);
                    } catch (final IOException ex) {
                        final var message = String.format(
                                "Failed to extract resources from artifact [%s].", artifact);
                        LOGGER.error(message, ex);
                        throw new MojoExecutionException(message, ex);
                    }
                }
            } catch (final DependencyResolverException ex) {
                final var message = String.format("Failed to resolve dependency [%s].", dependency);
                LOGGER.error(message, ex);
                throw new MojoFailureException(message, ex);
            }
        }
    }

    private FileSelector[] createSelectors(final List<String> includes) {
        final List<FileSelector> result = new ArrayList<>();

        final List<String> includesOrDefaults = new ArrayList<>();

        if (includes.isEmpty()) {
            LOGGER.debug("Using default includes [{}].", DEFAULT_INCLUDES);
            includesOrDefaults.addAll(DEFAULT_INCLUDES);
        } else {
            LOGGER.debug("Using provided includes [{}].", includes);
            includesOrDefaults.addAll(includes);
        }

        for (final var include : includesOrDefaults) {
            final var selector = new IncludeExcludeFileSelector();
            selector.setIncludes(new String[] { include });
            result.add(selector);
        }

        return result.toArray(new FileSelector[0]);
    }

The most important—POM

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

    <modelVersion>4.0.0</modelVersion>

    <groupId>global.sandbox.test.dependency.download.maven.plugin</groupId>
    <artifactId>dependency-download-maven-plugin</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>maven-plugin</packaging>

    <parent>
        <groupId>global.sandbox.test.dependency.download</groupId>
        <artifactId>dependency-download</artifactId>
        <version>1.0-SNAPSHOT</version>
    </parent>

    <properties>
        <maven.version>3.8.2</maven.version>
        <maven-plugin-plugin.version>3.8.2</maven-plugin-plugin.version>
        <sisu-maven-plugin.version>0.9.0.M2</sisu-maven-plugin.version>
        <plexus-archiver.version>4.7.1</plexus-archiver.version>
        <maven-artifact-transfer.version>0.13.1</maven-artifact-transfer.version>
    </properties>

    <prerequisites>
        <maven>3.8.2</maven>
    </prerequisites>

    <dependencies>
        <dependency>
            <groupId>org.apache.maven</groupId>
            <artifactId>maven-core</artifactId>
            <version>${maven.version}</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.maven</groupId>
            <artifactId>maven-artifact</artifactId>
            <version>${maven.version}</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.maven</groupId>
            <artifactId>maven-model</artifactId>
            <version>${maven.version}</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.maven</groupId>
            <artifactId>maven-plugin-api</artifactId>
            <version>${maven.version}</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.maven.plugin-tools</groupId>
            <artifactId>maven-plugin-annotations</artifactId>
            <version>${maven.version}</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.codehaus.plexus</groupId>
            <artifactId>plexus-archiver</artifactId>
            <version>${plexus-archiver.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.maven.shared</groupId>
            <artifactId>maven-artifact-transfer</artifactId>
            <version>${maven-artifact-transfer.version}</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <compilerArgs>-Xlint:all</compilerArgs>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-plugin-plugin</artifactId>
                <version>${maven-plugin-plugin.version}</version>
                <configuration>
                    <skipErrorNoDescriptorsFound>false</skipErrorNoDescriptorsFound>
                    <extractors>
                        <extractor>java-annotations</extractor>
                    </extractors>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.eclipse.sisu</groupId>
                <artifactId>sisu-maven-plugin</artifactId>
                <version>${sisu-maven-plugin.version}</version>
                <executions>
                    <execution>
                        <id>index-project</id>
                        <goals>
                            <goal>main-index</goal>
                            <goal>test-index</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

</project>

Parent POM

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

    <modelVersion>4.0.0</modelVersion>

    <groupId>global.sandbox.test.dependency.download</groupId>
    <artifactId>dependency-download</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>pom</packaging>

    <modules>
        <module>dependency</module>
        <module>dependency-download-maven-plugin</module>
        <module>usage</module>
    </modules>

    <properties>
        <maven.compiler.release>17</maven.compiler.release>
        <maven.compiler.target>${maven.compiler.release}</maven.compiler.target>
        <maven.compiler.source>${maven.compiler.release}</maven.compiler.source>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

        <maven-compiler-plugin.version>3.11.0</maven-compiler-plugin.version>
        <maven-resources-plugin.version>3.3.1</maven-resources-plugin.version>
        <maven-enforcer-plugin.version>3.3.0</maven-enforcer-plugin.version>
    </properties>

    <build>
        <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>${maven-compiler-plugin.version}</version>
                    <configuration>
                        <skip>true</skip>
                    </configuration>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-resources-plugin</artifactId>
                    <version>${maven-resources-plugin.version}</version>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-enforcer-plugin</artifactId>
                    <version>${maven-enforcer-plugin.version}</version>
                </plugin>
            </plugins>
        </pluginManagement>

        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-enforcer-plugin</artifactId>
                <executions>
                    <execution>
                        <id>enforce-maven</id>
                        <goals>
                            <goal>enforce</goal>
                        </goals>
                        <configuration>
                            <rules>
                                <requireMavenVersion>
                                    <version>3.8.0</version>
                                </requireMavenVersion>
                            </rules>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

</project>

Answer 2

Your plugin needs to create an Artifact using the ArtifactFactory and the groupId, artifactId and version of the artifact to be bootstrapped, then pass that artifact to an ArtifactResolver to handle the discovery/download.

The resolver needs access to the local repository and remote repositories. The good news is that all those are plexus components that you can declare as dependencies in your Mojo and have Plexus wire them in for you.

In another answer I showed how to do this. In your case you need a cut-down version with slightly different parameters to read the groupId, artifactId and version. In the plugin below, the various components are declared as plexus components, and the properties to declare the groupId, artifactId, version, and packaging type.

package name.seller.rich.maven.plugins.bootstrap;

import java.util.List;

import org.apache.maven.artifact.Artifact;
import org.apache.maven.artifact.factory.ArtifactFactory;
import org.apache.maven.artifact.repository.ArtifactRepository;
import org.apache.maven.artifact.resolver.ArtifactNotFoundException;
import org.apache.maven.artifact.resolver.ArtifactResolutionException;
import org.apache.maven.artifact.resolver.ArtifactResolver;
import org.apache.maven.plugin.AbstractMojo;
import org.apache.maven.plugin.MojoExecutionException;
import org.apache.maven.plugin.MojoFailureException;

/**
 * Obtain the artifact defined by the groupId, artifactId, and version
 * from the remote repository.
 * 
 * @goal bootstrap
 */
public class BootstrapAppMojo extends AbstractMojo {

    /**
     * Used to look up Artifacts in the remote repository.
     * 
     * @parameter expression=
     *  "${component.org.apache.maven.artifact.factory.ArtifactFactory}"
     * @required
     * @readonly
     */
    protected ArtifactFactory factory;

    /**
     * Used to look up Artifacts in the remote repository.
     * 
     * @parameter expression=
     *  "${component.org.apache.maven.artifact.resolver.ArtifactResolver}"
     * @required
     * @readonly
     */
    protected ArtifactResolver artifactResolver;

    /**
     * List of Remote Repositories used by the resolver
     * 
     * @parameter expression="${project.remoteArtifactRepositories}"
     * @readonly
     * @required
     */
    protected List remoteRepositories;

    /**
     * Location of the local repository.
     * 
     * @parameter expression="${localRepository}"
     * @readonly
     * @required
     */
    protected ArtifactRepository localRepository;

    /**
     * The target pom's artifactId
     * 
     * @parameter expression="${bootstrapArtifactId}"
     * @required
     */
    private String bootstrapArtifactId;

    /**
     * The target pom's groupId
     * 
     * @parameter expression="${bootstrapGroupId}"
     * @required
     */
    private String bootstrapGroupId;

    /**
     * The target pom's type
     * 
     * @parameter expression="${bootstrapType}"
     * @required
     */
    private String bootstrapType;

    /**
     * The target pom's version
     * 
     * @parameter expression="${bootstrapVersion}"
     * @required
     */
    private String bootstrapVersion;

    public void execute() throws MojoExecutionException, MojoFailureException {
        try {
            Artifact pomArtifact = this.factory.createArtifact(
                bootstrapGroupId, bootstrapArtifactId, bootstrapVersion,
                "", bootstrapType);

            artifactResolver.resolve(pomArtifact, this.remoteRepositories,
                this.localRepository);
        } catch (ArtifactResolutionException e) {
            getLog().error("can't resolve parent pom", e);
        } catch (ArtifactNotFoundException e) {
            getLog().error("can't resolve parent pom", e);
        }
    }
}

This is an example of a pom configured to use the plugin (and download the aspectjrt 1.6.4 pom):

<?xml version="1.0" encoding="UTF-8"?>
<project>
  <modelVersion>4.0.0</modelVersion>
  <groupId>name.seller.rich</groupId>
  <artifactId>bootstrap-test</artifactId>
  <version>1.0.0</version>
    <build>
      <plugins>
        <plugin>
          <groupId>name.seller.rich</groupId>
          <artifactId>maven-bootstrap-plugin</artifactId>
          <executions>
            <execution>
              <phase>package</phase>
              <goals>
                <goal>bootstrap</goal>
              </goals>
              <configuration>
                <bootstrapGroupId>org.aspectj</bootstrapGroupId>
                <bootstrapArtifactId>aspectjrt</bootstrapArtifactId>
                <bootstrapVersion>1.6.4</bootstrapVersion>
                <bootstrapType>pom</bootstrapType>
              </configuration>
            </execution>
          </executions>
        </plugin>
    </plugins>
  </build>
</project>