Reputation: 1140

Wiping out Maven local repository on build machine

On a CI build server, the local Maven repository fills up the file system repetitively (after a few days). What strategy are others doing to trim the local repository in such a case? -Max

Upvotes: 43

Answers (6)

cemonds

Reputation: 225

We use especially for this purpose the build-helper plugin. In our company parent pom is the remove-project-artifact goal embedded in the profile for our hudson builds. This way all old versions of this artifact are removed prior to installing the currently build version.

...
<profile>
  <id>hudson</id>
  <activation>
    <property>
      <name>BUILD_TAG</name>
    </property>
  </activation>
  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>build-helper-maven-plugin</artifactId>
        <version>1.7</version>
        <executions>
          <execution>
            <id>remove-old-artifacts</id>
            <phase>package</phase>
            <goals>
              <goal>remove-project-artifact</goal>
            </goals>
            <configuration>
              <removeAll>true</removeAll>
            </configuration>
          </execution>
        </executions>
      </plugin>
  ...

Using removeAll set to true will wipe out all other snapshots except the one your working on. This can be dangerous as it may mean snapshots for a branch will be wiped out as well.

For instance if you have a snapshot 1.0.0.18-SNAPSHOT representing HEAD and snapshot 1.0.1.17-SNAPSHOT representing a branch, running this plugin with 1.0.0.18-SNAPSHOT build will wipe the 1.0.1.17-SNAPSHOt folder.

To get around this scenario the removeAll should be set to false.

Upvotes: 3

Steven Christenson

Reputation: 11

We have employed a slightly different (and devious) technique. All artifacts that build "large things" (EARs, WARs, TARs) have their deploy location overriden like so:

<properties>
   <discard-me-in-bit-bucket>file://${basedir}/target/_DELETEME</discard-me-in-bit-bucket> 
</properties>

<distributionManagement>
  <repository>
    <id>upload-InternalSite</id>
    <name>SoftwareLibrary External</name>
    <url>${discard-me-in-bit-bucket}</url>
    <layout>legacy</layout>
    <uniqueVersion>false</uniqueVersion>
  </repository>
  <snapshotRepository>
    <id>upload-InternalSite</id>
    <name>Repository Name</name>
    <url>${discard-me-in-bit-bucket}</url>
    <layout>legacy</layout>
    <uniqueVersion>false</uniqueVersion>
  </snapshotRepository>
</distributionManagement>

This strategy causes the deploy goal to put things in the target directory, which of course is destroyed by the next CLEAN operation. To get even more aggressive, we have a postbuild step that does this:

find -type d -name '*_DELETEME' -exec rm -rf '{}' ';' -prune || echo $?

We employ yet one more strategy, too. In Hudson/Jenkins we provide a settings file to place the .m2 repository in the workspace for the job. This allows us to delete the entire repository before or after the job. It also makes artifacts visible in the workspace which aids in debugging some problems.

Upvotes: 1

Justin Searls

Reputation: 4866

In addition to purge-local-repository (which reads to me like a nuclear option, as it only offers an excludes configuration as opposed to an explicit includes), take a look at the Remove Project Artifact mojo. I'm looking to implement it now, as my exact use case is to clear out large WAR and EAR snapshots that are being built on my CI (and sometimes workstation) machines.

Upvotes: 4

Dominic Mitchell

Reputation: 12299

If you're using hudson, you can set up a scheduled job to just delete the entire repository once a day or something like that. I've got a job called hudson-maven-repo-clean which has this configuration:

Build / Execute shell: rm -rf ~hudson/.m2/repository
Build Triggers / Build periodically: 0 0 * * *

Upvotes: 19

Rich Seller

Reputation: 84038

The Maven dependency plugin has a purge-local-repository goal that allows you to delete the dependencies for a given project from the local repository, if this is run say once a day on each project the snapshots will not accumulate.

Alternatively there's a more scorched-earth approach you could take. As the problem is typically the timestamped snapshot artifacts, you could use the maven-antrun-plugin to delete all files that match the resource collection pattern.

For example (note this might need some tweaking as I've done it from memory):

<plugin>
  <artifactId>maven-antrun-plugin</artifactId>
  <executions>
    <execution>
      <phase>package</phase>
      <configuration>
        <tasks>
          <delete>
            <fileset dir="${settings.localRepository}">
              <include name="**/*.jar"/>
              <exclude name="**/*.pom"/>
              <exclude name="**/*.war"/>
              <exclude name="**/*.ear"/>
              <exclude name="**/*.md5"/>
              <exclude name="**/*.sha"/>
              <!--any other extensions?...-->
              <!--match the timestamp pattern-->
              <containsregexp expression="[0-9]{8}.[0-9]{6}-[0-9]+"/>
            </fileset>
          </delete>
        </tasks>
      </configuration>
      <goals>
        <goal>run</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Upvotes: 27

sal

Reputation: 23623

How big is the file system? We have 10gb allocated to builds and zap snapshots older than 30 days every night. That seems to work

Are you doing builds every X hours or when code changes? Switching to code changes will reduce the number of artifacts without reducing coverage.

Are you installing all snapshots locally? You don't need to do this in all cases. In most cases, just those snapshots that are actively developed dependancies need to be installed locally.

Are you installing EAR/WAR files locally? You probably don't need them either.

How many workspaces are you keeping? We use hudson and keep only the last 5 builds.

Upvotes: 0

Wiping out Maven local repository on build machine

Answers (6)

Related Questions