Subversion update externals to a date

Question

I'm working on a large, established project under SVN control. Many parts of the code base are being checked out as externals, but are being actively worked on by other people.
I wanted to update my entire working copy, externals and all, so that it reflected the various repositories' HEADs at a specific point in time. My initial attempt was:

svn up -r'{20090324}'

This updates the current directory to the specified date, but updates all externals to the current date. Updating the externals one at a time works as expected.
I understand that due to the nature of externals, a single update couldn't work with a revision number, but why doesn't it work with a date?
What's the best way to achieve the point-in-time effect that I'm looking for, without having to maintain a script that hard-codes the various externals?

I'm running a Linux system.

Answer 1

I like the windows solution:

forfiles /m .svn /s /c "cmd /c svn up -r {2010-08-30}"

Unfortunately, it does not work for files with extern definitons.

But the commands returns a message for each external file, so, it may be handled by hand:

Fetching external item into 'img\button_trigger.svg':
External at revision 8173.

Thank you all, Harald

Answer 2

This is inefficient, in that it calls svn update more often than (usually) required. Otherwise, it is short an sweet:

Unix:

find . -name .svn -execdir svn update -r {2010-08-30} \;

Windows:

forfiles /m .svn /s /c "cmd /c svn up -r {2010-08-30}"

Answer 3

Based on Dave Cohen's answer, but quite a bit faster:

find . -name '.svn' -execdir svn up --ignore-externals -r '{2014-02-05}' \;

Answer 4

Let svn do the recursion for you.

The tempfile and tee are only here so that you can see the full output:

SVN_UP_OUTPUT=$(mktemp SVN_UP_OUTPUT.XXXXX)
svn up -r$REVISION | tee $SVN_UP_OUTPUT
cat $SVN_UP_OUTPUT | egrep '^Fetching external' | egrep -o "'.*'" | sed -e "s/'//g" | while read DIR;do
    echo $$ svn up -r$REVISION "$DIR"
    svn up -r$REVISION "$DIR"
done
rm $SVN_UP_OUTPUT

If you don't care about the output, it can be shortened to this:

svn up -r$REVISION | egrep '^Fetching external' | egrep -o "'.*'" | sed -e "s/'//g" | while read DIR;do
    svn up -r$REVISION "$DIR"
done

And of course, in your case:

REVISION='{20090324}'

Answer 5

The problem here is that you and your colleagues have not been using explicit revision numbers with your externals. You should begin at once!

Intuitively I might have thought that checking out something from a specific date would recursively "peg" the externals look-up to that date, even where the externals point to some HEAD revision. But, well, it doesn't.

Answer 6

As I have recently had a similar problem, I wrote a little script to checkout a repository at a specific revision while also checking out the externals at the date of that revision: https://gist.github.com/3413952

Useful if you need to find the source of a bug and want to have something similar to git's bisect feature.

Answer 7

When using svn:externals, it is generally a bad idea to use an external without a revision number. It means that it becomes hard to correlate the version of the external with the version of the containing project; I know this the hard way, from trying to track down some history in a project that contained externals, and I would have to guess which revision corresponded to the revision in the containing project (sometimes it was earlier because someone had updated the external project and then updated the containing project, sometimes it was later because someone had edited files directly in the external checkout and then committed it).

Instead, as suggested by the tip box a couple paragraphs into the externals section in the subversion book, you should always commit externals with a revision number. That way, whenever you check out a particular revision of the containing project, the appropriate revision of the external will also be checked out. It does mean a little more work, as you have to update the revision number in the svn:externals property every time (we wrote a script to do it automatically), but in the long run it is a much better solution.

edit: Here is the skeleton of the script we used (a rake task) for conveniently updating the external and keeping everything in sync.

desc 'Update external for this project (rake update_external r=17789)'
task :update_external do |t|
  rev = ENV['r']
  rev =~ /^\d+$/ or raise "Invalid SVN revision number: r=<#{rev}>"

  # Update the project.
  sh "svn update"

  URL = 'svn+ssh://example.com/external/trunk'
  sh "svn propset svn:externals 'external -r#{rev} #{URL}' containing/directory"

  # Update again -- to put the externals back to the right revision.
  sh "svn update"
end

Answer 8

nice utility that will freeze externals given a path. we use this util to freeze externals after we create a tag from the trunk:

http://svnxf.codeplex.com/

Answer 9

This is the best solution to the problem I found to date (it's a tricky problem - subversions devs should fix it in the core). This example deals with mplayer in particular but you should easily see the logic.

; fetch the rev I want without including the externals
svn checkout -r "$REV" --ignore-externals \
    svn://svn.mplayerhq.hu/mplayer/trunk

; grab the date of that rev from the svn info output
DATE=`svn info trunk|sed -n '/^Last Changed Date/s/.*: \(.*\) (.*/\1/p'`

; fetch the externals using that date
svn checkout -r "{$DATE}" \
        svn://svn.mplayerhq.hu/ffmpeg/trunk/libavutil \
        svn://svn.mplayerhq.hu/ffmpeg/trunk/libavformat \
        svn://svn.mplayerhq.hu/ffmpeg/trunk/libavcodec \
        svn://svn.mplayerhq.hu/ffmpeg/trunk/libpostproc

Answer 10

I still haven't got a perfect solution, but this one comes close:

svn propget svn:externals | sed -e 's/ .*$//g' | xargs svn up -r'{20090324}'

This works in my case because there's no recursive externals, and all externals are defined with no spaces in the directory or a revision number, so the regular expression can easily chop off the trailing repository path.

I'm sure there's better regex that will solve the problem generically, though.

Edit: Actually the more I think about this, the more problems I see. The biggest of which is that it's using the svn:externals from the current version, rather than the svn:externals of the version at the specified date. This is even more complex than I first thought.

Answer 11

This is tricky, and I'm afraid I can't offer a good solution to your current situation - but Brian has given the answer on how to avoid it.

The avoidance comes down to a little bit of repository theory - basically it must not be possible to modify any source code for your project without a corresponding revision appearing in trunk.

By pointing all externals to tags or specific revisions, no changes from them can appear in the main project history without committing a change to the external reference. But if you point an external to a moving trunk, a change to the external will not show up in the main project's timeline at all - leaving you in the position you're in.

Personally, I've taken the view that externals should be treated and released as independent projects, hence all externals point to tags. During heavy parallel development, it's fine to 'switch' an external to trunk, or to have an unstable development branch temporarily pointing to an external trunk, but the mainline project trunk always points to a stable external, and it's a conscious decision to upgrade. This view may be overkill for your situation, but it's worth seeing other possibilities.

Answer 12

The revision/date/etc you're updating the main WC to does not get passed through to the externals when they are being updated. In the absence of a specific revision specified in the externals definition, they will always track the head of whatever they point at. If you specify a revision there, then that's the only revision you'll ever get. I'm pretty sure what you're trying to do is impossible -- it's an approach I tried to use to solve a problem I was having, as I describe in this question. (I never did solve that problem, though I think the proxy idea that's mentioned there could do it. It probably won't help you though)

Answer 13

KSvn might have this feature. http://subversion.tigris.org/links.html#desktop-integrations