CODEWITHSUNDEEP
svnsvnadmin
Stewart
Stewart

Reputation: 4982

Splitting SVN repositories

I have a giant VisualSVN repository with the following structure

/common
└── ^/trunk
    ├── proj1
    │   └── CMakeLists.txt
    ├── proj2
    │   └── CMakeLists.txt
    └── projN
        └── CMakeLists.txt

5 years, and 19,000 commits later ... I want branching that actually works. So I'd like to split that repository into several smaller ones

/proj1
├── ^/branches
└── ^/trunk
    └── CMakeLists.txt
/proj2
├── ^/branches
└── ^/trunk
    └── CMakeLists.txt
/projN
├── ^/branches
└── ^/trunk
    └── CMakeLists.txt

How do I do this?

I'm happy to use client-commands svn mv to get the structure right, but the biggest thing is filtereing out all of content I don't want.

I've started by using svnadmin to create a 4GB dump file of the full repository

D:\Repositories> svnadmin dump D:\Repositories\common > common.dump
* Dumped revision 0.
* Dumped revision 1. 
...
* Dumped revision 18968.
* Dumped revision 18969.

Then I'm using svndumpfilter to reduce this to only the revisions related to the project of interest:

D:\Repositories> svndumpfilter  \
    --drop-empty-revs           \
    --renumber-revs             \
    include trunk/proj1         \
    < common.dump > proj1.dump
Including (and dropping empty revisions for) prefixes: 
    '/trunk/proj1'
Revision 0 committed as 0.
Revision 1 skipped.
Revision 2 skipped.
...
Revision 561 committed as 1.
...
Revision 669 committed as 2.
...
Revision 13514 committed as 36.
svndumpfilter: E200003: Invalid copy source path: '/trunk/proj2/file' for `/trunk/proj1/file`

This failed because there was an svn mv ^/trunk/proj2/file ^/trunk/proj1/file performed at some point. I can deal with that by simply adding another include:

D:\Repositories> svndumpfilter             \
    --drop-empty-revs                      \
    --renumber-revs                        \
    include                                \
        trunk/proj1                        \
        trunk/proj2/file                   \
    < common.dump > proj1.dump
...
Revision 13515 comitted as 38.
svndumpfilter: E200003: Missing merge source path '/branches/proj1-tests'; try with --skip-missing-merge-sources

Getting closer, and I don't really need merge history, so I'll do what's recommended and add --skip-missing-merge-sources

D:\Repositories> svndumpfilter             \
    --drop-empty-revs                      \
    --renumber-revs                        \
    --skip-missing-merge-sources           \
    include                                \
        trunk/proj1                        \
        trunk/proj2/file                   \
    < common.dump > proj1.dump

Success! In reality, I have about 100 things that I've included. Now I'd like to use this to create that new repository:

D:\Repositories> svnadmin create proj1
D:\Repositories> svnadmin load --ignore-uuid proj1 < proj1.dump
<<< Started new transaction, based on original revision 1
    * editing path : trunk/proj2/file 
    ...svnadmin: E160013: File not found: transaction '0-0', path '/trunk/proj2/file'

And now I'm stuck. How do I deal with that?

Upvotes: 1

Views: 35

Answers (2)

bahrep
bahrep

Reputation: 30662

Repository history filtering is a rather complicated admin operation, and you should only consider it when other options are unavailable, or when you are absolutely sure they won't work for you.

Unless I'm missing something now, I think that branching and merging should work fine with your current repository layout. But if you want to change the layout to a different one anyway, it's typically possible without having to "split" the repository into multiple ones. You can use your SVN client to move the directories and create new ones to get the layout you want.

BTW, up to date versions of svnadmin dump have the --include, --exclude and --pattern options that can help you "split" the history of your repositories into multiple ones (check the help by running svnadmin help dump, and these options are mentioned in version 1.10 release notes).

Upvotes: 0

Stewart
Stewart

Reputation: 4982

The problem is the revision which created trunk/proj2 isn't included. You could add include trunk/proj2, but you could end up adding the whole tree and that just defeats the purpose of filtering.

Then you could spend time refining your filters, but there's a script deployed with svnadmin that already does that: svnrdump.

D:\Repositories> svnrdump dump file:///Repositories/common/trunk/proj1 > proj1.dump
D:\Repositories> svnadmin create proj1
D:\Repositories> svnadmin load proj1 < proj1.dump
D:\Repositories> del proj1.dump

Or using piping:

D:\Repositories> svnadmin create proj1
D:\Repositories> svnrdump dump file:///Repositories/common/trunk/proj1 | svnadmin load proj1

Then checkout a working copy and svn mv as appropriate.

D:\Temp> svn checkout file:///Repositories/proj1 proj1.wc
D:\Temp> cd proj1.wc
D:\Temp> svn move trunk/proj1/* trunk/
D:\Temp> svn remove trunk/proj1
D:\Temp> svn commit

There might be a way to include --drop-empty-revs --renumber-revs, but I haven't found it yet. I tried the following but it didn't filter out very much:

svnrdump dump file:///Repositories/common/trunk/proj1 > proj1.dump
svndumpfilter --drop-empty-revs --renumber-revs include / < proj1.dump > proj1-filtered.dump
svnadmin load proj1 < proj1-filtered.dump

Upvotes: 1

Related Questions