CODEWITHSUNDEEP
git
Axonn
Axonn

Reputation: 10346

Would "splitting & stitching" work to manage GIT repos with lots of files?

We have a project that has seen massive amount of refactoring through the years. We almost never look back more than 2 months ago, other than for fun, but keeping a history of the project is nevertheless important. The problem is that this has started to cost us a lot of time when doing some daily transfers using a certain cloud provider.

For this reason, I'd like to archive the entire project, including the GIT repo and start a brand new repo. Or perhaps just wipe everything that's older than 2 months and continue from there.

Easily done so far. But what if after 2 years I want to repeat the procedure with backing up all history on top of the old history.

To illustrate:

  1. Now: archive .git with old repo 2014-2017
  2. Later: archive .git with repo from 2017-2020 but do it with continuity with the old repo 2014-2017, as if nothing happened.

The purpose is to keep the project directory nimble and easy to transfer around, but not lose the history. If we want, we should always be able to access the full history somewhere, in a contiguous fashion (not separate archives).

Other suggestions are welcomed.

Upvotes: 0

Views: 39

Answers (2)

Axonn
Axonn

Reputation: 10346

Another answer: git-gc. This actually reduces the number of files considerably (it dropped from 1100 to 50 using the --aggressive option.

https://git-scm.com/docs/git-gc

Upvotes: 0

Caleb
Caleb

Reputation: 125037

The purpose is to keep the project directory nimble and easy to transfer around, but not lose the history. If we want, we should always be able to access the full history somewhere, in a contiguous fashion (not separate archives).

So let's say that your existing repository is called BigRepo. For what you want, I think you could just create a clone called NewRepo and retire BigRepo from daily use -- lock down access so that nobody can push to BigRepo directly and only a few people can merge. Next, remove most of the old commits (say, anything over two months old) from NewRepo, and have everybody start using NewRepo.

That gives you the much smaller repo that changes daily that you want, and you still have all the old commits safely stored in BigRepo. From time to time, you can make a pull request from NewRepo to BigRepo, so that all the new commits are copied up to BigRepo and you maintain the continuous history that you want. You'll only have an absolutely complete history in BigRepo immediately after you merge the newest commits from BigRepo, but you can merge the newest into BigRepo anytime you need that. And the whole point here is to batch up the changes to BigRepo so that your daily backups don't take forever.

Upvotes: 1

Related Questions