How to build/test code in a local repo while continuing to modify it at the same time

Question

I'm working in a repo where the build takes hours. I'd like to do a build of that code while I develop the next part of the functionality (in another repo which includes the changes being built).

I can't decompose the task. It might help to think of the "additional code" as the unit tests. Responses about how badly factored this repository will not help me. It might be helpful to think of my code as refactoring to make it more modular.

Currently, I'm being murdered by the cyclical time to make changes and build them.

Is there a way to setup two local git repos and base the second one on the checked-in code of the first?

I'd like to develop in this pattern:

1. develop code in repo 1
2. build repo 1 (clock starts on a 2 hour build)
3. in parallel, pull changes from repo 1 to repo 2
4. in parallel, make further changes in repo 2 that don't break the ongoing build in repo 1
5. in parallel, complete code in repo 2
6. build completes in repo 1
7. pull changes from repo 2 to repo 1

What are my options to accomplish this?

I'm not married to the steps above, but I'm interested in decreasing the time I must spend not developing code while an incredibly long build runs.

Answer 1

You have a few options depending on your situation and needs. Options 1 and 2 work entirely on your local machine, while option 3 works via a git server.

option 1: Make a local clone

git clone /path/to/source-repo-dir /path/to/new-repo-dir will make a local clone. From git help clone:

When the repository to clone from is on a local machine, this flag bypasses the normal "Git aware" transport mechanism and clones the repository by making a copy of HEAD and everything under objects and refs directories. The files under .git/objects/ directory are hardlinked to save space when possible.

Even though the repo you are cloning is local, it will be configured as a remote, and you will pull and push to it just as you would to a git server such as GitHub.

To make it easier to keep straight in your head, you can use the --origin option with git clone to give its local remote a name other than the default "origin", especially if you are used to using or are already using that name for an actual upstream remote repo.

I would probably use the downstream clone for builds and do my dev on the upstream repo, but you can do it the other way if you prefer. You can even make the relationship symmetrical by adding the new repo as a remote in your original repo (so that they are both remotes to each other).

See also git help clone for info, e.g. about the --no-hardlinks and --shared options, but I recommend sticking with the defaults.

---

option 2: Use git worktree

This option allows you to have one local repo but have two or more separate worktrees.

The main limitation of this approach is that no two worktrees can have the same branch checked out. This may not work for you if your continuous integration worktree needs to be on the same branch as your dev worktree. For more details and a workaround, see this answer.

See Multiple working directories with Git? for more info.

---

option 3: via central remote (e.g. a private git server or GitHub)

If you are pushing your code to a remote "upstream" repo, simply make a new clone of it. This works exactly like option 1, except that synchronizing changes between your two local clones requires the additional step of pushing to and pulling from the upstream repo. If you are doing this anyway, or if you are working on a team and you want your teammates to see the new code even before your long build is done, this is the way to go.

This is how teams do continuous integration. They usually have dedicated machine that monitors the central repo and kicks off continuous integration builds and tests whenever a branch that it is monitoring is updated.

You might also be able to take advantage of continuous integration services (e.g. Travis on GitHub).