git merge without losing new files on merge-to branch

Question

I have two branches that have a common parent in the recent past. Lets call them feature-br-A and feature-br-B.

Development work has continued in parallel on both branches. feature-br-B has mostly bug fixes. feature-br-A has some bug fixes and at least one new feature.

Now as come the time to bring the bug fixes made in feature-br-B forward into feature-br-A.

git checkout feature-br-A
git merge feature-br-B
<< conflicts >>

The few file conflicts are simple and easy to resolve. But there are a whole bunch of files that were added only to feature-br-A that are marked for deletion. And there are a set of few more files only added to feature-br-A that are in conflict that marked 'deleted by them'.

How can I merge the changes made on feature-br-B into feature-br-A without losing all the changes made already on feature-br-A?

I tried

git merge -Xours --no-commit feature-br-B

but that created the same issue.

My other concern is if this happening to added files, is this also happening to the other files that were changed to add features to feature-br-A?

I suppose I can unstage the deletions from the merge, but it doesn't seem right that it should be deleting the new files at all.

Answer 1

Problem(1) How can I merge the changes made on feature-br-B into feature-br-A without losing all the changes made already on feature-br-A?

The problem causing the issue was that at some point some of the original commits made in feature-br-A were once merged into feature-br-B and then deleted by a subsequent commit. git is dutifully replicating the delete. It's not clear to me at the moment that if the original merge was reverted instead of file delete's that git would do the same thing, but I suspect that it would still be the case.

The error made seems to prevent automatically merging from feature-br-B into feature-br-A. The only suggestion I have to solve this is to identify the commits made on feature-br-B and move them into feature-br-A with git cherry-pick.

I plan on doing some more tests now that I can explain the deleted files from the merge.

Answer 2

git merge starts by finding the merge base between your current (or "local" or --ours) commit and your target (or "remote" or --theirs or "other") commit. In this case, your local commit is the tip-most commit on feature-br-A, and the "other" commit is the tip-most commit on feature-br-B.

I don't have enough information here to identify the merge base, but you (and your Git) do. It's the commit at which the two branches "join up". In some cases there may be more than one merge base commit, but probably there is just the one. If we draw a diagram of your two branches—note that git log --graph --decorate feature-br-A feature-br-B will draw this too, in a somewhat different form¹—we will see something like this:

...--o--*--o--o--o--o   <-- HEAD -> feature-br-A
         \
          o--o--o--o    <-- feature-br-B

Note that the two branch names point to the two commits that are being merged. The commit I marked with * is where the graph "joins up"; this is the merge base.

From the merge base, run two diffs

Having now identified the merge base commit, git merge does its thing by running two git diff commands. Simplified a bit, these are:

git diff base feature-br-A
git diff base feature-br-B

These two diffs give Git—and you—all the information needed to perform a normal merge (adding strategies or strategy options changes the merge, of course). They tell you what you did—what you changed from base to the tip of feature-br-A—and what they did: what "they", whoever they are, changed from base to the tip of feature-br-B.

Conflicts

Conflicts occur whenever Git itself cannot reconcile "what you did" with "what they did".

You mentioned some regular conflicts, and then:

The few file conflicts are simple and easy to resolve. But there are a whole bunch of files that were added only to feature-br-A that are marked for deletion. And there are a set of few more files only added to feature-br-A that are in conflict that marked 'deleted by them'.

Files added to feature-br-A (as compared to base) would not be marked for deletion. Those are part of "what you did". They are only in conflict if they are also added to feature-br-B (as compared to base).

Files deleted in feature-br-B (as compared to base) are also not in conflict, with one exception. Suppose that, for path P, you changed something in feature-br-A, so that comparing base and feature-br-A, file P is modified. Suppose further that in comparing base and feature-br-B, Git sees P as deleted. It cannot combine your change ("modify some lines") with theirs ("delete the entire file"). In this case it leaves you with your modified file checked-out but in "needs merge" state.

Technical stuff: slot numbers in the index / staging-area

In order to do the merge work, Git makes use of a special feature of the index / staging-area. Normally, the index contains just one entry per file (path) name: the state of the file as it will be in the next commit, once you make it.

That is, git commit starts by turning the staging-area into a tree—more precisely, a series of trees, one for each directory, with a single top-level Git tree object for the top of the work tree—of whatever files are staged. Then Git makes a commit object that refers to this tree object and updates the branch pointer. That completes the process of committing, and is why git commit is so fast: there's very little work to do as all the files are already in the staging area.

During a merge, however, Git treats the staging area differently. The index now contains three entries per path—or really, up to three. These entries are numbered: stage slot 1 is the version of the file that is in the merge base, slot 2 is the "local" or --ours version, and slot 3 is the other or --theirs version. If a file was created in one or both branch tips, slot 1 is empty. If a file was deleted in one branch tip, one of slots 2 or 3 is empty. (If the file is deleted in both tips, Git resolves everything for you and does not leave a conflicted entry.)

To resolve something, you git add (or git rm) the path after editing the work-tree file as needed. This empties out slots 1, 2, and 3, writing the file into slot 0—which is where normal, not-conflicted files live. Once there are no slot 1-to-3 entries, the merge is complete and you may commit the staging area.

More technical stuff: renames

Git attempts to guess at which file(s), if any, were renamed. It does this when doing the two git diff steps. If you run git diff manually yourself, you may need to tell Git to use the same "find renames" settings that it uses for merges. By default, Git uses the equivalent of a plain git diff -M (50% similarity). You can control the merge rename threshold via various options and git config values. Most of the time the defaults are fine and you can simply leave them alone.

It's worth remembering that Git is doing this kind of rename detection, though, regardless of whether you try tweaking the settings. This is important because it can show up in your conflicts. Suppose, for instance, that Git decides that lib/foo.py was renamed to misc/zap.py in the change-set from base to feature-br-A, but was not renamed in the change-set from base to feature-br-B. In this case, Git will try to combine the two changes by keeping the rename, and also applying "their" changes it sees between lib/foo.py in base and lib/foo.py in feature-br-B, to the file whose name is misc/zap.py in feature-br-A.

If Git is correct—if this file really was renamed like this—that's exactly what you want. But if Git is wrong, it will mis-merge things. You will need to correct it by hand, or perhaps retry the merge with different rename detection values.

The general rule here is that Git keeps all the rename changes made in your current (local) commit—the tip of feature-br-A in this case—and sets up the index's stage slots so that their contents reflect the contents of the files as (Git thinks) they were named in the base and feature-br-B (other) commits. That's why you may want to refer to the files using the slot numbers, or the more convenient --ours and --theirs flags to git checkout.

Bottom line

If the automatic merge fails, you must fix things up by hand. Using git diff may help. Setting merge.conflictstyle to diff3 may also help (I find it very useful). Note that as long as there is only a single merge base commit (search for "virtual merge base" for more complex cases), you can simplify the two git diff commands. In this case, your current commit is feature-br-A and you are merging feature-br-B, so these two commands will get you the two diffs:

git diff feature-br-B...feature-br-A # "our" changes
git diff feature-br-A...feature-br-B # "their" changes

Note that there are three dots here. In this case, git diff finds the merge base commit (or, if there are multiple merge bases, picks one more or less randomly) and compares it to the commit named on the right. Hence the first git diff command finds "our" changes, and the second finds "theirs".

In any case, as with any merge, you must check the results of the merge. For simple merges with few changes, a quick glance ("that looks right") and/or passing whatever test suite(s) you have is sufficient. For complicated merges, test suites are a very good idea, and a more careful eyeball examination of the merge result is not a bad idea either.

---

¹This is like any other git log but it adds graph information. To get a more succinct drawing, add --oneline. To get just the commits on the two branches, excluding commits on both branches, use the three-dot syntax. To add the merge base(s) and other boundary commits to this drawing, add --boundary. Hence a more complete but shorter version is:

git log --graph --oneline --decorate --boundary feature-br-A...feature-br-B

Note that there are three—not two, not any other number, but exactly three—dots between the branch names in this syntax.