File disappeared after git rebase attempt

Question

I tried to do git rebase, and in the process lost a file. Not a very important file, but I'd like to understand what happened.

I wanted to merge branch 'master' into branch 'learning' using rebase command.

Here is a summary of my actions:

Did git rebase master, got a conflict, resolved it, tried to continue rebase, failed again with more conflicts, added/commited an untracked file, failed rebase again, tried rebase --skip, failed, aborted rebase, finally did git merge, which was successful.

The untracked file (test.py) I committed during rebase is now gone.

Here is the detailed record of my actions:

$git checkout learning
Switched to branch 'learning'
Your branch is up to date with 'origin/learning'.

$git rebase master
First, rewinding head to replay your work on top of it...
Applying: first commit: changed mnist example trainer
Using index info to reconstruct a base tree...
A       examples/mnist-convnet.py
Falling back to patching base and 3-way merge...
Auto-merging examples/basics/mnist-convnet.py
Applying: added print tensor callback to run after each epoch
Using index info to reconstruct a base tree...
A       examples/mnist-convnet.py
Falling back to patching base and 3-way merge...
Auto-merging examples/basics/mnist-convnet.py
Applying: trying to make callback for mnist-visualization
Using index info to reconstruct a base tree...
A       examples/mnist-convnet.py
A       examples/mnist-visualizations.py
Falling back to patching base and 3-way merge...
Auto-merging examples/basics/mnist-visualizations.py
Auto-merging examples/basics/mnist-convnet.py
Applying: still trying to convert visualization into a callback
Using index info to reconstruct a base tree...
A       examples/mnist-visualizations.py
Falling back to patching base and 3-way merge...
Auto-merging examples/basics/mnist-visualizations.py
Applying: misc changes to visualization
Using index info to reconstruct a base tree...
A       examples/mnist-visualizations.py
Falling back to patching base and 3-way merge...
Auto-merging examples/basics/mnist-visualizations.py
Applying: added VisualizeTensors callback
Using index info to reconstruct a base tree...
A       examples/mnist-visualizations.py
Falling back to patching base and 3-way merge...
Auto-merging examples/basics/mnist-visualizations.py
Applying: first commit: changed mnist example trainer
Using index info to reconstruct a base tree...
A       examples/mnist-convnet.py
Falling back to patching base and 3-way merge...
No changes -- Patch already applied.
Applying: added print tensor callback to run after each epoch
error: Failed to merge in the changes.
Using index info to reconstruct a base tree...
A       examples/mnist-convnet.py
Falling back to patching base and 3-way merge...
Auto-merging examples/basics/mnist-convnet.py
CONFLICT (content): Merge conflict in examples/basics/mnist-convnet.py
Patch failed at 0008 added print tensor callback to run after each epoch
The copy of the patch that failed is found in: .git/rebase-apply/patch

Resolve all conflicts manually, mark them as resolved with
"git add/rm <conflicted_files>", then run "git rebase --continue".
You can instead skip this commit: run "git rebase --skip".
To abort and get back to the state before "git rebase", run "git rebase --abort".

$git status
rebase in progress; onto 4142b9e7
You are currently rebasing branch 'learning' on '4142b9e7'.
  (fix conflicts and then run "git rebase --continue")
  (use "git rebase --skip" to skip this patch)
  (use "git rebase --abort" to check out the original branch)

Unmerged paths:
  (use "git reset HEAD <file>..." to unstage)
  (use "git add <file>..." to mark resolution)

        both modified:   examples/basics/mnist-convnet.py

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        examples/basics/test.py

no changes added to commit (use "git add" and/or "git commit -a")

$git add examples\basics\mnist-convnet.py

$git status
rebase in progress; onto 4142b9e7
You are currently rebasing branch 'learning' on '4142b9e7'.
  (all conflicts fixed: run "git rebase --continue")

Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

        modified:   examples/basics/mnist-convnet.py

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        examples/basics/test.py

$git commit -am "fixing mnist-convnet conflict"
[detached HEAD 2835adcc] fixing mnist-convnet conflict
 1 file changed, 2 insertions(+), 1 deletion(-)

$git status
rebase in progress; onto 4142b9e7
You are currently rebasing branch 'learning' on '4142b9e7'.
  (all conflicts fixed: run "git rebase --continue")

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        examples/basics/test.py

nothing added to commit but untracked files present (use "git add" to track)

$git rebase --continue
Applying: added print tensor callback to run after each epoch
No changes - did you forget to use 'git add'?
If there is nothing left to stage, chances are that something else
already introduced the same changes; you might want to skip this patch.

Resolve all conflicts manually, mark them as resolved with
"git add/rm <conflicted_files>", then run "git rebase --continue".
You can instead skip this commit: run "git rebase --skip".
To abort and get back to the state before "git rebase", run "git rebase --abort".

$git status
rebase in progress; onto 4142b9e7
You are currently rebasing branch 'learning' on '4142b9e7'.
  (all conflicts fixed: run "git rebase --continue")

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        examples/basics/test.py

nothing added to commit but untracked files present (use "git add" to track)

$git add examples\basics\test.py

$git commit -am "added test"
[detached HEAD 9f32ce6d] added test
 1 file changed, 23 insertions(+)
 create mode 100644 examples/basics/test_FP16_vs_FP32.py

$git status
rebase in progress; onto 4142b9e7
You are currently rebasing branch 'learning' on '4142b9e7'.
  (all conflicts fixed: run "git rebase --continue")

nothing to commit, working tree clean

$git rebase --continue
Applying: added print tensor callback to run after each epoch
No changes - did you forget to use 'git add'?
If there is nothing left to stage, chances are that something else
already introduced the same changes; you might want to skip this patch.

Resolve all conflicts manually, mark them as resolved with
"git add/rm <conflicted_files>", then run "git rebase --continue".
You can instead skip this commit: run "git rebase --skip".
To abort and get back to the state before "git rebase", run "git rebase --abort".

$git rebase --skip
Applying: trying to make callback for mnist-visualization
error: Failed to merge in the changes.
Using index info to reconstruct a base tree...
A       examples/mnist-convnet.py
A       examples/mnist-visualizations.py
Falling back to patching base and 3-way merge...
CONFLICT (modify/delete): examples/mnist-visualizations.py deleted in HEAD and modified in 
trying to make callback for mnist-visualization. Version trying to make 
callback for mnist-visualization of examples/mnist-visualizations.py left in tree.
Auto-merging examples/basics/mnist-convnet.py
CONFLICT (content): Merge conflict in examples/basics/mnist-convnet.py
Patch failed at 0009 trying to make callback for mnist-visualization
The copy of the patch that failed is found in: .git/rebase-apply/patch

Resolve all conflicts manually, mark them as resolved with
"git add/rm <conflicted_files>", then run "git rebase --continue".
You can instead skip this commit: run "git rebase --skip".
To abort and get back to the state before "git rebase", run "git rebase --abort".

$git status
rebase in progress; onto 4142b9e7
You are currently rebasing branch 'learning' on '4142b9e7'.
  (fix conflicts and then run "git rebase --continue")
  (use "git rebase --skip" to skip this patch)
  (use "git rebase --abort" to check out the original branch)

Unmerged paths:
  (use "git reset HEAD <file>..." to unstage)
  (use "git add/rm <file>..." as appropriate to mark resolution)

        both modified:   examples/basics/mnist-convnet.py
        deleted by us:   examples/mnist-visualizations.py

no changes added to commit (use "git add" and/or "git commit -a")

$git rebase --abort

$git status
On branch learning
Your branch is up to date with 'origin/learning'.

nothing to commit, working tree clean

$git merge master
Merge made by the 'recursive' strategy.
 ...                                   
 78 files changed, 880 insertions(+), 539 deletions(-)

$dir examples\basics

 Directory of \Code\tensorpack\examples\basics

03/01/2018  03:19 PM    <DIR>          .
03/01/2018  03:19 PM    <DIR>          ..
03/01/2018  03:19 PM             6,846 cifar-convnet.py
03/01/2018  03:19 PM             5,994 mnist-convnet.py
02/14/2018  12:58 PM             5,828 mnist-tflayers.py
02/12/2018  07:14 PM             3,488 mnist-tfslim.py
03/01/2018  03:01 PM             6,624 mnist-visualizations.py
02/12/2018  07:14 PM             4,166 svhn-digit-convnet.py
02/15/2018  03:43 PM    <DIR>          train_log
               6 File(s)         32,946 bytes
               3 Dir(s)  86,909,169,664 bytes free

As you can see, test.py which used to be in examples\basics is gone.

So, I'd like to understand two things: why did git rebase fail with conflicts but git merge went through fine? And what happened to the untracked file that I committed during rebase operation?

Answer 1

First, that does not merge master into learning: a git rebase master will replay learning on top of master.

Second, if at any point you added/committed test.py, you can find it back in the reflog. See "Query git reflog for all commits to a specific file".

git rev-list --all --  test.py 

Third, during a rebase, you are not supposed to commit, only to add, then git rebase --continue.

---

Another case: "git rebase BASE BRANCH" rebased/updated the tip of BRANCH and checked it out, even when the BRANCH is checked out in a different worktree.

This has been corrected with Git 2.26 (Q1 2020),

See commit b5cabb4, commit df126ca (23 Feb 2020) by Eric Sunshine (sunshineco).
^{(Merged by Junio C Hamano -- gitster -- in commit a0ab37d, 05 Mar 2020)}

rebase: refuse to switch to branch already checked out elsewhere

^{Reported-by: Mike Hommey
Signed-off-by: Eric Sunshine}

The invocation "git rebase <upstream> <branch>" switches to before performing the rebase operation.

However, unlike git switch, git checkout, and git worktree which all refuse to switch to a branch that is already checked out in some other worktree, git rebase switches to <branch> unconditionally.

Curb this careless behavior by making git rebase also refuse to switch to a branch checked out elsewhere.