Git fetch shallow-clone doesn't pick up a matching commit

Question

I have a commit that has timestamp (both commit and author) of: 2022-08-16T13:12:19Z

I tried this command pulling from GitHub into an empty (git init) repository.

git fetch '--shallow-since=2022-08-16T13:12:18Z' origin d9752d2f263d790eddfaf2e63b746ea361897a65

So that's same timestamp - 1 sec. I would expect that this repository contains the commit in question but it doesn't. The oldest commit it contains has timestamp 2022-08-16T13:13:24Z.

If I use a timestamp that's a whole day earlier to fetch then repository contains the commit needed.

Is there something obvious I am missing? Is the shallow-since parameter just a loose suggestion?

Edit:

git log '--since=2022-08-16T13:12:18Z'

on fully cloned repo will return the commit, but fetch shallow-since on same timestamp will not...

Answer 1

There is definitely something fishy going on.

Here is a command to have git log display the timestamps (author date and committer date) of each commit translated in UTC :

log_with_dates () {
    # - set env timezone to UTC
    # - use --graph to have a clearer view of how commits relate to each other
    # - translate timestamps in local timezone (the '-local' in 'format-local')
    #   and format them in ISO format
    # - use a single line format:
    #     <hash> <decorations> (<author_date> <committer_date>) <first_message_line>

    TZ=UTC git log --graph \
      --date=format-local:"%FT%TZ" \
      --format="%h%d (%ad %cd) %s" "$@"
}

Using the git repository https://github.com/git/git as a reference :

on a complete clone, here is what I see for log_with_dates --since="2022-08-29T00:00:00Z" origin/master (17 commits) :

# d42b38d is origin/master on 2022-08-31
$ log_with_dates --since="2022-08-29T00:00:00Z" d42b38d
*   d42b38dfb5 (HEAD -> master, origin/master, origin/main, origin/HEAD) (2022-08-30T17:27:16Z 2022-08-30T17:27:16Z) Sync with Git 2.37.3
|\  
| * ac8035a2af (tag: v2.37.3, origin/maint) (2022-08-30T17:22:10Z 2022-08-30T17:22:10Z) Git 2.37.3
* 6c8e4ee870 (2022-08-29T16:39:42Z 2022-08-29T21:55:15Z) The sixteenth batch
* 3658170b92 (2022-08-29T21:55:15Z 2022-08-29T21:55:15Z) Merge branch 'es/fix-chained-tests'
* 0d2cf16680 (2022-08-29T21:55:15Z 2022-08-29T21:55:15Z) Merge branch 'ds/github-actions-use-newer-ubuntu'
* f0deb3f2b5 (2022-08-29T21:55:15Z 2022-08-29T21:55:15Z) Merge branch 'ad/preload-plug-memleak'
* edc4f6d280 (2022-08-29T21:55:14Z 2022-08-29T21:55:14Z) Merge branch 'sg/xcalloc-cocci-fix'
* 56ba6245a4 (2022-08-29T21:55:14Z 2022-08-29T21:55:14Z) Merge branch 'en/ort-unused-code-removal'
* 10ccb50b16 (2022-08-29T21:55:13Z 2022-08-29T21:55:13Z) Merge branch 'tl/trace2-config-scope'
* 25402204fe (2022-08-29T21:55:13Z 2022-08-29T21:55:13Z) Merge branch 'vd/fix-perf-tests'
* b014a4416a (2022-08-29T21:55:13Z 2022-08-29T21:55:13Z) Merge branch 'mg/sequencer-untranslate-reflog'
* a0ab573bb1 (2022-08-29T21:55:12Z 2022-08-29T21:55:12Z) Merge branch 'jk/unused-fixes'
* a572a5d4c1 (2022-08-29T21:55:12Z 2022-08-29T21:55:12Z) Merge branch 'jd/prompt-show-conflict'
* bc820cf9e6 (2022-08-29T21:55:12Z 2022-08-29T21:55:12Z) Merge branch 'vd/scalar-enables-fsmonitor'
* 0b08ba7eb6 (2022-08-29T21:55:11Z 2022-08-29T21:55:11Z) Merge branch 'en/ancestry-path-in-a-range'
* 64cb4c34d1 (2022-08-29T21:55:11Z 2022-08-29T21:55:11Z) Merge branch 'mt/rot13-in-c'
* c068a3b8ee (2022-08-29T21:55:10Z 2022-08-29T21:55:11Z) Merge branch 'ds/decorate-filter-tweak'

Here is what I get when I start an empty repository and run
git fetch --shallow-since "2022-08-29T00:00:00Z" origin master (4 commits) :

$ log_with_dates origin/master
*   d42b38d (HEAD, origin/master) (2022-08-30T17:27:16Z 2022-08-30T17:27:16Z) Sync with Git 2.37.3
|\  
| * ac8035a (grafted) (2022-08-30T17:22:10Z 2022-08-30T17:22:10Z) Git 2.37.3
* 6c8e4ee (2022-08-29T16:39:42Z 2022-08-29T21:55:15Z) The sixteenth batch
* 3658170 (grafted) (2022-08-29T21:55:15Z 2022-08-29T21:55:15Z) Merge branch 'es/fix-chained-tests'

Deepening the shallow-since also yields an incomplete view.

I haven't debugged whether this comes from the list of commits returned by github's server or by some processing done on my side of git.

I'm running git version 2.35.0 on Ubuntu.

---

test script to reproduce locally :

#!/bin/bash

mkdir -p test_shallow_since
cd test_shallow_since

git init
git remote add origin git@github.com:git/git

# artificially set the "active commit" to that commit (was origin/master on 2022-08-31)
echo "d42b38dfb5edf1a7fddd9542d722f91038407819" > .git/HEAD

log_with_dates () {
    TZ=UTC git log --graph \
        --date=format-local:"%FT%TZ" \
        --format="%h %d (%ad %cd) %s" \
        "$@"
}

# test_since expects a date as argument, and will run :
#   git fetch --shallow-since=<date> origin master
#   git log ... --since=<date> d42b38d
test_since () {
    local date=$1
    echo "----- running: git fetch --shallow-since=\"$date\" origin master"
    git fetch --shallow-since="$date" origin master

    echo "----- list of commits --since=\"$date\" :"
    log_with_dates --since="$date" d42b38d
}

test_since "2022-08-29T00:00:00Z"
test_since "2022-08-25T00:00:00Z"

Answer 2

I can think of two things that might be going on here:

1. the fetch command you gave won't fetch anything that isn't an ancestor of commit d9752d2f263d790eddfaf2e63b746ea361897a65. If the commit you want to fetch isn't d9752d2f263d790eddfaf2e63b746ea361897a65 or one of it's ancestors, it won't get fetched. If you want to fetch all commits that are newer than the date specified, remove the last part of the command:

git fetch '--shallow-since=2022-08-16T13:12:18Z'

2. Every commit in git has 2 dates associated with it: The commit date, and the author date. The commit date gets updated every time you do anything related to that commit (e.g. git commit --amend, git cherry-pick, etc.), while the author date is usually preserved. It's possible that fetch --shallow-since is filtering based on one of those, and log --since is filtering on the other. This would be a little surprising to me, so I suspect this isn't it, but it's a possibility. If you want to see both dates, you can use the --format=fuller option (e.g. git show --format=fuller d9752d2f263d790eddfaf2e63b746ea361897a65). Check what the dates are for the commit you're interested in.

Answer 3

First, check the log of the commit displayed locally by git log.

Example:

Date:   Fri Aug 5 15:54:46 2022 -0700
                                ^^^^^

That means locally, I must use 15+7=22, for a local command like git log

git log -3 --since=2022-08-05T22:54:45Z # list the commit
git log -3 --since=2022-08-05T22:54:47Z # does not list the commit

Check then if that applies to your git fetch --shallow-since=<date>.