CODEWITHSUNDEEP
gittty
merlin2011
merlin2011

Reputation: 75589

Why is git log significantly slower when outputting to a tty?

In a large monorepo (.git directory 10GB), I have observed the following behavior.

# When outputing to a file, it is very fast:
time git  log   -2  > /tmp/X

real    0m0.076s
user    0m0.007s
sys     0m0.046s

# When outputting to the tty, it is more than 10x slower:
time git  log   -2
# ... Output omitted

real    0m0.830s
user    0m0.078s
sys     0m0.586s

Here are all of my log-related configs:

$ git config -l | grep log

log.mailmap=true
core.logallrefupdates=true

What causes the difference in performance, and how can I get the same performance when git log is outputting to a tty?

More observations:

Upvotes: 5

Views: 156

Answers (2)

LeGEC
LeGEC

Reputation: 52081

As written in your answer, I think you have identified the cause for your slowness: a very high number of references in your local repository slows down git log when it tries to print the decoration for each commit.

Check what kind of references make the bulk of these. You can compare the following numbers:

wc -l .git/packed-refs  # total refs

git branch | wc -l      # local branches
git branch -r | wc -l   # remote branches
git tag --list | wc -l  # tags

If you find out you have a huge amount of remote branches: try running git fetch --prune to get rid of remote refs (refs in refs/remotes/origin/*) that have been deleted upstream.

If the 3 numbers do not add up to the total amount of lines in packed-refs, check what other refs could bring the count higher.

  • you can manually check packed-refs: it is just a text file, one line per ref,
  • you can try a bit of script-foo on the output of git for-each-ref, for example:
git for-each-ref --format="%(refname)" | cut -d/ -f1-2 | sort | uniq -c

cut -d/ -f1-2 says "use / as a field separator, only keep the first 2 fields", you can narrow on the highest numbers, and get more details by raising the second number in -f1-x

# example: if you find out you have refs looking like `refs/history/*`:
git for-each ref --format=... refs/history | ... | cut -d/ -f1-3

Upvotes: 2

merlin2011
merlin2011

Reputation: 75589

Based on @LeGEC's excellent suggestion and some further experimentation, it looks like the cause of difference is decoration.

Thus, we have the following results:

# Slow
time git log -2
real    0m0.838s
user    0m0.082s
sys     0m0.597s

time git log --format='%C(auto)commit %H%d%nAuthor: %an <%ae>%nDate:   %ad%n%n%w(0,4,4)%s%n%n%-b%-C()%n' -2
real    0m0.850s
user    0m0.086s
sys     0m0.607s


# Fast
time git log -2  --no-decorate
real    0m0.078s
user    0m0.009s
sys     0m0.055s

time git log --format='%C(auto)commit %H%nAuthor: %an <%ae>%nDate:   %ad%n%n%w(0,4,4)%s%n%n%-b%-C()%n' -2
real    0m0.053s
user    0m0.007s
sys     0m0.024s

time git log -2 | cat
real    0m0.068s
user    0m0.007s
sys     0m0.049s

Upvotes: 3

Related Questions