CODEWITHSUNDEEP
gitperformanceversion-controlgit-lfs
James McMahon
James McMahon

Reputation: 49659

How well does Git LFS handle small files?

Is there a best practice for the type files stored in Git LFS? Specifically for the minimum size?

For instance, a 10mb music file would be a obvious fit, but what about a 25kb png? Is it worth putting in LFS or it better to just let Git handle it?

My concern is performance degradation when checking too many small files into an LFS repo. Is there any data on how the LFS extension stands up to a bunch of smaller binary files? Is it advisable to only store files over a certain size threshold?

Upvotes: 45

Views: 9178

Answers (1)

rpy
rpy

Reputation: 4023

I would not expect a an exact threshold value being given.

LFS saves on the amount of data the needs to be exchanged for for synchronizing with a remote repository. However, the saving only applies as long as the large file itself is not changing. Actually for a changed file you would need a second rountrip to process the change on an LFS object.

So, you may include smaller files with LFS if in your use case those are not changing (frequently). The specific break even will depend on the I/O speed of the server and mostly on the latency and throughput between repository and client.

In your example, I'd still expect improvements in case the pngs are close to never changing. As soon as they are going to change (almost) on each and every commit even larger files might not benefit from being put to LFS.

Also the extra cost of second round trip will become less and less important the larger the typical files will be. Especially when the size of a file class (suffix) will vary over a broad range and/or the change frequency within a file class is covering a wide spectrum there might not be a clear answer to your question.

Upvotes: 20

Related Questions