Do GitHub and GitLab support git clone's --filter parameter?

Question

I want to use git's partialClone feature. In this answer I saw the git clone --filter=tree:none <repo> command.

But when trying to execute on github, the prompt warning: filtering not recognized by server, ignoring. It did not work.

I want to know if it is not supported by the GitHub website, or if there is a problem with my settings.

I asked the feedback staff of GitHub and have not got the answer from the technician.

Answer 1

Although I can't find an official blog post or news regarding the support, GitHub does indeed appear to be rolling out --filter support.

$ git clone --bare --single-branch --depth=1 https://github.com/torvalds/linux

Resulted in downloading 195.82MiB worth of around 74k objects.

$ git clone --bare --single-branch --depth=1 --filter=blob:none https://github.com/torvalds/linux

Resulted in downloading 2.15MiB worth of around 4.7k objects. That's 91x less data if all you want to do is know what files are in a repo.

Since you mention in tree:none I tested that too. Now it results in fatal: expected 'tree:<depth>' and my following experimentation shows that only tree:0 works, which results in downloading 603bytes or so in a bare repo. If you try to clone and checkout then git will slowly figure out the objects it needs and clone the whole repo. Numbers higher than 0 result in: fatal: remote error: filter 'tree' not supported (maximum depth: 0, but got: 1)

Answer 2

This is supported by GitLab 13.0 (May 2020)

Exclude large files using Partial Clone

Storing large binary files in Git is normally discouraged, because every large file added will be downloaded by everyone who clones or fetches changes thereafter.
This is slow, if not a complete obstruction when working from a slow or unreliable internet connection.

In GitLab 13.0, Partial Clone has been enabled for blob size filters, as well as experimentally for other filters.

This allows troublesome large files to be excluded from clones and fetches. When Git encounters a missing file, it will be downloaded on demand.

When cloning a project, use the --filter=blob:none or --filer=blob:limit=1m to exclude blobs completely or by file size.
Note, Partial Clone requires at least Git 2.22.0.

(see also "What is the git clone --filter option's syntax?")

Read more in our recent blog, "How Git Partial Clone lets you fetch only the large file you need", from James Ramsay.

See documentation and issue.

Answer 3

Regarding partial clone/spare checkout support in GitLab:

As of GitLab 12.4 (released 2019-10-22) partial cloning has been added as an optional alpha feature for self hosted instances. You can enable it instance wide via the feature flags api:

curl --data "value=true" --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/features/gitaly_upload_pack_filter

You can get more information about this here: https://docs.gitlab.com/ee/topics/git/partial_clone.html

Just to be clear: You cannot use this feature with gitlab.com hosted repositories at the time of last edit of this answer.

Answer 4

This almost certainly isn't supported by GitHub or GitLab yet.

The --filter option is under active development and isn't really ready for general-purpose consumption yet. GitHub's blog post about the release of Git 2.19 in September, 2018 says

Note that most public servers do not yet support the feature, but you can play with git clone --filter=blob:none against your local Git 2.19 install.

Once this feature becomes more fully-developed and hosts start to support it I'm sure they won't do so quietly. As far as I know, no major cloud provider has made such an announcement yet.

Update from OP 2019-03-21:

Not long ago, I received an official reply from github. They think that the --filter parameter is still an immature feature and is accompanied by some security issues. Therefore, this feature will not be supported in the short term.