dealing with documentation in CI

Question

We have recently moved to Git. Our new repo is 350MB and ~215MB (61%) of that is directory named Documentation. Besides that repo contains source code and few WiX projects used by build process (TFS Build) to generate installers.

Primary users (developers) do not need documentation directory (it is full of semi-manually-crafted pdf files and other stuff that gets packaged into installers). Only final/CI build is using Documentation directory.

I am looking for an advice on how to move out documentation somewhere in such way that it:

• gets out of the way of developers (i.e. git clone needs to pull only ~135MB+ of compressed data)
• keeps build process simple and does not interfere with normal CI process

One idea I have is to move docs into a separate repo and import it as a Git submodule. Seems a bit inconvenient, tbh...

Answer 1

One idea I have is to move docs into a separate repo and import it as a Git submodule.

That would be the idea:

• git filter-repo can handle the split part. See for instance "Detach (move) subdirectory into separate Git repository".
  Note that it would rewrite the history of the main repository, so make sure everyone is aware of this migration.
• a submodule can track the latest commits from a branch, so your CI process can, through git submodule update --remote, build the latest documentation revision.

The OP adds in the comments:

Problem with moving docs into a submodule is that if there are other submodules (and devs need them) -- they'll start using git clone --recurse-submodules and this will clone docs too, making this move-out somewhat pointless.

It does not have to "clone docs too".

1. You can exclude submodules when cloning your main repository

   git clone -recurse-submodules=":(exclude)docs"
   

2. The developer can then declare the submodule docs not active.

   git config submodule.docs.active false
   

Said submodule won't be cloned or updated.