CODEWITHSUNDEEP
gitgit-submodules
Ivan
Ivan

Reputation: 103849

How can I specify a branch/tag when adding a Git submodule?

How does git submodule add -b work?

After adding a submodule with a specific branch, a new cloned repository (after git submodule update --init) will be at a specific commit, not the branch itself (git status on the submodule shows "Not currently on any branch").

I can't find any information on .gitmodules or .git/config about the submodule's branch or any specific commit, so how does Git figure it out?

Also, is it possible to specify a tag instead of a branch?

Upvotes: 1170

Views: 1118635

Answers (15)

jrbe228
jrbe228

Reputation: 578

Going back to the original question, "How can I specify a branch/tag when adding a Git submodule?", suppose I specify a tag in the .gitmodules file:

[submodule "module1"]
    path = module1
    url = https://github.com/<my-org>/module1
    tag = v0.0.1
[submodule "module2"]
    path = module2
    url = https://github.com/<my-org>/module2
    tag = v0.0.1

After cloning / init, you can checkout the tagged version using (PowerShell/Bash):

git submodule foreach 'git checkout $(git config -f $toplevel/.gitmodules submodule.$name.tag)'

Or if you just want to view submodule tags in .gitmodules, without doing a checkout:

git submodule foreach 'git config -f $toplevel/.gitmodules submodule.$name.tag'

Lastly, use the tag from .gitmodules OR default to latest tag if none found in .gitmodules:

git submodule foreach 'git checkout $(git config -f $toplevel/.gitmodules submodule.$name.tag || git describe --tags $(git rev-list --tags --max-count=1))'

Note these commands will fail when run from cmd.exe! Use Powershell or bash.

Upvotes: 1

H&#233;ber Costa Ferreira
H&#233;ber Costa Ferreira

Reputation: 11

As for now, in 2023, it works fine for me doing:

git submodule add -b branch_name -- ssh://submodule_url.git deps/submodule_location

As it shows on git documentation https://git-scm.com/docs/git-submodule: add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--depth <depth>] [--] <repository> [<path>]

Upvotes: 1

VonC
VonC

Reputation: 1323243

(Git 2.22, Q2 2019, has introduced git submodule set-branch --branch aBranch -- <submodule_path>)

Note that if you have an existing submodule which isn't tracking a branch yet, then (if you have git 1.8.2+):

  • Make sure the parent repo knows that its submodule now tracks a branch:

      cd /path/to/your/parent/repo
  git config -f .gitmodules submodule.<path>.branch <branch>

  • Make sure your submodule is actually at the latest of that branch:

      cd path/to/your/submodule
  git checkout -b branch --track origin/branch
    # if the branch already exist:
    git branch -u origin/branch branch

         (with 'origin' being the name of the upstream remote repo the submodule has been cloned from.
         A git remote -v inside that submodule will display it. Usually, it is 'origin')

  • Don't forget to record the new state of your submodule in your parent repo:

      cd /path/to/your/parent/repo
  git add path/to/your/submodule
  git commit -m "Make submodule tracking a branch"

  • Subsequent update for that submodule will have to use the --remote option:

      # update your submodule
  # --remote will also fetch and ensure that
  # the latest commit from the branch is used
  git submodule update --remote

  # to avoid fetching use
  git submodule update --remote --no-fetch

Lolorol suggests in the comments to set the branch in each submodule:

git submodule update --init --recursive
git submodule foreach -q --recursive \
  'branch="$(git config -f $toplevel/.gitmodules submodule.$name.branch)"; git switch ${branch}'
git submodule

Note that with Git 2.10+ (Q3 2016), you can use '.' as a branch name:

The name of the branch is recorded as submodule.<name>.branch in .gitmodules for update --remote.
A special value of . is used to indicate that the name of the branch in the submodule should be the same name as the current branch in the current repository.

But, as commented by LubosD

With git checkout, if the branch name to follow is ".", it will kill your uncommitted work!
Use git switch instead.

That means Git 2.23 (August 2019) or more.

See "Confused by git checkout"

If you want to update all your submodules following a branch:

    git submodule update --recursive --remote

Note that the result, for each updated submodule, will almost always be a detached HEAD, as Dan Cameron note in his answer.

(Clintm notes in the comments that, if you run git submodule update --remote and the resulting sha1 is the same as the branch the submodule is currently on, it won't do anything and leave the submodule still "on that branch" and not in detached head state.)

To ensure the branch is actually checked out (and that won't modify the SHA1 of the special entry representing the submodule for the parent repo), he suggests:

git submodule foreach -q --recursive 'branch="$(git config -f $toplevel/.gitmodules submodule.$name.branch)"; git switch $branch'

Each submodule will still reference the same SHA1, but if you do make new commits, you will be able to push them because they will be referenced by the branch you want the submodule to track.
After that push within a submodule, don't forget to go back to the parent repo, add, commit and push the new SHA1 for those modified submodules.

Note the use of $toplevel, recommended in the comments by Alexander Pogrebnyak.
$toplevel was introduced in git1.7.2 in May 2010: commit f030c96.

it contains the absolute path of the top level directory (where .gitmodules is).

dtmland adds in the comments:

The foreach script will fail to checkout submodules that are not following a branch.
However, this command gives you both:

 git submodule foreach -q --recursive 'branch="$(git config -f $toplevel/.gitmodules submodule.$name.branch)"; [ "$branch" = "" ] && git checkout master || git switch $branch' –

The same command but easier to read:

git submodule foreach -q --recursive \
    'branch="$(git config -f $toplevel/.gitmodules submodule.$name.branch)"; \
     [ "$branch" = "" ] && \
     git checkout master || git switch $branch' –

umläute refines dtmland's command with a simplified version in the comments:

git submodule foreach -q --recursive 'git switch $(git config -f $toplevel/.gitmodules submodule.$name.branch || echo master)'

multiple lines:

git submodule foreach -q --recursive \
  'git switch \
  $(git config -f $toplevel/.gitmodules submodule.$name.branch || echo master)'

Before Git 2.26 (Q1 2020), a fetch that is told to recursively fetch updates in submodules inevitably produces reams of output, and it becomes hard to spot error messages.

The command has been taught to enumerate submodules that had errors at the end of the operation.

See commit 0222540 (16 Jan 2020) by Emily Shaffer (nasamuffin).
(Merged by Junio C Hamano -- gitster -- in commit b5c71cc, 05 Feb 2020)

fetch: emphasize failure during submodule fetch

Signed-off-by: Emily Shaffer

In cases when a submodule fetch fails when there are many submodules, the error from the lone failing submodule fetch is buried under activity on the other submodules if more than one fetch fell back on fetch-by-oid.
Call out a failure late so the user is aware that something went wrong, and where.

Because fetch_finish() is only called synchronously by run_processes_parallel, mutexing is not required around submodules_with_errors.

Note that, with Git 2.28 (Q3 2020), Rewrite of parts of the scripted "git submodule" Porcelain command continues; this time it is "git submodule set-branch" subcommand's turn.

See commit 2964d6e (02 Jun 2020) by Shourya Shukla (periperidip).
(Merged by Junio C Hamano -- gitster -- in commit 1046282, 25 Jun 2020)

submodule: port subcommand 'set-branch' from shell to C

Mentored-by: Christian Couder
Mentored-by: Kaartic Sivaraam
Helped-by: Denton Liu
Helped-by: Eric Sunshine
Helped-by: Đoàn Trần Công Danh
Signed-off-by: Shourya Shukla

Convert submodule subcommand 'set-branch' to a builtin and call it via git submodule.sh.

Git 2.43 (Q4 2023) handles renamed submodules: in .gitmodules files, submodules are keyed by their names, and the path to the submodule whose name is $name is specified by the submodule.$name.path variable.

There were a few codepaths that mixed the name and path up when consulting the submodule database, which have been corrected.

It took long for these bugs to be found as the name of a submodule initially is the same as its path, and the problem does not surface until it is moved to a different path, which apparently happens very rarely.

See commit bd1c20c, commit 32bff36, commit 5fc8806, commit b027fb0, commit 387c122, commit 6327085 (03 Oct 2023) by Jan Alexander Steffens (heftig) (heftig).
(Merged by Junio C Hamano -- gitster -- in commit b32f5b6, 13 Oct 2023)

t7419: test that we correctly handle renamed submodules

Signed-off-by: Jan Alexander Steffens (heftig)

Add the submodule again with an explicitly different name and path.
Test that calling set-branch modifies the correct .gitmodules entries.
Make sure we don't create a section named after the path instead of the name.

Upvotes: 493

Passiondroid
Passiondroid

Reputation: 1589

git submodule add -b develop --name branch-name -- https://branch.git

Upvotes: 1

Johnny Z
Johnny Z

Reputation: 15439

I'd like to add an answer here that is really just a conglomerate of other answers, but I think it may be more complete.

You know you have a Git submodule when you have these two things.

  1. Your .gitmodules has an entry like so:

    [submodule "SubmoduleTestRepo"]
    path = SubmoduleTestRepo
    url = https://github.com/jzaccone/SubmoduleTestRepo.git

  2. You have a submodule object (named SubmoduleTestRepo in this example) in your Git repository. GitHub shows these as "submodule" objects. Or do git submodule status from a command line. Git submodule objects are special kinds of Git objects, and they hold the SHA information for a specific commit.

    Whenever you do a git submodule update, it will populate your submodule with content from the commit. It knows where to find the commit because of the information in the .gitmodules.

    Now, all the -b does is add one line in your .gitmodules file. So following the same example, it would look like this:

    [submodule "SubmoduleTestRepo"]
    path = SubmoduleTestRepo
    url = https://github.com/jzaccone/SubmoduleTestRepo.git
    branch = master

    Note: only branch name is supported in a .gitmodules file, but SHA and TAG are not supported! (instead of that, the branch's commit of each module can be tracked and updated using "git add .", for example like git add ./SubmoduleTestRepo, and you do not need to change the .gitmodules file each time)

    The submodule object is still pointing at a specific commit. The only thing that the -b option buys you is the ability to add a --remote flag to your update as per Vogella's answer:

    git submodule update --remote

    Instead of populating the content of the submodule to the commit pointed to by the submodule, it replaces that commit with the latest commit on the master branch, THEN it populates the submodule with that commit. This can be done in two steps by djacobs7 answer. Since you have now updated the commit the submodule object is pointing to, you have to commit the changed submodule object into your Git repository.

    git submodule add -b is not some magically way to keep everything up to date with a branch. It is simply adds information about a branch in the .gitmodules file and gives you the option to update the submodule object to the latest commit of a specified branch before populating it.

Upvotes: 1130

Dmitry Spikhalsky
Dmitry Spikhalsky

Reputation: 5820

Existing answers have the second step missing and overloaded with details.

To switch an existing submodule to track a new remote url and/or a new branch:

  1. Edit the source of truth in .gitmodules.

For example, from

[submodule "api"]
    path = api
    url = https://github.com/<original_repo>/api.git

to

[submodule "api"]
    path = api
    url = https://github.com/<another_repo>/api.git
    branch = work-in-progress

You can also use hexsha for a commit hash. Or tag, but see 3.

  1. git submodule sync: Updates the description of submodules cached by git in .git/modules from the just-edited source of truth specified in .gitmodules.

  2. Only if you specify a tag: git submodule foreach --recursive 'git fetch --tags' to fetch tags.

  3. git submodule update --init --recursive --remote: Updates the checked-out submodules in the working copy.

  4. Commit the changes.

Upvotes: 62

Pascal T.
Pascal T.

Reputation: 4081

I have this in my .gitconfig file. It is still a draft, but proved useful as of now. It helps me to always reattach the submodules to their branch.

[alias]

######################
#
# Submodules aliases
#
######################

# git sm-trackbranch: places all submodules on their respective branch specified in .gitmodules
# This works if submodules are configured to track a branch, i.e if .gitmodules looks like:
# [submodule "my-submodule"]
#   path = my-submodule
#   url = [email protected]/my-submodule.git
#   branch = my-branch
sm-trackbranch = "! git submodule foreach -q --recursive 'branch=\"$(git config -f $toplevel/.gitmodules submodule.$name.branch)\"; git checkout $branch'"

# sm-pullrebase:
# - pull --rebase on the master repo
# - sm-trackbranch on every submodule
# - pull --rebase on each submodule
#
# Important note:
# - have a clean master repo and subrepos before doing this!
# - this is *not* equivalent to getting the last committed
#   master repo + its submodules: if some submodules are tracking branches
#   that have evolved since the last commit in the master repo,
#   they will be using those more recent commits!
#
#   (Note: On the contrary, git submodule update will stick
#   to the last committed SHA1 in the master repo)
sm-pullrebase = "! git pull --rebase; git submodule update; git sm-trackbranch ; git submodule foreach 'git pull --rebase' "

# git sm-diff will diff the master repo *and* its submodules
sm-diff = "! git diff && git submodule foreach 'git diff' "

# git sm-push will ask to push also submodules
sm-push = push --recurse-submodules=on-demand

# git alias: list all aliases
# useful in order to learn git syntax
alias = "!git config -l | grep alias | cut -c 7-"

Upvotes: 16

Dan Cameron
Dan Cameron

Reputation: 756

In my experience switching branches in the superproject or future checkouts will still cause detached HEADs of submodules regardless if the submodule is properly added and tracked (i.e. @djacobs7 and @Johnny Z answers).

And instead of manually checking out the correct branch manually or through a script git submodule foreach can be used.

This will check the submodule config file for the branch property and checkout the set branch.

git submodule foreach -q --recursive 'branch="$(git config -f $toplevel.gitmodules submodule.$name.branch)"; git checkout $branch'

Upvotes: 43

vogella
vogella

Reputation: 26342

Git 1.8.2 added the possibility to track branches.

# add submodule to track branch_name branch
git submodule add -b branch_name URL_to_Git_repo optional_directory_rename

# update your submodule
git submodule update --remote

See also Git submodules

Upvotes: 327

djacobs7
djacobs7

Reputation: 11827

Note: Git 1.8.2 added the possibility to track branches. See some of the answers below.

It's a little confusing to get used to this, but submodules are not on a branch. They are, like you say, just a pointer to a particular commit of the submodule's repository.

This means, when someone else checks out your repository, or pulls your code, and does git submodule update, the submodule is checked out to that particular commit.

This is great for a submodule that does not change often, because then everyone on the project can have the submodule at the same commit.

If you want to move the submodule to a particular tag:

cd submodule_directory
git checkout v1.0
cd ..
git add submodule_directory
git commit -m "moved submodule to v1.0"
git push

Then, another developer who wants to have submodule_directory changed to that tag, does this

git pull
git submodule update --init

git pull changes which commit their submodule directory points to. git submodule update actually merges in the new code.

Upvotes: 974

Johan
Johan

Reputation: 20763

An example of how I use Git submodules.

  1. Create a new repository
  2. Then clone another repository as a submodule
  3. Then we have that submodule use a tag called V3.1.2
  4. And then we commit.

And that looks a little bit like this:

git init 
vi README
git add README
git commit 
git submodule add git://github.com/XXXXX/xxx.yyyy.git stm32_std_lib
git status

git submodule init
git submodule update

cd stm32_std_lib/
git reset --hard V3.1.2 
cd ..
git commit -a

git submodule status

Maybe it helps (even though I use a tag and not a branch)?

Upvotes: 76

LuKePicci
LuKePicci

Reputation: 61

The only effect of choosing a branch for a submodule is that, whenever you pass the --remote option in the git submodule update command line, Git will check out in detached HEAD mode (if the default --checkout behavior is selected) the latest commit of that selected remote branch.

You must be particularly careful when using this remote branch tracking feature for Git submodules if you work with shallow clones of submodules. The branch you choose for this purpose in submodule settings IS NOT the one that will be cloned during git submodule update --remote. If you pass also the --depth parameter and you do not instruct Git about which branch you want to clone -- and actually you cannot in the git submodule update command line!! -- , it will implicitly behave like explained in the git-clone(1) documentation for git clone --single-branch when the explicit --branch parameter is missing, and therefore it will clone the primary branch only.

With no surprise, after the clone stage performed by the git submodule update command, it will finally try to check out the latest commit for the remote branch you previously set up for the submodule, and, if this is not the primary one, it is not part of your local shallow clone, and therefore it will fail with

fatal: Needed a single revision

Unable to find current origin/NotThePrimaryBranch revision in submodule path 'mySubmodule'

Upvotes: 5

Engineer
Engineer

Reputation: 8847

To switch branch for a submodule (assuming you already have the submodule as part of the repository):

  • cd to root of your repository containing the submodules
  • Open .gitmodules for editing
  • Add line below path = ... and url = ... that says branch = your-branch, for each submodule; save file .gitmodules.
  • then without changing directory do $ git submodule update --remote

...this should pull in the latest commits on the specified branch, for each submodule thus modified.

Upvotes: 26

Love Sharma
Love Sharma

Reputation: 1999

We use Quack to pull a specific module from another Git repository. We need to pull code without the whole code base of the provided repository - we need a very specific module / file from that huge repository and should be updated every time we run update.

So we achieved it in this way:

Create configuration

name: Project Name

modules:
  local/path:
    repository: https://github.com/<username>/<repo>.git
    path: repo/path
    branch: dev
  other/local/path/filename.txt:
    repository: https://github.com/<username>/<repo>.git
    hexsha: 9e3e9642cfea36f4ae216d27df100134920143b9
    path: repo/path/filename.txt

profiles:
  init:
    tasks: ['modules']

With the above configuration, it creates one directory from the provided GitHub repository as specified in first module configuration, and the other one is to pull and create a file from the given repository.

Other developers just need to run

$ quack

And it pulls the code from the above configurations.

Upvotes: 3

Neall
Neall

Reputation: 27114

Git submodules are a little bit strange - they're always in "detached head" mode - they don't update to the latest commit on a branch like you might expect.

This does make some sense when you think about it, though. Let's say I create repository foo with submodule bar. I push my changes and tell you to check out commit a7402be from repository foo.

Then imagine that someone commits a change to repository bar before you can make your clone.

When you check out commit a7402be from repository foo, you expect to get the same code I pushed. That's why submodules don't update until you tell them to explicitly and then make a new commit.

Personally I think submodules are the most confusing part of Git. There are lots of places that can explain submodules better than I can. I recommend Pro Git by Scott Chacon.

Upvotes: 43

Related Questions