Git submodules fail to recurse at cloning from private git server

Question

This question has been asked zillions of times, but always in combination with the use of public git servers such as github. I am in the process of setting up my own private git server, which seems to be going alright, up to the point where I try to clone a project with submodules which I previously committed and pushed myself.

When this question is asked with respect to github, the answer always seems to be that a keypair needs to be generated and the public key needs to be registered with github. I would be happy to register my public key with my private git server, but I cannot find any documentation about this. Is it even possible? Even the documentation from git-scm.com funnel into a github reference.

My private git server is nothing more than a Raspberry Pi with an external harddrive, running Raspbian lite. I have made three projects of which one is a main project which has the other two added as submodule. I can create this as such, commit and push, but when I try to clone it with --recurse-submodules (or without and then submodule init/update), the submodules fail. I can, however, clone the submodules outside the scope of the main project.

I have been fooling around with the network setup, and the Pi has changed IP number a couple of times. When I ran into problems because of that, I setup a DNS entry that points to my router, and added a port forwarding rule to the Pi in the router so I can effectively reach my private server from all over the world. Because the URLs changed, and did not want to end up with remnants of the old URL somewhere, I simply started all three projects again, but that did not help me.

What is going on here? Following log has been anonymized. :)

Update 1: the problem is in the authentication when the submodules are cloned. /var/log/auth.log shows failed logins when a submodule is being cloned as part of the cloning of the main project:

Jan 13 15:07:56 raspberrypi sshd[3148]: Failed password for git from 192.168.1.18 port 62855 ssh2

Scenario I did that shows the error, and also that a clone of a submodule out of the context of the main project simply succeeds:

 E:\Projects>git clone --recurse-submodules ssh://git@git.domain.com/gitdata/Project1.git
Cloning into 'Project1'...
git@git.domain.com's password:
warning: You appear to have cloned an empty repository.

E:\Projects>cp -r Project1.cleanBU/* Project1

E:\Projects>cp -r Project1.cleanBU/.gitignore Project1

E:\Projects>git add Src .gitignore
fatal: not a git repository (or any of the parent directories): .git

E:\Projects>cd Project1

E:\Projects\Project1>git add Src .gitignore

E:\Projects\Project1>git commit
[master (root-commit) 1e52450] Initial commit
 1145 files changed, 726152 insertions(+)
 create mode 100644 .gitignore
...
<<<zapped!>>>
...
 create mode 100644 Src/build.bat

E:\Projects\Project1>git push
git@git.domain.com's password:
Enumerating objects: 550, done.
Counting objects: 100% (550/550), done.
Delta compression using up to 8 threads
Compressing objects: 100% (527/527), done.
Writing objects: 100% (550/550), 1.27 MiB | 3.21 MiB/s, done.
Total 550 (delta 272), reused 0 (delta 0)
remote: Resolving deltas: 100% (272/272), done.
To ssh://git.domain.com/gitdata/Project1.git
 * [new branch]      master -> master

E:\Projects\Project1>git submodule add ssh://git@git.domain.com/gitdata/project2.git Src/project2
Cloning into 'E:/Projects/Project1/Src/project2'...
git@git.domain.com's password:
remote: Counting objects: 202, done.
remote: Compressing objects: 100% (162/162), done.
remote: Total 202 (delta 36), reused 202 (delta 36)
Receiving objects: 100% (202/202), 586.16 KiB | 4.73 MiB/s, done.
Resolving deltas: 100% (36/36), done.

E:\Projects\Project1>git submodule add ssh://git@git.domain.com/gitdata/project3.git Src/project3
Cloning into 'E:/Projects/Project1/Src/project3'...
git@git.domain.com's password:
remote: Counting objects: 18, done.
remote: Compressing objects: 100% (18/18), done.
remote: Total 18 (delta 0), reused 0 (delta 0)Receiving objects:  44% (8/18)
Receiving objects: 100% (18/18), 15.94 KiB | 5.31 MiB/s, done.

E:\Projects\Project1>git add .gitmodules

E:\Projects\Project1>git commit
[master d93cfa4] Added project3 and project2 submodules
 3 files changed, 8 insertions(+)
 create mode 100644 .gitmodules
 create mode 160000 Src/project2
 create mode 160000 Src/project3

E:\Projects\Project1>git push
git@git.domain.com's password:
Enumerating objects: 6, done.
Counting objects: 100% (6/6), done.
Delta compression using up to 8 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 596 bytes | 596.00 KiB/s, done.
Total 4 (delta 0), reused 0 (delta 0)
To ssh://git.domain.com/gitdata/Project1.git
   1e52450..d93cfa4  master -> master

E:\Projects\Project1>cd ..

E:\Projects>rm -rf Project1

E:\Projects>git clone --recurse-submodules ssh://git@git.domain.com/gitdata/Project1.git
Cloning into 'Project1'...
git@git.domain.com's password:
remote: Counting objects: 554, done.
remote: Compressing objects: 100% (259/259), done.
remote: Total 554 (delta 273), reused 549 (delta 272)
Receiving objects: 100% (554/554), 1.27 MiB | 5.96 MiB/s, done.
Resolving deltas: 100% (273/273), done.
Submodule 'Src/project2' (ssh://git@git.domain.com/gitdata/project2.git) registered for path 'Src/project2'
Submodule 'Src/project3' (ssh://git@git.domain.com/gitdata/project3.git) registered for path 'Src/project3'
Cloning into 'E:/Projects/Project1/Src/project2'...
Permission denied, please try again.
Permission denied, please try again.
git@git.domain.com: Permission denied (publickey,password).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'ssh://git@git.domain.com/gitdata/project2.git' into submodule path 'E:/Projects/Project1/Src/project2' failed
Failed to clone 'Src/project2'. Retry scheduled
Cloning into 'E:/Projects/Project1/Src/project2'...
Permission denied, please try again.
Permission denied, please try again.
git@git.domain.com: Permission denied (publickey,password).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'ssh://git@git.domain.com/gitdata/project2.git' into submodule path 'E:/Projects/Project1/Src/project2' failed
Failed to clone 'Src/project2' a second time, aborting
Cloning into 'E:/Projects/Project1/Src/project3'...
Permission denied, please try again.
Permission denied, please try again.
git@git.domain.com: Permission denied (publickey,password).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'ssh://git@git.domain.com/gitdata/project3.git' into submodule path 'E:/Projects/Project1/Src/project3' failed
Failed to clone 'Src/project3'. Retry scheduled
Cloning into 'E:/Projects/Project1/Src/project3'...
Permission denied, please try again.
Permission denied, please try again.
git@git.domain.com: Permission denied (publickey,password).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'ssh://git@git.domain.com/gitdata/project3.git' into submodule path 'E:/Projects/Project1/Src/project3' failed
Failed to clone 'Src/project3' a second time, aborting

E:\Anheko\Projecten>git clone ssh://git@git.domain.com/gitdata/project2.git
Cloning into 'project2'...
git@git.domain.com's password:
remote: Counting objects: 18, done.
remote: Compressing objects: 100% (18/18), done.
Rremote: Total 18 (delta 0), reused 0 (delta 0)eceiving objects:  44% (8/18)
Receiving objects: 100% (18/18), 15.94 KiB | 5.31 MiB/s, done.

Answer 1

Using git daemon perhaps or http(s) this might work? Gonna try git daemon

git daemon has no authentication, so won't ask passwords, which means everyone accesing your server would be able to clone the repos.

I don't want everyone to be able to clone my repositories and 2: I am not sure how to set it up that you do not need a password.

If your server is relying on SSH, you need to your public key (~/.ssh/id_rsa.pub) published to git@yourserver:~git/.ssh/authorized_keys: any clone would not need for you to enter any password (assuming your private key is not passphrase protected)

ssh-copy-id git@yourserver

If your server is using an https server, then you would need to cache your credentials (username/password) through a Git credential storage (on a Windows client, for instance, that would be using the Microsoft GCM).