CODEWITHSUNDEEP
backupgithub
Walter White
Walter White

Reputation:

Backup / Mirror Github repositories

I'd like to periodically create a backup of my github repositories. Is there a quick way to pull all of them without knowing what the entire list is?

Walter

Upvotes: 6

Views: 2136

Answers (3)

Marius Gedminas
Marius Gedminas

Reputation: 11367

Now that the v2 API used in the accepted answer no longer works, it's time for an update that uses Github API v3.

You can get the list of repositories in JSON format with

curl -i https://api.github.com/users/username/repos

Watch out for pagination! By default the results are paginated to 30 items. If you have more repositories than fit into a single page, you'll get a Link HTTP response header with links to the other pages (with rel=next/last/first/prev). You can also ask for a larger page size (up to 100):

curl -i https://api.github.com/users/username/repos?per_page=100

A full backup script (assuming you have 100 or fewer repositories) would look something like this:

#!/usr/bin/python
import os
import json
import urllib
import subprocess

username = 'username'  # specify your github username
os.chdir(os.expanduser('~/github'))  # location for your backups, must exist

url = 'https://api.github.com/users/%s/repos?per_page=100' % username
for repo in json.load(urllib.urlopen(url)):
    print "+", repo['full_name']
    if os.path.exists(repo['name']):
        subprocess.call(['git', 'pull'], cwd=repo['name'])
    else:
        subprocess.call(['git', 'clone', repo['git_url']])

Upvotes: 3

Walter White
Walter White

Reputation:

The answer I was waiting for.

I decided to give Ruby a try and it is okay. I like how it is compact, but it isn't pretty looking :(.

This works:

#!/usr/bin/env ruby
require "yaml"
require "open-uri"

time = Time.new
backupDirectory = "/storage/backups/github.com/#{time.year}.#{time.month}.#{time.day}"
username = "walterjwhite"

#repositories =
# .map{|r| %Q[#{r[:name]}] }

#FileUtils.mkdir_p #{backupDirectory}

YAML.load(open("http://github.com/api/v2/yaml/repos/show/#{username}"))['repositories'].map{|repository|

    puts "found repository: #{repository[:name]} ... downloading ..."
    #exec
    system "git clone [email protected]:#{username}/#{repository[:name]}.git #{backupDirectory}/#{repository[:name]}"
}

Walter

Upvotes: 2

Jörg W Mittag
Jörg W Mittag

Reputation: 369633

You can get the entire list via GitHub's API:

curl http://github.com/api/v2/yaml/repos/show/walterjwhite

For example, this simple DOS/Unix shell one-liner:

ruby -ryaml -ropen-uri -e "puts YAML.load(open('http://github.com/api/v2/yaml/repos/show/walterjwhite'))['repositories'].map {|r| %Q[* **#{r[:name]}** (#{r[:description]}) is at <#{r[:url]}/>] }"

prints (assuming you have Ruby installed):

Upvotes: 2

Related Questions