CODEWITHSUNDEEP
pythongithubgithub-api
Denis SkS
Denis SkS

Reputation: 856

How to create a commit and push into repo with GitHub API v3?

I want to create a repository and Commit a few files to it via any Python package. How do I do?

I do not understand how to add files for commit.

Upvotes: 47

Views: 62166

Answers (9)

malthe
malthe

Reputation: 1449

You can use the GraphQL API, specifically the createCommitOnBranch mutation.

The advantage compared to using the create or update file contents endpoint (as suggested in some of the other answers) is that the commit will be marked as verified:

This commit was created on GitHub.com and signed with GitHub's verified signature.

This means that if you trust GitHub's authority, you can be sure that the commit happened exactly at that point in time, something that's not possible otherwise.

See this answer for more information, including an example of using the GraphQL API using curl from the command-line.

Upvotes: 0

sreehari.codes
sreehari.codes

Reputation: 1

Solution using Github API and requests package

You can commit files to a repository by Create or update file contents endpoint.

PUT https://api.github.com/repos/{owner}/{repo}/contents/{path}
{
   "message": "<commit_message>",
   "committer": {
       "name": "<username>",
       "email": "<email>"
   },
   "content": <content-encoded-in-base64>
}

Sharing the python snippet

import base64
import requests


class GitHubRepo:
    def __init__(self):
        self.base_url = "https://api.github.com/repos"
        self.repo = "GITHUB_REPOSITORY_NAME"
        self.owner = "GITHUB_USERNAME"
        self.token = "GITHUB_ACCESS_TOKEN"
        self.branch = "BRANCH_NAME"
        self.email = "EMAIL"

    def get_headers(self):
        headers = {
            'Authorization': 'Bearer {}'.format(self.token),
            'User-Agent': (
                'Mozilla/5.0 (Windows NT 6.1; Win64; '
                'x64; rv:47.0) Gecko/20100101 Firefox/47.0'
            ),
            'X-GitHub-Api-Version': '2022-11-28',
            'Accept': 'application/vnd.github+json'
        }
        return headers

    def put(self, url, data):
        try:
            response = requests.put(
                url,
                headers=self.get_headers(),
                json=data
            )
            return response.json()
        except requests.exceptions.RequestException:
            return None

    def create_file(self, content_in_bytes, path, commit_message):
        encoded = str(base64.b64encode(content_in_bytes).decode("utf-8"))
        data = {
            "message": commit_message,
            "committer": {
                "name": self.owner,
                "email": self.email
            },
            "content": encoded
        }
        url = "{}/{}/{}/contents/{}".format(
            self.base_url,
            self.owner,
            self.repo,
            path
        )
        response = self.put(url, data)
        return response

Upvotes: 0

Arthur Miranda
Arthur Miranda

Reputation: 429

Solution using the requests library:

NOTES: I use the requests library to do the calls to GitHub REST API v3.

1. Get the last commit SHA of a specific branch

# GET /repos/:owner/:repo/branches/:branch_name
last_commit_sha = response.json()['commit']['sha']

2. Create the blobs with the file's content (encoding base64 or utf-8)

# POST /repos/:owner/:repo/git/blobs
# {
#  "content": "aGVsbG8gd29ybGQK",
#  "encoding": "base64"
#}
base64_blob_sha = response.json()['sha']

# POST /repos/:owner/:repo/git/blobs
# {
#  "content": "hello world",
#  "encoding": "utf-8"
#}
utf8_blob_sha = response.json()['sha']

3. Create a tree which defines the folder structure

# POST repos/:owner/:repo/git/trees/
# {
#   "base_tree": last_commit_sha,
#   "tree": [
#     {
#       "path": "myfolder/base64file.txt",
#       "mode": "100644",
#       "type": "blob",
#       "sha": base64_blob_sha
#     },
#     {
#       "path": "file-utf8.txt",
#       "mode": "100644",
#       "type": "blob",
#       "sha": utf8_blob_sha
#     }
#   ]
# }
tree_sha = response.json()['sha']

4. Create the commit

# POST /repos/:owner/:repo/git/commits
# {
#   "message": "Add new files at once programatically",
#   "author": {
#     "name": "Jan-Michael Vincent",
#     "email": "[email protected]"
#   },
#   "parents": [
#     last_commit_sha
#   ],
#   "tree": tree_sha
# }
new_commit_sha = response.json()['sha']

5. Update the reference of your branch to point to the new commit SHA (on master branch example)

# PATCH /repos/:owner/:repo/git/refs/heads/:branch
# {
#     "sha": new_commit_sha
# }

Finally, for a more advanced setup read the docs.

Upvotes: 42

Jose Celano
Jose Celano

Reputation: 569

I have created an example for committing with multiple files using Python:

import datetime
import os
import github
   
# If you run this example using your personal token the commit is not going to be verified.
# It only works for commits made using a token generated for a bot/app 
# during the workflow job execution.

def main(repo_token, branch):

    gh = github.Github(repo_token)

    repository = "josecelano/pygithub"

    remote_repo = gh.get_repo(repository)

    # Update files:
    #   data/example-04/latest_datetime_01.txt
    #   data/example-04/latest_datetime_02.txt
    # with the current date.

    file_to_update_01 = "data/example-04/latest_datetime_01.txt"
    file_to_update_02 = "data/example-04/latest_datetime_02.txt"

    now = datetime.datetime.now()

    file_to_update_01_content = str(now)
    file_to_update_02_content = str(now)

    blob1 = remote_repo.create_git_blob(file_to_update_01_content, "utf-8")
    element1 = github.InputGitTreeElement(
        path=file_to_update_01, mode='100644', type='blob', sha=blob1.sha)

    blob2 = remote_repo.create_git_blob(file_to_update_02_content, "utf-8")
    element2 = github.InputGitTreeElement(
        path=file_to_update_02, mode='100644', type='blob', sha=blob2.sha)

    commit_message = f'Example 04: update datetime to {now}'

    branch_sha = remote_repo.get_branch(branch).commit.sha
   
    base_tree = remote_repo.get_git_tree(sha=branch_sha)
 
    tree = remote_repo.create_git_tree([element1, element2], base_tree)

    parent = remote_repo.get_git_commit(sha=branch_sha)

    commit = remote_repo.create_git_commit(commit_message, tree, [parent])

    branch_refs = remote_repo.get_git_ref(f'heads/{branch}')

    branch_refs.edit(sha=commit.sha)

Upvotes: 1

Chmouel Boudjnah
Chmouel Boudjnah

Reputation: 2559

Based on previous answer, here is a complete example. Note that you need to use POST if you upload the commit to a new branch, or PATCH to upload to an existing one.


    import whatsneeded
    
    GITHUB_TOKEN = "WHATEVERWILLBEWILLBE"
    
    def github_request(method, url, headers=None, data=None, params=None):
        """Execute a request to the GitHUB API, handling redirect"""
        if not headers:
            headers = {}
        headers.update({
            "User-Agent": "Agent 007",
            "Authorization": "Bearer " + GITHUB_TOKEN,
        })
    
        url_parsed = urllib.parse.urlparse(url)
        url_path = url_parsed.path
        if params:
            url_path += "?" + urllib.parse.urlencode(params)
    
        data = data and json.dumps(data)
        conn = http.client.HTTPSConnection(url_parsed.hostname)
        conn.request(method, url_path, body=data, headers=headers)
        response = conn.getresponse()
        if response.status == 302:
            return github_request(method, response.headers["Location"])
    
        if response.status >= 400:
            headers.pop('Authorization', None)
            raise Exception(
                f"Error: {response.status} - {json.loads(response.read())} - {method} - {url} - {data} - {headers}"
            )
    
        return (response, json.loads(response.read().decode()))
      
    def upload_to_github(repository, src, dst, author_name, author_email, git_message, branch="heads/master"):
        # Get last commit SHA of a branch
        resp, jeez = github_request("GET", f"/repos/{repository}/git/ref/{branch}")
        last_commit_sha = jeez["object"]["sha"]
        print("Last commit SHA: " + last_commit_sha)
    
        base64content = base64.b64encode(open(src, "rb").read())
        resp, jeez = github_request(
            "POST",
            f"/repos/{repository}/git/blobs",
            data={
                "content": base64content.decode(),
                "encoding": "base64"
            },
        )
        blob_content_sha = jeez["sha"]
    
        resp, jeez = github_request(
            "POST",
            f"/repos/{repository}/git/trees",
            data={
                "base_tree":
                last_commit_sha,
                "tree": [{
                    "path": dst,
                    "mode": "100644",
                    "type": "blob",
                    "sha": blob_content_sha,
                }],
            },
        )
        tree_sha = jeez["sha"]
    
        resp, jeez = github_request(
            "POST",
            f"/repos/{repository}/git/commits",
            data={
                "message": git_message,
                "author": {
                    "name": author_name,
                    "email": author_email,
                },
                "parents": [last_commit_sha],
                "tree": tree_sha,
            },
        )
        new_commit_sha = jeez["sha"]
    
        resp, jeez = github_request(
            "PATCH",
            f"/repos/{repository}/git/refs/{branch}",
            data={"sha": new_commit_sha},
        )
        return (resp, jeez)

Upvotes: 3

Martin Monperrus
Martin Monperrus

Reputation: 2071

Here is a complete snippet:

def push_to_github(filename, repo, branch, token):
    url="https://api.github.com/repos/"+repo+"/contents/"+filename

    base64content=base64.b64encode(open(filename,"rb").read())

    data = requests.get(url+'?ref='+branch, headers = {"Authorization": "token "+token}).json()
    sha = data['sha']

    if base64content.decode('utf-8')+"\n" != data['content']:
        message = json.dumps({"message":"update",
                            "branch": branch,
                            "content": base64content.decode("utf-8") ,
                            "sha": sha
                            })

        resp=requests.put(url, data = message, headers = {"Content-Type": "application/json", "Authorization": "token "+token})

        print(resp)
    else:
        print("nothing to update")

token = "lskdlfszezeirzoherkzjehrkzjrzerzer"
filename="foo.txt"
repo = "you/test"
branch="master"

push_to_github(filename, repo, branch, token)

Upvotes: 19

eQ19
eQ19

Reputation: 10711

I'm on Google App Engine (GAE) so beside of python, I can create a new file, update it, even delete it via a commit and push into my repo in GitHub with GitHub API v3 in php, java and go.

Checking and reviewing some of the available third party libraries to create like the example script that presented in perl, I would recommend to use the following:

As you aware, you can get one site per GitHub account and organization, and unlimited project sites where the websites are hosted directly from your repo and powered by Jekyll as default.

Combining Jekyll, Webhooks, and GitHub API Script on GAE, along with an appropriate GAE Setting, it will give you a wide possibility like calling external script and create a dynamic page on GitHub.

Other than GAE, there is also an option run it on Heroku. Use JekyllBot that lives on a (free) Heroku instance to silently generates JSON files for each post and pushing the changes back to GitHub.

Upvotes: 0

VonC
VonC

Reputation: 1327624

You can see if the new update GitHub CRUD API (May 2013) can help

The repository contents API has allowed reading files for a while. Now you can easily commit changes to single files, just like you can in the web UI.

Starting today, these methods are available to you:

Upvotes: 21

Jai Pandya
Jai Pandya

Reputation: 2329

Github provides a Git database API that gives you access to read and write raw objects and to list and update your references (branch heads and tags). For a better understanding of the topic, I would highly recommend you reading Git Internals chapter of Pro Git book.

As per the documentation, it is a 7 steps process to commit a change to a file in your repository:

  1. get the current commit object
  2. retrieve the tree it points to
  3. retrieve the content of the blob object that tree has for that particular file path
  4. change the content somehow and post a new blob object with that new content, getting a blob SHA back
  5. post a new tree object with that file path pointer replaced with your new blob SHA getting a tree SHA back
  6. create a new commit object with the current commit SHA as the parent and the new tree SHA, getting a commit SHA back
  7. update the reference of your branch to point to the new commit SHA

This blog does a great job at explaining this process using perl. For a python implementation, you can use PyGithub library.

Upvotes: 12

Related Questions