How to access GitHub via Personal Access Token in URL

Question

I maintain a private repository but want to make one file publicly available.

GitHub documentation states that the CURL command below can retrieve a file:

curl -u username:token https://api.github.com/user

But I would like to provide access through a URL. E.g.

https://username:token@raw.githubusercontent.com/me/repo/master/README.md

This always return a 404. Am I missing something?

Answer 1

For those of you wondering the "why" on 404 vs 401, it's basically a security measure on GitHub's part to output 404 instead of 401: https://docs.github.com/en/github-ae@latest/rest/overview/other-authentication-methods#basic-authentication

For those wondering why we get a 404 in the browser while cURL gives us a success response, you might've assumed that providing the username and password in the URL like https://username:password@somesite.com would pass the credentials along in the initial request. That is not the case - the browser steps in and sees if the page you are requesting returns a WWW-Authenticate response header, and only then does it send your credentials. In the case of your GitHub url, the resource doesn't send back a WWW-Authenticate. If it did return WWW-Authenticate, then you obviously wouldn't run into this problem.

And then there's cURL. cURL assumes Basic Authentication by default and automatically sets the Authorization header to your username and password (either from the url like my previous example, or set through CLI options like in your example), and it sends it regardless of whether or not the server returns a WWW-Authenticate response header.

Unfortunately for us, there's no way to force the browser to send it with the initial request. As to why GitHub doesn't send a WWW-Authenticate response header, it's probably because they don't want to promote the least secure way of authentication - they no longer allow account passwords to be sent this way, after all. However, they do realize its ease of use and have mitigated some of its weaker points by allowing users to use oAuth access token, GitHub App installation access token, or Personal Access Token in its place that can limit its scope of access. So really, it's the browser that is following standards, and GitHub allowing a form of Basic Authentication with some alterations, and cURL immediately passing our credentials into the Authorization header. I believe the below is what's happening behind your requests:

cURL sends a request along with Authorization header → GitHub: "Well, I didn't ask, but yeah, your creds check out" → GitHub: Authorized and redirects to resource

Browser sends request and waits for WWW-Authenticate response before handing credentials → GitHub: "Umm, you don't have permission to access this resource but I can't let you know whether it actually exists") → GitHub: Returns 404 (instead of 401 with WWW-Authenticate header) stopping the browser short from receiving the WWW-Authenticate header response and sending out an Authorization header with the credentials on hand.

Answer 2

From "How can I download a single raw file from a private github repo using the command line?", you wouldneed to use a PAT (Personnal Access Token) without the username:

curl -s https://$TOKEN@raw.githubusercontent.com/....

But I would not recommend making that token visible in any way: it would give access to that file and the rest of the repository.

Putting that file in a separate location (be it a separate public repository, or any other online text storage service) would be safer.