Two Firestore projects: one works, one fails with "Error: 7 PERMISSION_DENIED: Missing or insufficient permissions."

Question

TLDR: One credential works but the other doesn't?

I have 2 Google Cloud projects for Cloud Firestore. One for production, one for development.

Note: I don't use firebase, just firestore.

Project 1 (prod): My code deployed to GAE (Google App Engine) is able to access Firestore just fine using production credentials. Both GAE and Firestore share the same project.

Project 2 (dev): My local code is able to access Firestore using development credentials (a project I've designated as my development sandbox)

Now here's the issue:

Error: 7 PERMISSION_DENIED: Missing or insufficient permissions.

That's the error I get when I run the following code.

// script.js
//
// In my local environment (my laptop), I want to run a script that queries my
// "Production Firestore" and do some analysis.


const Firestore = require('@google-cloud/firestore');
const prodProjectKey = require('./keys/prod.json')

const db = new Firestore({
  credentials: {
    client_email: prodProjectKey.client_email,
    private_key: prodProjectKey.private_key,
  },
})

;(async () => {
  const doc = await db.collection('users').doc('123').get()
})()

Now, no surprise to me, if I change the credentials to use my development credentials, everything works fine.

// To be super clear, the following works
// (the only difference is using dev.json instead of prod.json)

const devProjectKey = require('./keys/dev.json')
const db = new Firestore({
  credentials: {
    client_email: devProjectKey.client_email,
    private_key: devProjectKey.private_key,
  },
})
;(async () => {
  const doc = await db.collection('users').doc('123').get()
})()

As far as I can tell, the only difference between these keys is the project ID. I've followed the same steps to create them: Google Cloud Console -> Service Accounts -> Create Service Accounts -> Assign Project Owner role -> Create Key -> Download Key

I've seen other questions on SO point to "Database Rules" but

1. these answers seem to only apply for users of Firebase (I only user Firestore)
2. doesn't seem to explain why my prod and dev projects are behaving differently

Answer 1

Kept trying and I finally found the solution.

Correct:

const db = new Firestore({
  projectId: 'myproject',
  keyFilename: '/absolute/path/to/prodProjectKey.json',
})

Wrong:

const db = new Firestore({
  credentials: {
    client_email: prodProjectKey.client_email,
    private_key: prodProjectKey.private_key,
  },
})

I should note for anybody who runs into this same problem, the "wrong" code works perfectly fine when it runs on Google Cloud. It's only when it's run locally that the PERMISSION_DENIED error pops up.

I think the reason is obvious in retrospect when you look at the contents of prodProjectKey.json:

{
  "type": "service_account",
  "project_id": "xxxx",
  "private_key_id": "xxxx",
  "private_key": "-----BEGIN PRIVATE KEY-----xxxxxx\n-----END PRIVATE KEY-----\n",
  "client_email": "xxxx@xxx.iam.gserviceaccount.com",
  "client_id": "xxxx",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "xxx"
}

When you provide client_email and private_key while running on Google Cloud, Google Cloud can infer the other important parameters like project_id from the server environment. When it's run locally, it can't.

Answer 2

The behavior you are describing is actually expected and correct. You cannot use a key from a different project because the key is generated based on each service account and projectId it is related to, you can check the example on this documentation for more details.

As per the firebase rules, you cannot edit them on Cloud Console, but you can deploy with version control using the Firebase CLI also, there is a how to on this documentation.

Note: Being a Cloud Console user, you have access to Firebase Console without additional costs, so I don't see why that would be a problem for your project should you want to edit the rules through there. Also, if you choose to use Firebase CLI, as mentioned on the previosly shared documentation:

When you deploy security rules using the Firebase CLI, the rules defined in your project directory overwrite any existing rules in the Firebase console. So, if you choose to define or edit your security rules using the Firebase console, make sure that you also update the rules defined in your project directory.

Hope this helps.