Unable to decrypt RSA-OAEP message encrypted with PyCryptodome using SubtleCrypto Web Crypto API

Question

On the server side I am using PyCryptodome to encrypt a message with RSA-OAEP (with SHA-256).
I'm trying to decrypt the message using SubtleCrypto Web Crypto API on the client side, but it give me a DOMException error without no further details.
On SubtleCrypto I can import the private key generated in PyCryptodome without problems, but it gives me the error when I'm trying to decrypt the message.

I have also tried to import the public key generated on PyCryptodome on client side to encrypt the same message with SubtleCrypto. In that case I can decrypt it without problems, using the same flow as before.

Are the RSA-OAEP algorithms between these two libraries incompatible? I noticed that PyCryptodome references RFC 8017(v2.2) and SubtleCrypto RFC 3447(v2.1) in their respective documentation.

Edit:

• Server side code (pycryptodome==3.9.8):

    from Crypto.PublicKey import RSA
    from Crypto.Cipher import PKCS1_OAEP
  
    class Cipher:
  
    def rsa_encrypt(self, data, key_string):
        key = RSA.importKey(key_string)
        rsa_encryption_cipher = PKCS1_OAEP.new(key)
        ciphertext = rsa_encryption_cipher.encrypt(data)
        return base64.b64encode(ciphertext)
  
    def rsa_decrypt(self, data, key_string):
        data = base64.b64decode(data)
        key = RSA.importKey(key_string)
        rsa_decryption_cipher = PKCS1_OAEP.new(key)
        plaintext = rsa_decryption_cipher.decrypt(data)
        return plaintext
  
  ( ... ) 
  

• Client side code

  private decryptRSAString (encryptedText: string, privateKey: string) : Observable<ArrayBuffer> {
  
    return Observable.create ((observer: any) => {
  
      let keyBuffer: ArrayBuffer = this.str2ab(window.atob(privateKey));
      let encryptedTextBuffer: ArrayBuffer = this.str2ab(window.atob(encryptedText));
      let algorithmParams: RsaHashedImportParams = {
        name: "RSA-OAEP",
        hash: "SHA-256"
      };
      window.crypto.subtle.importKey(
        'pkcs8',
        keyBuffer,
        algorithmParams,
        true,
        ["decrypt"]
      ).then (
        (cryptoKey: CryptoKey) => {
          window.crypto.subtle.decrypt(
            {
              name: "RSA-OAEP"
            },
            cryptoKey,
            encryptedTextBuffer
          ).then (
            (decryptedMessage: ArrayBuffer) => {
              observer.next (decryptedMessage);
              observer.complete();
            },
            (error: any) => {
              observer.error (error)
            }
          )
        },
        (error: any) => {
          observer.error (error)
        }
      );
    });
  }
  

Answer 1

PyCryptodome does not apply SHA-256 as default digest for OAEP, but SHA-1, here. Accordingly SHA-1 must be used on the WebCrypto side:

let algorithmParams: RsaHashedImportParams = {
    name: "RSA-OAEP",
    hash: "SHA-1"
};

Of course you can also apply SHA-256 on the PyCryptodome side, then no changes are necessary on the WebCrypto side.

from Crypto.Hash import SHA256
...
rsa_encryption_cipher = PKCS1_OAEP.new(key, SHA256) # default: Crypto.Hash.SHA1

With consistent digests on both sides I can successfully decrypt a ciphertext with your WebCrypto code, which I have previously generated with your PyCryptodome code (using my own keys).