How to solve UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa1 in position 3: invalid start byte?

Question

print(cipher.decrypt(pad(base64.b64decode(encrypted_string),16)).decode('utf-8')) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa1 in position 3: invalid start byte

Data flow: Javascript:

1. Input = {"name":"abcd","password":"abcd"}
2. JSON.stringify(Input)
3. CryptoJS encrypt AES with CBC MODE
4. Base64 encode
5. AJAX to Django

Python3/Django/PyCryptodome

1. base64 decode
2. Crypto.pad 16B
3. Crypto.cipher.decrypt [successful]

The output received is

b'\xf5R\x0ck_\x90\xca\xcc\n\xe6S:\xed\xf1\x0f\x14\x1f\x8c_\x00\xd8m\x82\x96\xa3\x19K\xca\xc2\xfe\x14\x80\x8bD8\xcc\xdb\x91~\xa2~\xbf\xe5U\xe4\x12E`\xb5\r\xd8t\xbd=\xc0\x862\x837W6|\x07i\x8b\xb1"\xe9\xac}\xef\xf0\xf5\xa8\x0e\xf5z\xdc\xb0\xb5'

How to decode this string to get the output.

---

Code:

function toWordArray(str){
    return CryptoJS.enc.Utf8.parse(str);
}

function toString(words){
    return CryptoJS.enc.Utf8.stringify(words);
}

function toBase64String(words){
    return CryptoJS.enc.Base64.stringify(words);
}

function encrypt(input, key){
    console.log("Input: " + input);  
    var PROTOCOL_AES256 = 2;
    var secret_key = CryptoJS.SHA256(key);
    var header = toWordArray("AMAZON" + String.fromCharCode(PROTOCOL_AES256));
    var iv = CryptoJS.lib.WordArray.random(16);
    var body = CryptoJS.AES.encrypt(input, secret_key, {mode:CryptoJS.mode.CBC,iv: iv,padding:CryptoJS.pad.Pkcs7});

    // construct the packet
    // HEADER + IV + BODY
    header.concat(iv);
    header.concat(body.ciphertext);

    console.log("Bytes before Base64 encoding: " + header);
    // encode in base64
    return toBase64String(header);
}



$("#submitid").click(function(event) {
            event.preventDefault()
            // var encrypted = CryptoJS.AES(...);
            // var encrypted = CryptoJS.SHA256(...);
            var data = {};
            data["name"] = $("#nameid").val();
            data["password"] = $("#passwordid").val();
            var json_payload = JSON.stringify(data);

            var payload = encrypt(json_payload, key);
            console.log("Payload: " + payload);

            $.ajax({
                type: 'GET',
                url: "{% url 'Shenzen:actsignin' %}",
                data: {encrypted_string: payload},
                success: function(data,status,xhr){
                    console.log("AES sucessful withs status: "+status);
                },
                error: function(xhr,status,e) {
                    console.log("AES failed.");
                },
                async: true,
                datatype: 'json'
            })
        })

    },
    error: function(xhr,status,e) {
        console.log("error");
    },
    async: true,
    datatype: 'json'
});

Python3

def actsignin(request):
    global key
    global prime
    global base_num

    encrypted_string = request.GET['encrypted_string']

    print("Encrypted string decoded: ",base64.b64decode(encrypted_string).hex())
    print("----")
    protocol_aes = str(2)
    header_skip = 'AMAZON'+protocol_aes

    sha256_key = SHA256.new(data=bytes(key))
    cipher = AES.new(sha256_key.digest(),AES.MODE_CBC)
    print(cipher.decrypt(pad(base64.b64decode(encrypted_string),16)))
    print('in signin')
    # student = +Student.objects.get(name = name)
    # salt = student.salt
    # key = hashlib.pbkdf2_hmac('sha256',password.encode('utf-8'),salt,100000)
    # # Hash the password here to check
    # password_to_check_hashed = key
    # print(key)
    # print(student.password)
    # if student.password == key:
    return render(request,"Shenzen/display.html",{'student':'student'})
    # else:
    #     return render(request,"Shenzen/display.html",{'status':'failed'})

Answer 1

base64 decode
Crypto.pad 16B
Crypto.cipher.decrypt [successful]

implemented by

cipher = AES.new(sha256_key.digest(),AES.MODE_CBC)
print(cipher.decrypt(pad(base64.b64decode(encrypted_string),16)))

This doesn't make sense. The order is pad and then encrypt, and decrypt then unpad. The unpadding is commonly performed by the decrypt method itself.

Decryption could fail during unpadding, but if you skip the unpadding then you'll just receive garbage. That the last part is garbage is certain if you pad the ciphertext. Decoding the characters is just the first place where you notice that the result is garbage.

Note too that the Python code defaults to an all zero IV, while CryptoJS clearly uses an all random IV, prefixed to the ciphertext.

Without authentication tag (generated by a MAC or an authenticated cipher) it may always be possible to decrypt ciphertext. Despite common belief, block cipher decryption never fails.