ValueError: sequence too large; cannot be greater than 32

Question

I wrote this code:

from Crypto.Cipher import AES
import numpy as np
import cv2, base64

BLOCK_SIZE = 16
PADDING = '{'

pad = lambda s: s + (BLOCK_SIZE - len(s) % BLOCK_SIZE) * PADDING

EncodeAES = lambda c, s: base64.b64encode(c.encrypt(pad(s)))
DecodeAES = lambda c, e: c.decrypt(base64.b64decode(e)).rstrip(PADDING)

secret = "youwanttoknowmysecretdontyouhaha"
cipher = AES.new(secret)

video = cv2.VideoCapture(0)
a = 0

while True:
    a = a + 1

    check, frame = video.read()

    encrypted = EncodeAES(cipher, str(frame))

    img = DecodeAES(cipher, encrypted)

    cv2.imshow("Capturing", np.ndarray(img))
    key = cv2.waitKey(1)

    if key == ord('q'):
        break

But it doesnt work like it supposed to.

I want it to play my live camera, but it gives me the following error:

Traceback (most recent call last): File "tester.py", line 28, in cv2.imshow("Capturing", np.ndarray(img)) ValueError: sequence too large; cannot be greater than 32

I really would like to know what Im doing wrong and dont give me only the solution but explain to please, im still learning

EDIT

I use python 2.7

EDIT

The following code works, but I use it in a program where the host sends the encrypted to an other computer. In this case you cant use frame.dtype.

from Crypto.Cipher import AES
import numpy as np
import cv2, base64

BLOCK_SIZE = 16
PADDING = '{'

pad = lambda s: s + (BLOCK_SIZE - len(s) % BLOCK_SIZE) * PADDING

EncodeAES = lambda c, s: base64.b64encode(c.encrypt(pad(s)))
DecodeAES = lambda c, e: c.decrypt(base64.b64decode(e)).rstrip(PADDING)

secret = "youwanttoknowmysecretdontyouhaha"
cipher = AES.new(secret)

video = cv2.VideoCapture(0)
a = 0

while True:
    a = a + 1

    check, frame = video.read()

    encrypted = EncodeAES(cipher, frame.tobytes())

    decrypted = DecodeAES(cipher, encrypted)

    img = np.frombuffer(decrypted, dtype=frame.dtype).reshape(frame.shape)

    cv2.imshow("Capturing", img)
    key = cv2.waitKey(1)

    if key == ord("q"):
        break

video.release()

However I can send frame.dtype to the other computer.

When I send the the encrypted form of frame.dtype and frame_shape to the computer and decrypt it and use it there, I get the following error

img = np.frombuffer(decod_ed, dtype=img_dtype).reshape(img_shape)
TypeError: data type "ûÿùùÖÿùÖÿùÖÿÖÖÿÖÖÿÜÜÖøøÜ£ÜÖ£ÜÖÜÜÖÜÜÖÿÜÜÿÜÜÿÜøÖø£øøøøøøØ£ÿØ£ÿØ£ÿƒ×ÜØ×Ü׃ø؃ÖøØùØØùØØùƒ£ÿ×øùƒ£ÿƒ£ÿƒø܃øÜíØ£íØ£íØ£ó×Øó×Øó×Øóƒøóƒøúá£óƒøáƒøáƒø׃ø׃ø£ƒø£ƒø£ƒø£ƒøƒ×ÜáƒøúíøúíøÑáøÑáøñ×øу£ñØ×Ñ׃ñáƒñáƒóíØíá£áó£ƒíø×áÜ×áÜøƒÖ£áÜ£ƒø£ƒøƒØ£ƒØ£á×Øá×ØØØ£ØØ£××؃ƒ×ƒƒ×××Ø׃ø׃ø£ƒø£ƒø£×Ø£×Ø£×Ø£×ØØá£Øá£ØíøØíøáó£áó£×ó£Øíø܃øù£ÿôÜÿôÜÿÿÜÜÿÜÜøøøØØØíƒ×óáƒíƒ×íƒ×óáƒóáƒñúƒñúƒªÑíñúƒñúƒñúƒúó×úó×úó×úó×óó£óó£úñøúñøúúØññ×óñ×óñ×óúƒóúƒóóíóóíóúƒñÑíóªááñ××ñ××ñ××úƒáÑí׺á׺á£ñƒ£ñƒáúƒáúƒññúóóíúíáúíáííáññúññúññúñÑíñÑíóúƒñÑíñÑíñÑíóÑíóÑíóúƒóúƒñÑíñÑíñÑíóúƒáúƒóÑíóÑíáúƒƒó×áúƒóÑíóÑíóÑíáúƒáúƒƒó׃ó××íØØíøØíøØóÖØóÖØíø£áÜáá܃ƒÖíƒÖƒØùØ×òƒáùØáùÜØöÜØöÜØöÖ×òÿØöù£ôù£ôûÜöûÜööÜööÜöÆøöÆøööÜööÜöûÜööÿÆûùôûùôÿûòûöôÆöôöûòöûòöûòöûòÆöôÉòæÉòæÉûÉÉûÉÅòÅÅòÅæòÅÉöÄÉöÄÉöÄÉöÄÅôìÉÆîÅæïææïÉÉèÉÅåÉÅåîÅåèìäçîâåïéêîüêîüçï~çï~àïÇàïÇë|å|" not understood

Answer 1

I'm assuming that frame is a [SciPy.Docs]: numpy.ndarray. There are 2 problems. Below is an example:

code00.py:

#!/usr/bin/env python3

import sys
from Crypto.Cipher import AES
import numpy as np
import cv2
import base64


BLOCK_SIZE = 16
PADDING = b"{"

pad = lambda s: s + (BLOCK_SIZE - len(s) % BLOCK_SIZE) * PADDING

EncodeAES = lambda c, s: base64.b64encode(c.encrypt(pad(s)))
DecodeAES = lambda c, e: c.decrypt(base64.b64decode(e)).rstrip(PADDING)

CIPHER_MODE = AES.MODE_CBC
SECRET = b"youwanttoknowmysecretdontyouhaha"


def main():

    img0 = cv2.imread("c:/valmand.png")
    encrypt_cipher = AES.new(SECRET, CIPHER_MODE)  # Python 2: AES.new(SECRET)
    print("Original image data - Type: {:}, Size: {:d}, Attrs: {:}".format(type(img0), img0.size, img0.shape))
    img_stream_wrong = str(img0).encode()
    print("\nWrong img0 stream length: {:}".format(len(img_stream_wrong)))
    print("\nWrong img0 stream: {:}".format(img_stream_wrong))
    img_stream = img0.tobytes()
    print("\nCorrect img0 stream length: {:}".format(len(img_stream)))
    encrypted = EncodeAES(encrypt_cipher, img_stream)
    print("\nEncrypted length: {:d}".format(len(encrypted)))
    decrypt_cipher = AES.new(SECRET, CIPHER_MODE)  # Python 2: AES.new(SECRET)
    decrypted = DecodeAES(decrypt_cipher, encrypted)
    print("\nDecrypted length: {:d}".format(len(decrypted)))
    img1 = np.frombuffer(decrypted, dtype=img0.dtype).reshape(img0.shape)
    print("\nFinal image data - Type: {:}, Size: {:d}, Attrs: {:}".format(type(img1), img1.size, img1.shape))
    #cv2.imshow("Capturing", img1)


if __name__ == "__main__":
    print("Python {:s} on {:s}\n".format(sys.version, sys.platform))
    main()

Notes:

• I simplified the example to only load an image
• Since I don't have all the required packages installed on Python 2, I used Python 3. That's why some changes unrelated to the question were required:
  • Switch from string to bytes
  • AES changes:
    • AES.new signature changed ([ReadTheDocs.PyCryptodome]: AES)
    • Same cipher can't be used for decryption after it was used for encryption, so create another with the same properties (maybe there's some cleaner way of doing this, but I didn't spend too much time on it)

• The errors:

  • As seen, applying str on an ndarray, gives a user friendly representation of that ndarray, rather than its contents, so it will be syntactically correct, but semantically wrong
    • To fix that, the array should be properly serialized using its tobytes method

  • Conversely, when deserialization is attempted, the ndarray constructor is invoked on the byte stream, (which is silently converted to a tuple of ints - that the constructor expects - having the same length as the stream)

    • The problem here is that the constructor doesn't accept sequences larger than 32:

      >>> np.ndarray([0] * 32)
      array([], shape=(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0),
            dtype=float64)
      >>> np.ndarray([0] * 33)
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
      ValueError: sequence too large; cannot be greater than 32
      

    • To fix it, [SciPy.Docs]: numpy.frombuffer should be used. For the new ndarray to be equal to the original one, 2 more things are required (data needed from the original):

      • Its dtype should be specified (default is float and we need uint8)
      • Needs to be reshaped

Output:

e:\Work\Dev\StackOverflow\q054326620>"e:\Work\Dev\VEnvs\py_064_03.06.08_test0\Scripts\python.exe" code00.py
Python 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit (AMD64)] on win32

Original image data - Type: <class 'numpy.ndarray'>, Size: 1493331, Attrs: (799, 623, 3)

Wrong img0 stream length: 629

Wrong img0 stream: b'[[[255 255 255]\n  [255 255 255]\n  [255 255 255]\n  ...\n  [255 255 255]\n  [255 255 255]\n  [255 255 255]]\n\n [[255 255 255]\n  [255 255 255]\n  [255 255 255]\n  ...\n  [255 255 255]\n  [255 255 255]\n  [255 255 255]]\n\n [[255 255 255]\n  [255 255 255]\n  [255 255 255]\n  ...\n  [255 255 255]\n  [255 255 255]\n  [255 255 255]]\n\n ...\n\n [[255 255 255]\n  [255 255 255]\n  [255 255 255]\n  ...\n  [255 255 255]\n  [255 255 255]\n  [255 255 255]]\n\n [[255 255 255]\n  [255 255 255]\n  [255 255 255]\n  ...\n  [255 255 255]\n  [255 255 255]\n  [255 255 255]]\n\n [[255 255 255]\n  [255 255 255]\n  [255 255 255]\n  ...\n  [255 255 255]\n  [255 255 255]\n  [255 255 255]]]'

Correct img0 stream length: 1493331

Encrypted length: 1991128

Decrypted length: 1493331

Final image data - Type: <class 'numpy.ndarray'>, Size: 1493331, Attrs: (799, 623, 3)

@EDIT0:

Apparently, there are some compatibility problems. I developed the answer using Python 3 and pycrpytodome 3.7.2. I upgraded to latest (3.7.3), I've also installed it on Python 2, and the code above works. Here's what I get in the console:

>>> from Crypto.Cipher import AES
>>> AES
<module 'Crypto.Cipher.AES' from 'e:\Work\Dev\VEnvs\py_064_02.07.15_test0\lib\site-packages\Crypto\Cipher\AES.pyc'>
>>> AES.new("")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: new() takes at least 2 arguments (1 given)
>>> import Crypto
>>> Crypto.__version__
'3.7.3'

So, you probably have a terribly old pycryptodome version. Anyway, I'm blindly posting the changes required by your code, to incorporate the fixes (the while loop only):

# The rest of your code (exactly as in the question)

while True:
    a = a + 1

    check, frame = video.read()

    original_meta = frame.dtype.name, frame.shape

    encrypted_meta = EncodeAES(cipher, bytes(original_meta))
    encrypted_data = EncodeAES(cipher, frame.tobytes())

    # Here is the separation

    decrypted_meta = DecodeAES(cipher, encrypted_meta)
    decrypted_data = DecodeAES(cipher, encrypted_data)

    meta = ast.literal_eval(decrypted_meta)

    img = np.frombuffer(decrypted_data, dtype=np.dtype(meta[0])).reshape(meta[1])

    cv2.imshow("Capturing", img)
    key = cv2.waitKey(1)

    if key == ord("q"):
        break