CODEWITHSUNDEEP
pythonserializationdeserialization
prosseek
prosseek

Reputation: 191069

python encode/decoder for serialization/deserialization (Java's kyro equivalence in python)

I need to convert python value into byte array and vice versa.

For example:

  1. Integer 256 -> [1, 1] // 256 = 1 * 255 + 1
  2. String 'ab' -> [97, 98] // 'a' -> 97, 'b' -> 98
  3. Floating point 1.23 -> [63, 157, 112, 164] // 1.23 as 4 byte representation

For java kryo can be used for this purpose: byte[] serializedValue = kryoSerializer.writeObjectData(value); gives me the serialized result of value.

I tried pickle, but I can't use it as it consumes 6 bytes for storing an integer object.

import pickle

Foo = 256
picklestring = pickle.dumps(Foo)
print len(picklestring) # returns 6

Any hints for this?

ADDED

# http://docs.python.org/2/library/struct.html
# http://stackoverflow.com/questions/16818463/python-encode-decoder-for-serialization-deserialization-javas-kyro-equivalence
# http://stackoverflow.com/questions/11624190/python-convert-string-to-byte-array
import struct

# >f for 
def encode(value):
    formatString = ""
    if type(value) is float:
        formatString = ">f"
    elif type(value) is int:
        formatString = ">i"
    elif type(value) is str:
        formatString = ">s"
    else:
        raise Exception("Wrong data input: only supports float/int/string")
    packed = struct.pack(formatString, value)
    result = []
    for i in packed:
        # i is a string
        result.append(ord(i[0]))
    return result

Upvotes: 1

Views: 1610

Answers (1)

Martijn Pieters
Martijn Pieters

Reputation: 1123860

Use the struct module:

>>> import struct
>>> struct.pack('>f', 1.23)
'?\x9dp\xa4'
>>> len(struct.pack('>f', 1.23))
4

Struct packs values following C conventions; the above format packs one single-precision float value (4 bytes) in big-endian order.

Upvotes: 1

Related Questions