How to generate aerospike key edigest from namespace,set,userkey without aerospike lib

Question

To better understand how aerospike works I want to write a python script that generates the the key from namespace, set, and userkey.

I know that aerospike is using RIPEMD-160 hash and it looks like that for key generation it does not use namspace, but i wasn't able to write python script that creates the same key as aerospike creates.

Here is the code example that is generating key with aerospike:

#!/usr/local/bin/python

import sys
import aerospike

config = {
    'hosts': [ ('aerospike', 3000) ],
    'policies': {"write": {"key": aerospike.POLICY_KEY_SEND}}
}

try:
    client = aerospike.client(config).connect()
except:
    print("Failed to connect to the cluster with", config['hosts'])
    sys.exit(1)

key = ('test', 'demo', 'one')
print('key:', key)

try:
    client.put(key, {'bin1': 'value1'})
except Exception as e:
    print("error: {0}".format(e), file=sys.stderr)
    sys.exit(1)

(key, metadata, record) = client.get(key)
print("key from aerospike:", key)

This is the output of this script:

root@d1716cafa3fb:/app# ./a.py
key: ('test', 'demo', 'one')
key from aerospike: ('test', 'demo', None, bytearray(b'\xe4I\x9bJ\xa5r1fL\xe0\x19\xe5\xc2\x9d"Xr\xdb\xbf\x03'))
root@d1716cafa3fb:/app#

And here is what I see in aerospike client after I execute this script:

aql> set record_print_metadata true
RECORD_PRINT_METADATA = true
aql> select * from test
+-------+----------+--------------------------------+--------+---------+-------+
| PK    | bin1     | {edigest}                      | {set}  | {ttl}   | {gen} |
+-------+----------+--------------------------------+--------+---------+-------+
| "one" | "value1" | "5EmbSqVyMWZM4Bnlwp0iWHLbvwM=" | "demo" | 2591964 | 1     |
+-------+----------+--------------------------------+--------+---------+-------+
1 row in set (0.024 secs)

So for the namespace "test", set "demo" and userkey "one" the base64 of key is "5EmbSqVyMWZM4Bnlwp0iWHLbvwM=".

And here is my attempt to generate the same base64:

#!/usr/local/bin/python

from Crypto.Hash import RIPEMD
import base64

message = "demo one"

hash_obj = RIPEMD.new(data=message.encode('utf-8'))
hash_bytes = hash_obj.digest()
hash_base64 = base64.b64encode(hash_bytes).decode('utf-8')

print(f"RIPEMD-160 hash of '{message}' in base64 is: {hash_base64}")

This does not work. This output

root@d1716cafa3fb:/app# ./b.py
RIPEMD-160 hash of 'demo one' in base64 is: Lz0mUZgml2l+NLfnhwc+5RwlmEY=

How can I modify my scrip so I get "5EmbSqVyMWZM4Bnlwp0iWHLbvwM=" ?

Answer 1

I'm a bit late to the party, but I just saw this, and figured I'll add I created a tool that does that here: https://github.com/rglonek/SpikeTools

It's a simple golang application (you can check the source there), with a docker image for ease of use (no download/compile needed).

Answer 2

Since the current Python client seems to still depend on the C client, did you check how other clients do it?

For example, for the Java Client: https://github.com/aerospike/aerospike-client-java/blob/38261859940093e24c9bd04a0ba1c6d31bda2205/client/src/com/aerospike/client/util/Crypto.java#L27

    /**
 * Generate unique server hash value from set name, key type and user defined key.
 * The hash function is RIPEMD-160 (a 160 bit hash).
 */
public static byte[] computeDigest(String setName, Value key) {
    int size = Buffer.estimateSizeUtf8Quick(setName) + 1 + key.estimateKeySize();
    byte[] buffer = new byte[size];
    int setLength = Buffer.stringToUtf8(setName, buffer, 0);

    buffer[setLength] = (byte)key.getType();
    int keyLength = key.write(buffer, setLength + 1);

    RipeMD160 hash = new RipeMD160();
    hash.update(buffer, 0, setLength);
    hash.update(buffer, setLength, keyLength + 1);
    return hash.digest();

This would hopefully give enough clues for how to adapt it to Python code.