CODEWITHSUNDEEP
copensslendianness
RobinG
RobinG

Reputation: 135

OpenSSL - BIGNUM check issue

I use OpenSSL 1.0.2k to handle private and public key on a software, and I have some issue with a few OSes (linux-s390, aix power, HPUX IA64), on 64-bits arch only.

It's not so troublesome, but it bothers me. I get a RSA* structure from a buffer (DER data), and I just check the RSA public exponent to ensure it's odd. It works on every OSes, except the three above : it seems the RSA->e is even (and, obviously, it's not)

Here is the BIGNUM structure :

struct bignum_st {
    BN_ULONG *d;
    int top;
    int dmax;
    int neg;
    int flags;
};

When debugging, I saw that on a functional OS, d[0] == 65537, d[1] == 0. On a no functional OS, d[0] == 0, d[1] == 65537. BN_is_odd is a macro :

# define BN_is_odd(a)        (((a)->top > 0) && ((a)->d[0] & 1))

#include <stdio.h>
#include <openssl/pem.h>

int get_key(const unsigned char *buf, int len) {
    RSA *rsa = d2i_RSA_PUBKEY(NULL, &buf, len);
    if (rsa != NULL) {
        if (rsa->e != NULL) {
            printf("BN : <%s> (hex) -- <%s> (dec)\n", BN_bn2hex(rsa->e), BN_bn2dec(rsa->e));
            if (BN_is_odd(rsa->e) == 0) {
                printf("Error : RSA public exponent is even\n");
            } else {
                printf("RSA public exponent is OK.\n");
                return 0;
            }
        }
        RSA_free(rsa);
    } else {
        printf("Error : RSA is NULL\n");
    }
    return 1;
}

int main() {
    const unsigned char data[] = { 0x30, 0x81, 0x9f, 0x30, 0x0d, 0x06, 0x09, 0x2a, 0x86, 0x48, 0x86, 0xf7, 0x0d, 0x01, 0x01, 0x01, 0x05, 0x00, 0x03, 0x81, 0x8d, 0x00, 0x30, 0x81, 0x89, 0x02, 0x81, 0x81, 0x00, 0xd6, 0x70, 0x5d, 0x67, 0xf2, 0xe1, 0x34, 0x82, 0xd5, 0x2d, 0x79, 0xdd, 0x42, 0x55, 0x41, 0xaf, 0x0c, 0xc2, 0xb4, 0xb0, 0x94, 0xc6, 0xa0, 0x40, 0x54, 0x2e, 0x0f, 0xa5, 0x12, 0x3d, 0x43, 0x96, 0x13, 0x2d, 0x17, 0x50, 0xe5, 0x9a, 0x5a, 0x6e, 0x99, 0xc7, 0xd2, 0x63, 0x4c, 0xcd, 0x57, 0xcb, 0x57, 0x7e, 0x1e, 0x5f, 0x97, 0xaa, 0xbd, 0xe5, 0xc0, 0x98, 0xd9, 0x07, 0x52, 0xdc, 0x27, 0xa4, 0x19, 0xb2, 0x81, 0x5d, 0xd5, 0x03, 0x5c, 0xd2, 0xb3, 0xb8, 0x28, 0xaa, 0xd7, 0xaf, 0x02, 0x08, 0x1c, 0x6c, 0xc2, 0xa4, 0x6c, 0x41, 0xd3, 0xa6, 0xae, 0x51, 0x69, 0xb7, 0xd5, 0x79, 0xb8, 0x62, 0x68, 0x9e, 0xa9, 0x44, 0x8e, 0xbe, 0xb1, 0x2e, 0x1a, 0x3c, 0x4b, 0x21, 0x7b, 0x7d, 0x36, 0xf0, 0x97, 0x98, 0x81, 0x63, 0xa6, 0xfa, 0xf8, 0x28, 0x22, 0x72, 0xfe, 0x16, 0xa8, 0x16, 0x89, 0xbb, 0x02, 0x03, 0x01, 0x00, 0x01 }; /* A DER buffer, valid with openssl rsa -pubin -in <file> -inform DER */
    return get_key(data, sizeof data);
}

On most system, I get the output "RSA public exponent is OK". However, on some others, I got "RSA public exponent is even". The BIGNUM printed is correct (010001 -- 65537).

I understand why the BIGNUM is considered as even. But why is there a leading zero on those three OSes ? (no problem on those OSes with 32-bits, though). Any thought appreciated on that :)

Upvotes: 2

Views: 563

Answers (1)

Lorinczy Zsigmond
Lorinczy Zsigmond

Reputation: 1910

Most likely, it's because you have compiled 64-bit executable with OpenSSL-header-files meant for 32-bit.

To be precise, it is opensslconf.h which contains one the following two parts (first for 32-bit, second for 64-bit):

181  /* Only one for the following should be defined */
182  #undef SIXTY_FOUR_BIT_LONG
183  #undef SIXTY_FOUR_BIT
184  #define THIRTY_TWO_BIT
185  #endif

or

181  /* Only one for the following should be defined */
182  #define SIXTY_FOUR_BIT_LONG
183  #undef SIXTY_FOUR_BIT
184  #undef THIRTY_TWO_BIT
185  #endif

To make it compatible with both 32 and 64-bit, edit it this way:

181  /* Only one for the following should be defined */
182  #undef SIXTY_FOUR_BIT_LONG
183  #undef SIXTY_FOUR_BIT
184  #undef THIRTY_TWO_BIT
185  #if defined(__64BIT__)
186  #define SIXTY_FOUR_BIT_LONG
187  #else
188  #define THIRTY_TWO_BIT
189  #endif
190  #endif

http://lzsiga.users.sourceforge.net/aix-linking.html#Q0044

Upvotes: 1

Related Questions