How to change a boost::multiprecision::cpp_int from big endian to little endian

Question

I have a boost::multiprecision::cpp_int in big endian and have to change it to little endian. How can I do that? I tried with boost::endian::conversion but that did not work.

boost::multiprecision::cpp_int bigEndianInt("0xe35fa931a0000*);
boost::multiprecision::cpp_int littleEndianInt;

littleEndianIn = boost::endian::endian_reverse(m_cppInt);

Answer 1

This is both trivial and in the general case unanswerable, let me explain:

• For a general N-bit integer, where N is a large number, there is unlikely to be any well defined byte order, indeed even for 64 and 128 bit integers there are more than 2 possible orders in use: https://en.wikipedia.org/wiki/Endianness#Middle-endian.
• On any platform, with any native endianness you can always extract the bytes of a cpp_int, the first example here: https://www.boost.org/doc/libs/1_73_0/libs/multiprecision/doc/html/boost_multiprecision/tut/import_export.html#boost_multiprecision.tut.import_export.examples shows you how. When exporting bytes like this, they are always most significant byte first, so you can subsequently rearrange them how you wish. You should not however, rearrange them and load them back into a cpp_int as the class won't know what to do with the result!
• If you know that the value is small enough to fit into a native integer type, then you can simply cast to the native integer and use a system API on the result. As in endian_reverse(static_cast<int64_t>(my_cpp_int)). Again, don't assign the result back into a cpp_int as it requires native byte order.

• If you wish to check whether a value is small enough to fit in an N-bit integer for the approach above, you can use the msb function, which returns the index of the most significant bit in the cpp_int, add one to that to obtain the number of bits used, and filter out the zero case and the code looks like:

  unsigned bits_used = my_cpp_int.is_zero() ? 0 : msb(my_cpp_int) + 1;

Note that all of the above use completely portable code - no hacking of the underlying implementation is required.

Answer 2

The memory layout of boost multi-precision types is implementation detail. So you cannot assume much about it anyways (they're not supposed to be bitwise serializable).

Just read a random section of the docs:

MinBits

Determines the number of Bits to store directly within the object before resorting to dynamic memory allocation. When zero, this field is determined automatically based on how many bits can be stored in union with the dynamic storage header: setting a larger value may improve performance as larger integer values will be stored internally before memory allocation is required.

It's not immediately clear that you have any chance at some level of "normal int behaviour" in memory layout. The only exception would be when MinBits==MaxBits.

Indeed, we can static_assert that the size of cpp_int with such backend configs match the corresponding byte-sizes.

It turns out that there's even a promising tag in the backend base-class to indicate "triviality" (this is truly promising): trivial_tag, so let's use it:

Live On Coliru

#include <boost/multiprecision/cpp_int.hpp>
namespace mp = boost::multiprecision;

template <int bits> using simple_be =
    mp::cpp_int_backend<bits, bits, mp::unsigned_magnitude>;
template <int bits> using my_int =
    mp::number<simple_be<bits>, mp::et_off>;

using my_int8_t = my_int<8>;
using my_int16_t = my_int<16>;
using my_int32_t = my_int<32>;
using my_int64_t = my_int<64>;
using my_int128_t = my_int<128>;
using my_int192_t = my_int<192>;
using my_int256_t = my_int<256>;

template <typename Num>
    constexpr bool is_trivial_v = Num::backend_type::trivial_tag::value;

int main() {
    static_assert(sizeof(my_int8_t) == 1);
    static_assert(sizeof(my_int16_t) == 2);
    static_assert(sizeof(my_int32_t) == 4);
    static_assert(sizeof(my_int64_t) == 8);
    static_assert(sizeof(my_int128_t) == 16);

    static_assert(is_trivial_v<my_int8_t>);
    static_assert(is_trivial_v<my_int16_t>);
    static_assert(is_trivial_v<my_int32_t>);
    static_assert(is_trivial_v<my_int64_t>);
    static_assert(is_trivial_v<my_int128_t>);

    // however it doesn't scale
    static_assert(sizeof(my_int192_t) != 24);
    static_assert(sizeof(my_int256_t) != 32);
    static_assert(not is_trivial_v<my_int192_t>);
    static_assert(not is_trivial_v<my_int256_t>);
}

Conluding: you can have trivial int representation up to a certain point, after which you get the allocator-based dynamic-limb implementation no matter what.

• Note that using unsigned_packed instead of unsigned_magnitude representation never leads to a trivial backend implementation.

• Note that triviality might depend on compiler/platform choices (it's likely that cpp_128_t uses some builtin compiler/standard library support on GCC, e.g.)

---

Given this, you MIGHT be able to pull of what you wanted to do with hacks IF your backend configuration support triviality. Sadly I think it requires you to manually overload endian_reverse for 128 bits case, because the GCC builtins do not have __builtin_bswap128, nor does Boost Endian define things.

I'd suggest working off the information here How to make GCC generate bswap instruction for big endian store without builtins?

Final Demo (not complete)

#include <boost/multiprecision/cpp_int.hpp>
#include <boost/endian/buffers.hpp>
namespace mp = boost::multiprecision;
namespace be = boost::endian;

template <int bits> void check() {
    using T = mp::number<mp::cpp_int_backend<bits, bits, mp::unsigned_magnitude>, mp::et_off>;

    static_assert(sizeof(T) == bits/8);
    static_assert(T::backend_type::trivial_tag::value);

    be::endian_buffer<be::order::big, T, bits, be::align::no> buf;
    buf = T("0x0102030405060708090a0b0c0d0e0f00");

    std::cout << std::hex << buf.value() << "\n";
}

int main() {
    check<128>();
}

(Changing be::order::big to be::order::native obviously makes it compile. The other way to complete it would be to have an ADL accessible overload for endian_reverse for your int type.)