AVRO encode api crash when trying to encode a string

Question

I am new to programming and AVRO. So, please kindly bear with me. I am trying to encode and decode few strings in AVRO format. I have followed the same example as mentioned in the APACHE AVRO CPP page. reference :- https://avro.apache.org/docs/1.10.2/api/cpp/html/index.html .

This is my JSON Schema.

 {
     "type": "record",
     "name": "cpx",
     "fields" : [
         {"name": "re", "type": "string"},
         {"name": "im", "type" : "string"}
     ]
 }

This is my generated header file.

namespace c {
struct cpx {
    std::string re;
    std::string im;
    cpx() :
        re(std::string()),
        im(std::string())
        { }
};

}
namespace avro {
template<> struct codec_traits<c::cpx> {
    static void encode(Encoder& e, const c::cpx& v) {
        avro::encode(e, v.re);
        avro::encode(e, v.im);
    }
    static void decode(Decoder& d, c::cpx& v) {
        if (avro::ResolvingDecoder *rd =
            dynamic_cast<avro::ResolvingDecoder *>(&d)) {
            const std::vector<size_t> fo = rd->fieldOrder();
            for (std::vector<size_t>::const_iterator it = fo.begin();
                it != fo.end(); ++it) {
                switch (*it) {
                case 0:
                    avro::decode(d, v.re);
                    break;
                case 1:
                    avro::decode(d, v.im);
                    break;
                default:
                    break;
                }
            }
        } else {
            avro::decode(d, v.re);
            avro::decode(d, v.im);
        }
    }
};

}

This is my CPP source file.

#include <stdlib.h>
#include "bla.hh"
#include "avro/Encoder.hh"
#include "avro/Decoder.hh"


 int main()
 {
     std::unique_ptr<avro::OutputStream> out = avro::memoryOutputStream();
     avro::EncoderPtr e = avro::binaryEncoder();
     e->init(*out);
     c::cpx c1;
     c1.re = "hello";
     c1.im = "thanks";
     avro::encode(*e, c1);

     std::unique_ptr<avro::InputStream> in = avro::memoryInputStream(*out);
     avro::DecoderPtr d = avro::binaryDecoder();
     d->init(*in);

     c::cpx c2;
     avro::decode(*d, c2);
     std::cout << '(' << c2.re << ", " << c2.im << ')' << std::endl;
     std::cout << "DECODED" << std::endl;
     return 0;
 }

this is how I compiled my binary

g++ test.cpp -std=c++11 -lavrocpp

And when I run this, I got a crash and this is my core dump.

[root@2-9 bin_kw]# gdb ./a.out /data/storage/corefiles/core.a.out

[root@2-9 bin_kw]# gdb

./a.out /data/storage/corefiles/core.a.out.30222.2-9.com.1658156296 GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-115.el7 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/... Reading symbols from /root/Nightly_Suite_Workspace/MS/CU_CP/bin_kw/a.out...(no debugging symbols found)...done. [New LWP 30222] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `./a.out'. Program

terminated with signal 11, Segmentation fault.

#0 __memcpy_ssse3_back () at ../sysdeps/x86_64/multiarch/memcpy-ssse3-back.S:896 896
movaps 0x37(%rsi), %xmm5 warning: File "/usr/local/lib64/libstdc++.so.6.0.21-gdb.py" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load:/usr/bin/mono-gdb.py". To enable execution of this file add add-auto-load-safe-path /usr/local/lib64/libstdc++.so.6.0.21-gdb.py line to your configuration file "/root/.gdbinit". To completely disable this security protection add set auto-load safe-path / line to your configuration file "/root/.gdbinit". For more information about this security protection see the "Auto-loading safe path" section in the GDB manual. E.g., run from the shell: info "(gdb)Auto-loading safe path" Missing separate debuginfos, use: debuginfo-install boost-filesystem-1.53.0-28.el7.x86_64 boost-iostreams-1.53.0-28.el7.x86_64 boost-program-options-1.53.0-28.el7.x86_64 boost-regex-1.53.0-28.el7.x86_64 boost-system-1.53.0-28.el7.x86_64 bzip2-libs-1.0.6-13.el7.x86_64 libicu-50.2-3.el7.x86_64

(gdb) bt

#0 __memcpy_ssse3_back () at ../sysdeps/x86_64/multiarch/memcpy-ssse3-back.S:896 #1 0x00007f28e9dad9e1 in avro::BinaryEncoder::encodeString(std::string const&) () from /usr/local/lib/libavrocpp.so.1.11.0 #2 0x0000000000401778 in avro::codec_traits<std::__cxx11::basic_string<char, std::char_traits, std::allocator >

::encode(avro::Encoder&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) () #3 0x0000000000401a6e in void avro::encode<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >(avro::Encoder&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) () #4 0x0000000000401830 in avro::codec_traits<c::cpx>::encode(avro::Encoder&, c::cpx const&) () #5 0x0000000000401ddc in void avro::encode<c::cpx>(avro::Encoder&, c::cpx const&) () #6 0x00000000004014ac in main ()**

I tried to figure it out , but I couldn't get any clue. Please help me, Thanks in Advance. This is the page I was referring to.

Answer 1

I installed the library from the link and built the code, mostly unmodified:

#include <string>
//#include "bla.hh"
#include "api/Compiler.hh"
#include "api/DataFile.hh"
#include "api/Decoder.hh"
#include "api/Encoder.hh"
#include "api/Generic.hh"
#include "api/Stream.hh"

namespace c {
    struct cpx {
        std::string re;
        std::string im;
        cpx() :
            re(std::string()),
            im(std::string())
        { }
    };
}

namespace avro {
template<>
struct codec_traits<c::cpx>
{
    static void encode(Encoder& e, const c::cpx& v)
    {
        avro::encode(e, v.re);
        avro::encode(e, v.im);
    }
    static void decode(Decoder& d, c::cpx& v)
    {
        if(avro::ResolvingDecoder* rd =
               dynamic_cast<avro::ResolvingDecoder*>(&d))
        {
            const std::vector<size_t> fo = rd->fieldOrder();
            for(std::vector<size_t>::const_iterator it = fo.begin();
                it != fo.end();
                ++it)
            {
                switch(*it)
                {
                case 0: avro::decode(d, v.re); break;
                case 1: avro::decode(d, v.im); break;
                default: break;
                }
            }
        } else
        {
            avro::decode(d, v.re);
            avro::decode(d, v.im);
        }
    }
};
} // namespace avro

#include <stdlib.h>

int main()
{
    std::unique_ptr<avro::OutputStream> out =
        avro::memoryOutputStream();
    avro::EncoderPtr e = avro::binaryEncoder();
    e->init(*out);
    c::cpx c1;
    c1.re = "hello";
    c1.im = "thanks";
    avro::encode(*e, c1);

    std::unique_ptr<avro::InputStream> in =
        avro::memoryInputStream(*out);
    avro::DecoderPtr d = avro::binaryDecoder();
    d->init(*in);

    c::cpx c2;
    avro::decode(*d, c2);
    std::cout << '(' << c2.re << ", " << c2.im << ')' << std::endl;
    std::cout << "DECODED" << std::endl;
    return 0;
}

It runs seemingly fine and prints

(hello, thanks)
DECODED

I even enabled -fsanitize=address,undefined for the entire project (including inside avro/lang/c++/CMakeLists.txt) and it still runs cleanly.

You might try by linking statically:

g++ test.cpp -std=c++11 -lavrocpp_s

If that does work then in all likelihood, when building you're finding a different version of avrocpp than is installed and found at runtime. You can also verify this by using

ldd a.out

and see what shared library is resolved at runtime for libavrocpp.so.