C++ code compiles, but when run fails to find library function (1-character difference in function name)

Question

I'm writing a Python extension module wrapping a C++ library. In my module I call the function Rule.set_filter in the library (code), and this compiles, but when I run it it fails with ImportError: dlopen(/Users/larsga/cvs-co/fhdb/reports/pymapnik3.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace. This is on MacOS, as you can see.

The symbol that's not found is quite long:

__ZN6mapnik4rule10set_filterERKNSt3__110shared_ptrIN6mapbox4util7variantIJNS_10value_nullEbidN6icu_7113UnicodeStringENS_9attributeENS_16global_attributeENS_23geometry_type_attributeENS4_17recursive_wrapperINS_10unary_nodeINS_4tags6negateEEEEENSC_INS_11binary_nodeINSE_4plusEEEEENSC_INSI_INSE_5minusEEEEENSC_INSI_INSE_4multEEEEENSC_INSI_INSE_3divEEEEENSC_INSI_INSE_3modEEEEENSC_INSI_INSE_4lessEEEEENSC_INSI_INSE_10less_equalEEEEENSC_INSI_INSE_7greaterEEEEENSC_INSI_INSE_13greater_equalEEEEENSC_INSI_INSE_8equal_toEEEEENSC_INSI_INSE_12not_equal_toEEEEENSC_INSD_INSE_11logical_notEEEEENSC_INSI_INSE_11logical_andEEEEENSC_INSI_INSE_10logical_orEEEEENSC_INS_16regex_match_nodeEEENSC_INS_18regex_replace_nodeEEENSC_INS_19unary_function_callEEENSC_INS_20binary_function_callEEEEEEEE

Demangled:

_mapnik::rule::set_filter(std::__1::shared_ptr<mapbox::util::variant<mapnik::value_null, bool, int, double, icu_71::UnicodeString, mapnik::attribute, mapnik::global_attribute, mapnik::geometry_type_attribute, mapbox::util::recursive_wrapper<mapnik::unary_node<mapnik::tags::negate> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::plus> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::minus> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::mult> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::div> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::mod> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::less> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::less_equal> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::greater> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::greater_equal> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::equal_to> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::not_equal_to> >, mapbox::util::recursive_wrapper<mapnik::unary_node<mapnik::tags::logical_not> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::logical_and> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::logical_or> >, mapbox::util::recursive_wrapper<mapnik::regex_match_node>, mapbox::util::recursive_wrapper<mapnik::regex_replace_node>, mapbox::util::recursive_wrapper<mapnik::unary_function_call>, mapbox::util::recursive_wrapper<mapnik::binary_function_call> > > const&)

When I look at the mapnik dylib I find that a very similar function is defined there:

__ZN6mapnik4rule10set_filterERKNSt3__110shared_ptrIN6mapbox4util7variantIJNS_10value_nullEbxdN6icu_7113UnicodeStringENS_9attributeENS_16global_attributeENS_23geometry_type_attributeENS4_17recursive_wrapperINS_10unary_nodeINS_4tags6negateEEEEENSC_INS_11binary_nodeINSE_4plusEEEEENSC_INSI_INSE_5minusEEEEENSC_INSI_INSE_4multEEEEENSC_INSI_INSE_3divEEEEENSC_INSI_INSE_3modEEEEENSC_INSI_INSE_4lessEEEEENSC_INSI_INSE_10less_equalEEEEENSC_INSI_INSE_7greaterEEEEENSC_INSI_INSE_13greater_equalEEEEENSC_INSI_INSE_8equal_toEEEEENSC_INSI_INSE_12not_equal_toEEEEENSC_INSD_INSE_11logical_notEEEEENSC_INSI_INSE_11logical_andEEEEENSC_INSI_INSE_10logical_orEEEEENSC_INS_16regex_match_nodeEEENSC_INS_18regex_replace_nodeEEENSC_INS_19unary_function_callEEENSC_INS_20binary_function_callEEEEEEEE

Demangled:

_mapnik::rule::set_filter(std::__1::shared_ptr<mapbox::util::variant<mapnik::value_null, bool, long long, double, icu_71::UnicodeString, mapnik::attribute, mapnik::global_attribute, mapnik::geometry_type_attribute, mapbox::util::recursive_wrapper<mapnik::unary_node<mapnik::tags::negate> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::plus> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::minus> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::mult> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::div> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::mod> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::less> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::less_equal> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::greater> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::greater_equal> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::equal_to> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::not_equal_to> >, mapbox::util::recursive_wrapper<mapnik::unary_node<mapnik::tags::logical_not> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::logical_and> >, mapbox::util::recursive_wrapper<mapnik::binary_node<mapnik::tags::logical_or> >, mapbox::util::recursive_wrapper<mapnik::regex_match_node>, mapbox::util::recursive_wrapper<mapnik::regex_replace_node>, mapbox::util::recursive_wrapper<mapnik::unary_function_call>, mapbox::util::recursive_wrapper<mapnik::binary_function_call> > > const&)

It's not the same, though. Character 91 (0-based) is i in my code, but x in the mapnik library.

The piece that's different is 10value_nullEbidN6icu in my library and 10value_nullEbxdN6icu in mapnik. See the Ebid vs Ebxd.

Now, my question is why is there a one-character difference here? Somehow I must be compiling these two codebases slightly differently, but I have no idea how. Mapnik is build with SConsm, while the Python extension is built with distutils.

Mapnik is built with make, then installed at /usr/local/lib/libmapnik.dylib. The extension module is built with python3 setup.py build. I used DYLD_PRINT_LIBRARIES=YES to verify which mapnik library it's loading.

The compiler used is clang, if that helps.

Answer 1

Using the tip from @mat that i means int and x means long long I was able to figure it out.

The argument to set_filter is of type std::shared_ptr<expr_node>. (Code.) expr_node turns out to be a util::variant type mixing a whole lot of types. Those are the ones listed in the missing symbol.

The position of the i/x in the mangled name is pretty close to where value_integer appears in the list of alternative types.

Digging further I found that value_integer is 32-bit if BIGINT is not defined, and 64-bit if it is. (Code.)

So I figured out that mapnik must have been compiled with BIGINT, whereas my module was compiled without. By adding -DBIGINT to extra_compile_args in my setup.py I was able to solve the problem. (Thanks @mat!)