Patching a Mach-O binary with LC_LOAD_DYLIB requires reboot to work

Question

I'm patching a Mach-O binary with a dylib (by appending a LC_LOAD_DYLIB command at the end of the load commands + adjusting the mach header's sizeofcmds + incrementing its ncmds) and I would expect to be able to compile over and over the dylib itself and just rerun the binary.

However, for some reason, when doing so, after the first run, the binary crashes for various reasons (e.g EXC_CRASH or EXC_BAD_ACCESS at different addresses)

If I restart the machine, everything works perfectly fine(the lib is patched and the exec runs). I bet this has to do with some obscure (for me at least) memory mapping or caching the dynamic linker does, but my expertise in that area is slim. Any chance you guys know something about it?

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libobjc.A.dylib                 0x00007fffd43d3829 map_images_nolock + 588
1   libobjc.A.dylib                 0x00007fffd43d3521 map_2_images + 43
2   dyld                            0x00000001164e5d4c dyld::notifyBatchPartial(dyld_image_states, bool, char const* (*)(dyld_image_states, unsigned int, dyld_image_info const*), bool, bool) + 1124
3   dyld                            0x00000001164e5f25 dyld::registerObjCNotifiers(void (*)(unsigned int, char const* const*, mach_header const* const*), void (*)(char const*, mach_header const*), void (*)(char const*, mach_header const*)) + 50
4   libdyld.dylib                   0x00007fffd4cbd584 _dyld_objc_notify_register + 77
5   libobjc.A.dylib                 0x00007fffd43d3074 _objc_init + 115
6   libdispatch.dylib               0x00007fffd4c84c64 _os_object_init + 13
7   libdispatch.dylib               0x00007fffd4c84c13 libdispatch_init + 295
8   libSystem.B.dylib               0x00007fffd36f1a02 libSystem_initializer + 121
9   dyld                            0x00000001164f6063 ImageLoaderMachO::doModInitFunctions(ImageLoader::LinkContext const&) + 385
10  dyld                            0x00000001164f6266 ImageLoaderMachO::doInitialization(ImageLoader::LinkContext const&) + 40
11  dyld                            0x00000001164f1bf0 ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 338
12  dyld                            0x00000001164f1b87 ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 233
13  dyld                            0x00000001164f1b87 ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 233
14  dyld                            0x00000001164f1b87 ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 233
15  dyld                            0x00000001164f1b87 ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 233
16  dyld                            0x00000001164f1b87 ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 233
17  dyld                            0x00000001164f1b87 ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 233
18  dyld                            0x00000001164f1b87 ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 233
19  dyld                            0x00000001164f0c60 ImageLoader::processInitializers(ImageLoader::LinkContext const&, unsigned int, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 138
20  dyld                            0x00000001164f0cf5 ImageLoader::runInitializers(ImageLoader::LinkContext const&, ImageLoader::InitializerTimingList&) + 75
21  dyld                            0x00000001164e32f6 dyld::initializeMainExecutable() + 195
22  dyld                            0x00000001164e7459 dyld::_main(macho_header const*, unsigned long, int, char const**, char const**, char const**, unsigned long*) + 3789
23  dyld                            0x00000001164e2249 dyldbootstrap::start(macho_header const*, int, char const**, long, macho_header const*, unsigned long*) + 470
24  dyld                            0x00000001164e2036 _dyld_start + 54