An error after adding a library to the Android project (More than one file was found with OS independent path 'META-INF/kotlinx-io.kotlin_module')

Question

I've been playing with the "Hello World" project created in Android Studio 3.3 (New Empty Project) and decided to add a library (ktor) to the project. I followed the README instructions and simply added the required things to the gradle file. As I worked with the websockets, the following lines have been added to the app's gradle build file:

    implementation "io.ktor:ktor-client-websocket:$ktor_version"
    implementation "io.ktor:ktor-client-cio:$ktor_version"
    implementation "io.ktor:ktor-client-android:$ktor_version"
    implementation "io.ktor:ktor-client-cio:$ktor_version"

Right after adding these the project stopped to compile and I received the following error:

More than one file was found with OS independent path 'META-INF/kotlinx-io.kotlin_module'

Could someone please explain why do I get this error? I did not do anything illegal, I just added library to the simple, standard empty project created in Android Studio and somehow after adding a new dependency, things stopped to work well.

I found some similar questions on StackOverflow and GitHub Issues, but I did not find a good explanation why does it happen and how to prevent such errors. I just found a bunch of quick and short answers / solutions, written in a manner "Add this to your project configuration and it'll work, trust me, I'm an expert", but I'm not satisfied with it as I think that as a developer I must understand what am I doing and why it works this way and IMO it's not professional just to add things the one does not fully understand, hoping that it'll solve the issue.

So, I tried my best to understand the issue. As far as I understood based on other answers, the issue arises from the fact that I somehow ended up having 2 instances of one library inside my project (?), and so Gradle cannot understand which one to pick. But then I have cannot understand how? (I did not add any other "external dependencies")

As for the solution, the most common answer was adding the 'META-INF/kotlinx-io.kotlin_module' to the excludePath packaging options, but I don't understand why is it supposed to be correct. For me it sounds like I'm telling the build system "please exclude these paths from my project / don't scan them for libraries". But I'm not sure if it's a correct approach, because in this case I effectively exclude several libraries / dependencies from my project which might lead to the runtime exceptions NoClassDefFoundError or something similar in future.

The second most common answer was adding the pickFirst 'META-INF/kotlinx-io.kotlin_module' to the packaging options, which looks a bit better as it tells the build system "when you have multiple entries of some library, please take the first one you've found", i.e. the library will be included into my project and so I'm on the safe side, but still I have a concern about this solution as a person coming from C/C++/Rust/System_Programming world: in terms of C (simplified) let's imagine that I ended up having 2 libraries of 2 different versions (1.1, 1.2) and I have a header file which expects that I'm linked against v.1.2 of the library, if I picked up a wrong version library (1.1), but use the header from the newer one and expect that it works, well... it obviously won't work and I assume that the same issue might happen on Android (but I'm not sure about that as I'm not an expert), i.e. the pickFirst seems to be a good solution as long as we can guarantee that "multiple entries" of the library we're using, have the same version.

So I tried to use a pickFirst approach for now and ended up having an additional section in my gradle build file:

    packagingOptions {
        pickFirst 'META-INF/kotlinx-io.kotlin_module'
        pickFirst 'META-INF/atomicfu.kotlin_module'
        pickFirst 'META-INF/kotlinx-coroutines-io.kotlin_module'
    }

which did the job, but I'm still not sure if it's the best solution. I think the proper approach would be to understand why the error occurs and combat the root cause of it instead of trying to workaround it with additional packaging options.

Answer 1

Android does not require any of these files in the META-INF directory of the APK package.

and these duplicate files in the META-INF directory originate from referenced libraries -

there is nothing else to do about, except to use packagingOptions.

also see: What is the purpose of META-INF? (Android APK != Java JAR).

these are indeed similar package formats, but still not exactly the same.