Google Drive API doesn't play well with ProGuard (NPE)

Question

Currently, I'm having experience that, a piece of code, which makes use of Google Drive API is running fine without introducing ProGuard.

However, after introducing ProGuard, I'm getting the following run-time error.

    at java.lang.Thread.run(Thread.java:856)
Caused by: java.lang.NullPointerException
    at com.google.api.client.util.Types.getActualParameterAtPosition(Types.java:329)
    at com.google.api.client.util.Types.getIterableParameter(Types.java:309)
    at com.google.api.client.json.JsonParser.parseValue(JsonParser.java:546)
    at com.google.api.client.json.JsonParser.parse(JsonParser.java:350)
    at com.google.api.client.json.JsonParser.parseValue(JsonParser.java:586)
    at com.google.api.client.json.JsonParser.parse(JsonParser.java:289)
    at com.google.api.client.json.JsonObjectParser.parseAndClose(JsonObjectParser.java:76)
    at com.google.api.client.json.JsonObjectParser.parseAndClose(JsonObjectParser.java:71)
    at com.google.api.client.http.HttpResponse.parseAs(HttpResponse.java:491)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:456)
    at com.jstock.c.b.a(CloudFile.java:136)

Note, the crash happens at my code (which is com.jstock.c.b.a if I retrace using mapping.txt)

// request is Files.List
FileList files = request.execute();

In my proguard, I thought having the following 2 key instructions, able to prevent the crash from happen : I tell ProGuard never touch on jackson and Google libraries.

-keep class org.codehaus.** { *; }
-keep class com.google.** { *; }
-keep interface org.codehaus.** { *; }
-keep interface com.google.** { *; }

But that doesn't work. NPE still happen at Types.java

Note that, I had another try is that, I thought obfuscate process causes NPE happens. Hence, I try to disable it using -dontobfuscate. But this time, I will not able to generate APK file, and getting a popular error message : Conversion to Dalvik format failed with error 1

Here is the proguard configuration which causes NPE at Google Drive API.

-optimizationpasses 1
-dontusemixedcaseclassnames
-dontskipnonpubliclibraryclasses
-dontpreverify
-verbose
-optimizations !code/simplification/arithmetic,!field/*,!class/merging/*

# Comment out the following line, will cause popular "Conversion to Dalvik format failed with error 1"
##-dontobfuscate

-dontwarn sun.misc.Unsafe
-dontwarn com.google.common.collect.MinMaxPriorityQueue
-dontwarn javax.swing.**
-dontwarn java.awt.**
-dontwarn org.jasypt.encryption.pbe.**
-dontwarn java.beans.**
-dontwarn org.joda.time.**
-dontwarn com.google.android.gms.**
-dontwarn org.w3c.dom.bootstrap.**
-dontwarn com.ibm.icu.text.**
-dontwarn demo.**

# Hold onto the mapping.text file, it can be used to unobfuscate stack traces in the developer console using the retrace tool
-printmapping mapping.txt

# Keep line numbers so they appear in the stack trace of the develeper console 
-keepattributes *Annotation*,EnclosingMethod,SourceFile,LineNumberTable

-keep public class * extends android.app.Activity
-keep public class * extends android.app.Application
-keep public class * extends android.app.Service
-keep public class * extends android.content.BroadcastReceiver
-keep public class * extends android.content.ContentProvider
-keep public class * extends android.app.backup.BackupAgentHelper
-keep public class * extends android.preference.Preference
-keep public class com.android.vending.licensing.ILicensingService

-keep class android.support.v4.app.** { *; }
-keep interface android.support.v4.app.** { *; }
-keep class com.actionbarsherlock.** { *; }
-keep interface com.actionbarsherlock.** { *; }

# https://sourceforge.net/p/proguard/discussion/182456/thread/e4d73acf
-keep class org.codehaus.** { *; }
-keep class com.google.** { *; }
-keep interface org.codehaus.** { *; }
-keep interface com.google.** { *; }

-assumenosideeffects class android.util.Log {
  public static int d(...);
  public static int i(...);
  public static int e(...);
  public static int v(...);  
}

-keepclasseswithmembernames class * {
    native <methods>;
}

-keepclasseswithmembers class * {
    public <init>(android.content.Context, android.util.AttributeSet);
}

-keepclasseswithmembers class * {
    public <init>(android.content.Context, android.util.AttributeSet, int);
}

-keepclassmembers class * extends android.app.Activity {
   public void *(android.view.View);
}

-keepclassmembers enum * {
    public static **[] values();
    public static ** valueOf(java.lang.String);
}

-keep class * implements android.os.Parcelable {
  public static final android.os.Parcelable$Creator *;
}

-assumenosideeffects class android.util.Log {
    public static *** d(...);
    public static *** v(...);
    public static *** i(...);
}

-keepclasseswithmembers class com.google.common.base.internal.Finalizer{
    <methods>;
}

Is there anything else I can try?

I'm not sure it might be caused by the combination of the libraries. (Although things run pretty well without introducing ProGuard)

If I look at the NPE crash location (Types.getActualParameterAtPosition(Types.java:329))

private static Type getActualParameterAtPosition(Type type, Class<?> superClass, int position) {
    ParameterizedType parameterizedType = Types.getSuperParameterizedType(type, superClass);
    Type valueType = parameterizedType.getActualTypeArguments()[position];
    // this is normally a type variable, except in the case where the class of iterableType is
    // superClass, e.g. Iterable<String>
    if (valueType instanceof TypeVariable<?>) {
      Type resolve = Types.resolveTypeVariable(Arrays.asList(type), (TypeVariable<?>) valueType);
      if (resolve != null) {
        return resolve;
      }
    }
    return valueType;
}

I suspect Types.getSuperParameterizedType returning null. So, I further look into Types.getSuperParameterizedType.

public static ParameterizedType getSuperParameterizedType(Type type, Class<?> superClass) {
    if (type instanceof Class<?> || type instanceof ParameterizedType) {
    outer: while (type != null && type != Object.class) {
     Class<?> rawType;
     if (type instanceof Class<?>) {
       // type is a class
       rawType = (Class<?>) type;
     } else {
       // current is a parameterized type
       ParameterizedType parameterizedType = (ParameterizedType) type;
       rawType = getRawClass(parameterizedType);
       // check if found Collection
       if (rawType == superClass) {
         // return the actual collection parameter
         return parameterizedType;
       }
       if (superClass.isInterface()) {
         for (Type interfaceType : rawType.getGenericInterfaces()) {
           // interface type is class or parameterized type
           Class<?> interfaceClass =
               interfaceType instanceof Class<?> ? (Class<?>) interfaceType : getRawClass(
                   (ParameterizedType) interfaceType);
           if (superClass.isAssignableFrom(interfaceClass)) {
             type = interfaceType;
             continue outer;
           }
         }
       }
     }
     // move on to the super class
     type = rawType.getGenericSuperclass();
    }
    }
    return null;
}

What is the possible root cause that may cause getSuperParameterizedType returning null, after processed by ProGuard?

Answer 1

This worked for me.

-keepattributes Signature,RuntimeVisibleAnnotations,AnnotationDefault
-keepclassmembers class * {
  @com.google.api.client.util.Key <fields>;
}
-dontwarn com.google.api.client.extensions.android.**
-dontwarn com.google.api.client.googleapis.extensions.android.**
-dontwarn com.google.android.gms.**
-dontnote java.nio.file.Files, java.nio.file.Path
-dontnote **.ILicensingService
-dontnote sun.misc.Unsafe
-dontwarn sun.misc.Unsafe
-keep class * extends com.google.api.client.json.** { *; }
-keep class * extends com.google.api.client.util.** { *; }
-keep class com.google.api.services.drive.** { *; }

Answer 2

Proper combination is :

-keepattributes Signature,RuntimeVisibleAnnotations,AnnotationDefault

There's proguard configuration prepared by Google for project google-api-java-client

https://github.com/google/google-api-java-client/blob/57fe35766cbba0a0d5a9a296be81468d730a29f8/google-api-client-assembly/proguard-google-api-client.txt

Answer 3

There have been few update to GooglePlayServices lately. I don't like the new API. I had the same problems.

I couldn't compile signed app with proguard. Proguard template from Google didn't work for me.

I add these four lines to my proguard config and it is working:

-dontwarn com.google.android.gms.**
-keep interface com.google.** { *; }
-keep class * extends com.google.api.client.json.GenericJson {*;}
-keep class com.google.api.services.drive.** {*;}

This is strange. Previous version of google-api-services-drive-v2 compiled without any problems.

I'm using the latest version at the moment: google-api-services-drive-v2-rev47-1.12.0-beta.jar

Answer 4

A combination of the following has worked for me:

-keep class com.google.** { *;}
-keep interface com.google.** { *;}
-dontwarn com.google.**

-dontwarn sun.misc.Unsafe
-dontwarn com.google.common.collect.MinMaxPriorityQueue
-keepattributes *Annotation*,Signature
-keep class * extends com.google.api.client.json.GenericJson {
*;
}
-keep class com.google.api.services.drive.** {
*;
}

This provided a working proguard compatible solution for a recent Google Drive project.

Cannot take all credit for this solution though, originally found at this link here

Answer 5

does your code use any thing that implement Serializable? All of those need to be excluded too.

Answer 6

The method Types.getSuperParameterizedType relies on information about generics. Generics are erased in Java. The compiler only adds them as annotation attributes, the JVM ignores them, and ProGuard discards them unless you tell it not to. So this might help:

-keepattributes *Annotation*

Answer 7

First -keeping a class does not mean to not touch it. It means do not change its name, and use it as a basis for determining if other classes are not referenced & can be deleted.

Optimization still occurs, which is likely your problem. Next step I would do is try with: -dontoptimize

This should cause your other optimizations to be ignored.

BTW, not sure what version of SDK you are using. Am using 15, 20 is latest, and a proguard-project.txt file is create with the project. The optimization options it uses is:

-optimizations !code/simplification/arithmetic,!code/simplification/cast,!field/*,!class/merging/*

If turning off optimization gets it running, maybe turning off all the optimizations (that's what ! does) the SDK does, will allow you to do optimization as well.