Google Cloud Platform: connecting to Bigtable

Question

I'm using Google Cloud Platform with Apache Beam 2.1.0. I want to connect it to Cloud Bigtable, i'm following the official documentation in Apache Beam but i'm getting this error:

java.lang.NoSuchMethodError: io.grpc.internal.GrpcUtil.getThreadFactory(Ljava/lang/String;Z)Ljava/util/concurrent/ThreadFactory;
    at com.google.cloud.bigtable.grpc.BigtableSession.performWarmup(BigtableSession.java:135)
    at com.google.cloud.bigtable.grpc.BigtableSession.<clinit>(BigtableSession.java:100)
    at org.apache.beam.sdk.io.gcp.bigtable.BigtableServiceImpl.tableExists(BigtableServiceImpl.java:77)
    at org.apache.beam.sdk.io.gcp.bigtable.BigtableIO$Read.validate(BigtableIO.java:336)
    at org.apache.beam.sdk.Pipeline$ValidateVisitor.enterCompositeTransform(Pipeline.java:610)
    at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:590)
    at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:594)
    at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$500(TransformHierarchy.java:276)
    at org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:210)

Answer 1

I have resolved this problem by exclusing grpc from apache beam and adding it manually in pom.xml.

Now I'm getting this problem.

    java.lang.VerifyError: Bad type on operand stack
Exception Details:
  Location:
    com/google/cloud/bigtable/hbase/adapters/IncrementAdapter.adapt(Lorg/apache/hadoop/hbase/client/Increment;)Lcom/google/bigtable/v2/ReadModifyWriteRowRequest$Builder; @97: invokestatic
  Reason:
    Type 'org/apache/hadoop/hbase/client/Increment' (current frame, stack[0]) is not assignable to 'org/apache/hadoop/hbase/client/Mutation'
  Current Frame:
    bci: @97
    flags: { }
    locals: { 'com/google/cloud/bigtable/hbase/adapters/IncrementAdapter', 'org/apache/hadoop/hbase/client/Increment', 'com/google/bigtable/v2/ReadModifyWriteRowRequest$Builder', 'java/util/Iterator', 'java/util/Map$Entry', 'java/lang/String' }
    stack: { 'org/apache/hadoop/hbase/client/Increment', '[B' }
  Bytecode:
    0x0000000: 2bb6 0002 b600 039a 000d bb00 0459 1205
    0x0000010: b700 06bf b800 074d 2c2b b600 08b8 0009
    0x0000020: b600 0a57 2bb6 000b b900 0c01 00b9 000d
    0x0000030: 0100 4e2d b900 0e01 0099 009c 2db9 000f
    0x0000040: 0100 c000 103a 0419 04b9 0011 0100 c000
    0x0000050: 12b8 0013 3a05 2b19 04b9 0011 0100 c000
    0x0000060: 12b8 0014 3a06 1906 b900 1501 003a 0719
    0x0000070: 07b9 000e 0100 9900 5c19 07b9 000f 0100
    0x0000080: c000 163a 082c b600 173a 0919 0919 08b9
    0x0000090: 0018 0100 1908 b900 1901 0019 08b9 001a
    0x00000a0: 0100 b800 1bb6 001c 5719 0919 05b6 001d
    0x00000b0: 5719 0919 08b9 001e 0100 1908 b900 1f01
    0x00000c0: 0019 08b9 0020 0100 b800 21b6 0022 57a7
    0x00000d0: ffa0 a7ff 612c b0                      
  Stackmap Table:
    same_frame(@20)
    append_frame(@51,Object[#72],Object[#73])
    full_frame(@111,{Object[#74],Object[#75],Object[#72],Object[#73],Object[#76],Object[#77],Object[#78],Object[#73]},{})
    full_frame(@210,{Object[#74],Object[#75],Object[#72],Object[#73]},{})
    chop_frame(@213,1)

    at com.google.cloud.bigtable.hbase.adapters.Adapters.<clinit>(Adapters.java:46)
    at com.google.cloud.bigtable.beam.CloudBigtableScanConfiguration$Builder.build(CloudBigtableScanConfiguration.java:157)
    at com.orange.pipelines.simplification.Simplification.run(Simplification.java:219)

Answer 2

i delete all jar in WEB-INF > lib and it's works

Answer 3

It looks like you have a dependency issue. It would make it easier to debug this if you could post a list of your dependencies from:

mvn dependency:tree or gradle dependencies

I'm guessing one of your other dependencies is transitively pulling in an older version of grpc-core or grpc-all on your classpath ( < 1.0 ). Apache beam depends on 1.2.0. Please make sure that you are using grpc > 1.2.0 and don't have grpc-all on your classpath.