CODEWITHSUNDEEP
dockerhbasetestcontainers
Nick Allen
Nick Allen

Reputation: 1467

HBase + TestContainers - Port Remapping

I am trying to use Test Containers to run an integration test against HBase launched in a Docker container. The problem I am running into may be a bit unique to how a client interacts with HBase.

When the HBase Master starts in the container, it stores its hostname:port in Zookeeper so that clients can find it. In this case, it stores "localhost:16000".

In my test case running outside the container, the client retrieves "localhost:16000" from Zookeeper and cannot connect. The connection fails because the port has been remapped by TestContainers to some other random port, other than 16000.

Any ideas how to overcome this?

(1) One idea is to find a way to tell the HBase Client to use the remapped port, ignoring the value it retrieved from Zookeeper, but I have yet to find a way to do this.

(2) If I could get the HBase Master to write the externally accessible host:port in Zookeeper that would also fix the problem. But I do not believe the container itself has any knowledge about how Test Containers is doing the port remapping.

(3) Perhaps there is a different solution that Test Containers provides for this sort of situation?

Upvotes: 0

Views: 1108

Answers (2)

user1853984
user1853984

Reputation: 369

we build a new image of hbase to be compliant with test container.

Use this image:

docker run --env HBASE_MASTER_PORT=16000 --env HBASE_REGION_PORT=16020 jcjabouille/hbase-standalone:2.4.9

Then create this Container (in scala here)

private[test] class GenericHbase2Container
    extends GenericContainer[GenericHbase2Container](
      DockerImageName.parse("jcjabouille/hbase-standalone:2.4.9")
    ) {
  private val randomMasterPort: Int = FreePortFinder.findFreeLocalPort(18000)
  private val randomRegionPort: Int = FreePortFinder.findFreeLocalPort(20000)
  private val hostName: String = InetAddress.getLocalHost.getHostName

  val hbase2Configuration: Configuration = HBaseConfiguration.create
  addExposedPort(randomMasterPort)
  addExposedPort(randomRegionPort)
  addExposedPort(2181)
  withCreateContainerCmdModifier { cmd: CreateContainerCmd =>
    cmd.withHostName(hostName)
    ()
  }
  waitingFor(Wait.forLogMessage(".*0 row.*", 1))
  withStartupTimeout(Duration.ofMinutes(10))
  withEnv("HBASE_MASTER_PORT", randomMasterPort.toString)
  withEnv("HBASE_REGION_PORT", randomRegionPort.toString)
  setPortBindings(Seq(s"$randomMasterPort:$randomMasterPort", s"$randomRegionPort:$randomRegionPort").asJava)

  override protected def doStart(): Unit = {
    super.doStart()
    hbase2Configuration.set("hbase.client.pause", "200")
    hbase2Configuration.set("hbase.client.retries.number", "10")
    hbase2Configuration.set("hbase.rpc.timeout", "3000")
    hbase2Configuration.set("hbase.client.operation.timeout", "3000")
    hbase2Configuration.set("hbase.client.scanner.timeout.period", "10000")
    hbase2Configuration.set("zookeeper.session.timeout", "10000")
    hbase2Configuration.set("hbase.zookeeper.quorum", "localhost")
    hbase2Configuration.set("hbase.zookeeper.property.clientPort", getMappedPort(2181).toString)
  }
}

More details here: https://hub.docker.com/r/jcjabouille/hbase-standalone

Upvotes: 2

bsideup
bsideup

Reputation: 3073

You can take a look at KafkaContainer's implementation where we start a Socat (fast tcp proxy) container first to acquire a semi-random port and use it later to configure the target container.

The algorithm is:

  1. In doStart, first start Socat targetting the original container's network alias & port like 12345
  2. Get mapped port (it will be something like 32109 pointing to 12345)
  3. Make the original container (e.g. with environment variables) use the mapped port in addition to the original one, or, if only one port can be configured, see CouchbaseContainer for the more advanced option
  4. Return Socat's host & port to the client

Upvotes: 2

Related Questions